Generation apparatus, generation method, and program

Abstract

This generation device that generates, from an input sequence that is an information sequence, an output sequence that is another information sequence, comprises: a generation unit that generates the output sequence from the input sequence and given constraint information, by using a sequence conversion model; and a correction unit that corrects the output sequence obtained by the generation unit so that the output sequence includes the constraint information.

Description

Technical Field

[0001] The present invention relates to the technical field of machine translation.

Background Art

[0002] When converting sentences in a certain domain to another domain (e.g., another language), machine translation with vocabulary constraints is a method that imposes the constraint of including all specified phrases. Since machine translation with vocabulary constraints can unify the translations for specific words, it is an important technology in the translation of patent / legal / technical documents that require consistency.

Prior Art Documents

Non-Patent Documents

[0003]

Non-Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] As conventional techniques for machine translation with vocabulary constraints, there are various techniques such as the technique disclosed in Non-Patent Document 1. In all of these conventional techniques, due to preprocessing of the sequence to be converted, etc., there may be a possibility that information is missing from the original sequence when the sequence is input to the machine translation model. Therefore, there is a problem that the constraints may not be fully satisfied in the output sequence. Note that such a problem can occur in general sequence conversion (e.g., summarization tasks, etc.) not limited to machine translation.

[0005] The present invention has been made in view of the above points, and aims to provide a technique that enables all given constraint vocabulary to be included in the output sequence in a sequence transformation with vocabulary constraints. [Means for solving the problem]

[0006] According to the disclosure technology, string From an input sequence that is another string A generating device that generates an output sequence, Vocabulary constraints A generation unit that generates the output sequence from the input sequence and given constraint information using a sequence transformation model, A modification unit modifies the output sequence obtained by the generation unit so that the constraint information is included in the output sequence. , equipped with, The modification unit uses the constraint information as a query, normalizes at least one of the output sequence from the generation unit and the query, searches the output sequence using the query, and replaces the portion of the output sequence that matches the query with the original constraint information corresponding to the query, which has not undergone the normalization process. A generating device is provided. [Effects of the Invention]

[0007] According to the disclosed technology, in a sequence transformation with lexical constraints, it is possible to ensure that all given constraint vocabulary are included in the output sequence. [Brief explanation of the drawing]

[0008] [Figure 1] This figure shows an example of machine translation with vocabulary constraints. [Figure 2] This figure shows an example configuration of the learning device 100 and the generating device 200. [Figure 3] This figure shows an example configuration of the generating device 200. [Figure 4] This figure shows an example configuration of the generating device 200. [Figure 5] This diagram shows the operation flow of the learning device 100. [Figure 6] This diagram shows the operation flow of the generation device 200. [Figure 7] This figure shows an example of a machine translation model. [Figure 8]It is a diagram showing the embedded representation of the input in the series conversion unit 230. [Figure 9] It is a diagram showing the processing procedure of the correction unit 250. [Figure 10] It is a diagram showing the processing procedure of the correction unit 250. [Figure 11] It is a diagram showing the base detailed settings and hyperparameters in each setting used in the experiment. [Figure 12] It is a diagram showing the evaluation results of each setting. [Figure 13] It is a diagram showing an example of the translated text of the proposed system. [Figure 14] It is a diagram showing the BLEU score when using various beam sizes. [Figure 15] It is a diagram showing an example of the hardware configuration of the device.

Mode for Carrying Out the Invention

[0009] Hereinafter, embodiments (these embodiments) of the present invention will be described with reference to the drawings. The embodiments described below are merely examples, and the embodiments to which the present invention is applied are not limited to the following embodiments.

[0010] In the embodiments described below, an example of applying the present invention to machine translation is shown. However, the present invention is applicable to any series conversion in any field as long as it is a series conversion using a series conversion model such as an encoder-decoder model. For example, the present invention can also be used for tasks such as summarization tasks, utterance generation tasks, tasks of attaching captions to images, etc.

[0011] Also, in the embodiments described below, as an example, the unit of translation is a sentence, but the unit of translation may be arbitrary.

[0012] All the generation devices 200 described below provide specific improvements over the prior art that performs constrained sequence conversion, and show the improvement of the technical field related to constrained sequence conversion.

[0013] (Regarding the issues) Before describing the configuration of this embodiment in detail, we will first describe the prior art and the problems associated with it. Note that the following description of problems is not publicly known. Furthermore, the problems described below relate to the technology of this embodiment.

[0014] As already explained, when translating a sentence from one domain to another domain (e.g., another language), a constraint is imposed that all specified words or phrases must be included. This is called lexically constrained machine translation. For reference, Figure 1 shows an example of input and output in lexically constrained machine translation.

[0015] In the example in Figure 1, the machine translation (MT Output), constraints, and constrained machine translation (Constrained MT Output) are shown for the original sentence, "A geometrical optical theory of standing waves based on ray coincidence was developed." The underlined parts indicate the constraint vocabulary.

[0016] <Task 1> As a conventional technique for vocabulary-constrained machine translation, the reference "Chen et al. (2020) proposed a method that augment the input of the translation model." discloses a method of inputting by concatenating the source language sentence and the constraint vocabulary. However, such conventional techniques could not guarantee that all constraints would be satisfied.

[0017] <Challenge 2> Non-patent document 1, "Post and Vilar (2018) proposed a grid beam search decoder. They guarantee to satisfy all constraints," discloses a vocabulary-constrained machine translation technique using grid beam search, which is an improved version of beam search that uses the output of a machine translation model's decoder to search for good translation results. While grid beam search makes it possible to satisfy constraints, it has the drawback of increasing search time and reducing translation accuracy when the constraint vocabulary is long.

[0018] In other words, the technology disclosed in Non-Patent Document 1 requires a large beam size (e.g., larger than 60) to include all constraint vocabulary in the output sequence, which increases the search time. Furthermore, the translation accuracy is low.

[0019] <Challenge 3> In conventional vocabulary-constrained machine translation, information about whitespace or multibyte characters in the input sequence may be lost during the preprocessing stage, or substrings that the machine translation model cannot handle may be present in the input sequence, resulting in those strings being replaced with special tokens.

[0020] In such cases, even using a method that guarantees lexical constraints will not result in the output sequence containing all of the given constraint vocabulary.

[0021] (Summary of the embodiment) <Outline of embodiments corresponding to problems 1 and 2> The problem with the technology disclosed in Non-Patent Document 1 is thought to stem from the fact that, because information about the constraint vocabulary is not input into the machine translation model, it is not possible to set high scores for those vocabulary words at appropriate locations.

[0022] Therefore, the generation device 200 according to this embodiment inputs a sequence formed by linking the source language sentence and the constraint vocabulary into a machine translation model, and applies grid beam search to the output from the machine translation model (the probability of each word) for that input. This shortens the search time. Furthermore, experimental results show that translation accuracy can be improved and processing speed can also be increased.

[0023] <Outline of an embodiment corresponding to Problem 3> To address the problem that constraints are not fully satisfied due to missing information in the sequence when it is input to the machine translation model, the generation device 200 according to this embodiment modifies the output sentence by referring to the notation information of the original input data before processing the input data. As a result, all given constraint phrases are included in the output sequence.

[0024] (Example of device configuration) Figure 2 shows an example configuration of the learning device 100 and the generation device 200 in this embodiment. As shown in Figure 2, the learning device 100 includes a parallel text data DB 110, an input unit 120, a vocabulary constraint generation unit 130, an input / output generation unit 140, a learning data DB 150, a model learning unit 160, and an output unit 170. Note that the parallel text data DB 110 and the learning data DB 150 may be provided outside the learning device 100.

[0025] The output unit 170 of the learning device 100 outputs a vocabulary-constrained machine translation model (hereinafter referred to as the machine translation model), which is stored in the model DB 180. The machine translation model is read from the model DB 180 by the generation device 200 and used for machine translation by the generation device 200. The "machine translation model" stored in the DB is actually data consisting of functions and weight parameters that constitute the neural network.

[0026] As shown in Figure 2, the generation device 200 includes an input unit 210, an input generation unit 220, a sequence conversion unit 230, a search unit 240, a correction unit 250, and an output unit 260.

[0027] Furthermore, the generation device 200 may be configured to include an input unit 210, an input generation unit 220, a sequence conversion unit 230, a search unit 240, and an output unit 260, without a correction unit 250, as shown in Figure 3. In Figures 2 and 3, the "sequence conversion unit 230 + search unit 240" may be referred to as the search unit.

[0028] The generation device 200 shown in Figure 3 is an example of a generation device that generates an output sequence, which is a sequence of information, from an input sequence, which is a sequence of information, and comprises: an input generation unit that generates an input sequence with constraint information based on the input sequence and constraint information; a sequence conversion unit that generates output information by inputting the input sequence with constraint information into a sequence conversion model; and a search unit that generates the output sequence by performing a constrained search using the output information so that the constraint information is included in the output sequence.

[0029] Furthermore, all (or part) of the processing in the above-described generation device may be performed by a neural network. The parameters of the neural network are pre-trained. The training for learning these parameters may be performed by the learning device 100 in the procedure shown in Figure 5.

[0030] Furthermore, as a modified generation device 200, a configuration comprising an input unit 210, a lexical-constrained sequence generation unit 270, a modification unit 250, and an output unit 260 may be adopted, as shown in Figure 4. The lexical-constrained sequence generation unit 270 may be the "input generation unit 220 + sequence conversion unit 230 + search unit 240" shown in Figure 2, or it may be a lexical-constrained machine translation model in the prior art (for example, the model disclosed in Non-Patent Literature 1), or it may be any other model. The lexical-constrained sequence generation unit may also be called the "generation unit".

[0031] Furthermore, the generation device 200 shown in Figure 4 is an example of a generation device that generates an output sequence, which is a sequence of information, from an input sequence, which is a sequence of information, and comprises a generation unit that generates the output sequence from the input sequence and given constraint information using a sequence conversion model, and a modification unit that modifies the output sequence obtained by the generation unit so that the constraint information is included in the output sequence.

[0032] The generation unit described above may be composed of a neural network, and the sequence transformation model may be pre-trained parameters of the neural network. The training for learning these parameters may be performed by the learning device 100 in the procedure shown in Figure 5.

[0033] The functions and operations of each part of the learning device 100 and the generation device 200 described above will be explained later.

[0034] (Overall flow of operations) The overall operation flow of the device configuration shown in Figure 2 will be explained with reference to Figures 5 and 6. Note that the parallel texts used below will be those that have already undergone word segmentation processing for both the source language and target language texts. However, if segmentation processing has not been performed, it may be performed in the input / output generation unit 140 or similar. Any segmentation method and unit may be used. This segmentation process may also be called tokenization.

[0035] Furthermore, the units obtained through the partitioning process may also be called tokens. Tokens may be words, further partitioned words, or characters partitioned into individual characters. Note that tokenization refers to the partitioning process itself, while a tokenizer refers to the function (software, etc.) that performs the partitioning process.

[0036] <Learning Flow> First, with reference to Figure 5, the operation flow of the learning device 100 will be explained. It is assumed that the parallel text data DB 110 already contains parallel text data. In S101, the input unit 120 reads parallel text from the parallel text data DB 110 and inputs the read parallel text to the vocabulary constraint generation unit 130 and the input / output generation unit 140.

[0037] In S102, the vocabulary constraint generation unit 130 generates constraint vocabulary from the parallel text and passes the constraint vocabulary to the input / output generation unit 140. In S103, the input / output generation unit 140 generates pairs of "extended input" and "output" from the constraint vocabulary and the parallel text, and stores the generated pairs of data in the training data DB 150 as training data for vocabulary-constrained machine translation.

[0038] In S104, the model learning unit 160 reads training data consisting of pairs of "extended inputs" and "outputs" from the training data DB 150, and uses this training data to train a vocabulary-constrained machine translation model. The trained machine translation model is stored in the model DB 180 by the output unit 170.

[0039] <Translation execution flow> Next, the flow of vocabulary-constrained machine translation execution by the generation device 200 will be explained with reference to Figure 6. Here, it is assumed that the sequence conversion unit 230 reads and stores the machine translation model from the model DB 180.

[0040] In S201, the input unit 210 receives the input sentence to be translated and the constraint vocabulary, and passes them to the input generation unit 220. In S202, the input generation unit 220 expands the input sentence (input sequence) using the constraint vocabulary.

[0041] In S203, the sequence conversion unit 230 performs sequence conversion prediction by inputting the input sequence generated in S202 into the machine translation model. The output (prediction result) of the machine translation model is passed to the search unit 240.

[0042] In S204, the search unit 240 searches for a translated sentence that satisfies the constraint vocabulary based on the prediction results from the machine translation model. The translated sentence obtained through the search is passed to the correction unit 250.

[0043] In S205, the correction unit 250 corrects the translated text and passes the corrected translated text to the output unit 260. In S206, the output unit 260 outputs the corrected translated text.

[0044] (About machine translation models) Here, the machine translation model in this embodiment will be described. In this embodiment, as shown in Figure 7, a general encoder-decoder model (e.g., Transformer) having an encoder and a decoder is used. However, the present invention can also be implemented using models other than the encoder-decoder model. Note that the machine translation model is an example of a sequence transformation model.

[0045] (Operation of each part) The operation of each part that performs processing in the device configuration shown in Figures 2 to 4 will be explained below. Only a brief explanation is given for parts that perform conventional general operations. Furthermore, the input units 120 and 210, and the output units 170 and 260 have already been explained.

[0046] <Learning device 100's lexical constraint generation unit 130> Since the parallel text data for training does not contain information on constraint vocabulary, the vocabulary constraint generation unit 130 takes the parallel text (source language sentence and target language sentence) as input and generates constraint vocabulary for that parallel text.

[0047] Specifically, for example, the lexical constraint generation unit 130 samples a number k from 0 to 14, randomly extracts k words from the tokenized target language sentence, and uses the extracted words as constraint vocabulary. Note that consecutively occurring words are treated as a single constraint.

[0048] <Input generation unit 220 of generation device 200> The input generation unit 220 receives the source language sentence and constraint vocabulary as input, and creates an input sequence with added information from the constraint vocabulary by extending the source language sentence using the constraint vocabulary. The input generation unit 220 outputs the extended input sequence to the sequence conversion unit 230.

[0049] More specifically, the input generation unit 220 first performs preprocessing. This preprocessing may also be called tokenization. In preprocessing, the input sentence and constraint vocabulary are divided into predefined arbitrary processing units.

[0050] The input generation unit 220 receives the input sequence, which is the source language sentence X, and each constraint vocabulary C. i And, as follows <sep>A sequence of input words with a constraint vocabulary is created by concatenating (linking) them together via a special delimiter string. <eos>This string indicates the end of a sentence.

[0051] [X, <sep> ,C1, <sep>,C2,…,C N , <eos>] The above method of concatenating an input sequence with a constraint vocabulary to generate an input sequence with a constraint vocabulary is just one example. Any process that uses an input sequence and a constraint vocabulary can be used to generate an input sequence with a constraint vocabulary.

[0052] <Input / output generation unit 140 of learning device 100> The input / output generation unit 140 generates a training input sequence by performing the same processing as the input generation unit 220 of the generation device 200, using the source language sentence of the parallel text and the generated constraint vocabulary. The training output uses the target language sentence of the parallel text.

[0053] In other words, the input / output generation unit 140 outputs pairs of expanded input sequences and target language sentences as training data for vocabulary-constrained machine translation.

[0054] <Model learning unit 160 of learning device 100> The model learning unit 160 calculates the loss between the model's output for an input sequence and the target language sentence from pairs of input sequences and target language sentences of lexically constrained machine translation training data, and updates the model's parameters to minimize this loss.

[0055] <Series conversion unit 230 of generation device 200> The sequence conversion unit 230 uses a machine translation model to generate a sentence from the extended input sequence. More specifically, it outputs the probability of each word in the set of words that can constitute the output sequence.

[0056] The sequence conversion unit 230 of the generation device 200 changes the embedding vector representation in the embedding layer of the encoder in the machine translation model from a general embedding vector representation in accordance with the expansion of the input sequence. This process is performed to distinguish between the source language sentence and each constraint vocabulary.

[0057] More specifically, the embedding vector representations generated by the sequence conversion unit 230 include token embeddings, positional embeddings, and segment embeddings. Figure 8 shows examples of token embeddings, positional embeddings, and segment embeddings converted by the sequence conversion unit 230.

[0058] Positional embeddings are information indicating the position of each token, and segment embeddings are information identifying each segment in the input sequence. Token embeddings, positional embeddings, and segment embeddings themselves are commonly used in general encoder-decoder models.

[0059] The sequence conversion unit 230 starts the position of the constraint vocabulary at a value sufficiently larger than the length of the original language sentence in order to avoid overlap between the original language sentence and the constraint vocabulary. It also assigns different values ​​as segments to the original language sentence and each constraint phrase.

[0060] <Search unit 240 of generation device 200> The search unit 240 uses the output probabilities of the decoder in the machine translation model to search for an output sequence (or an approximate solution thereof) that maximizes the generation probability given an input sequence. The search unit 240 ensures that the output sequence satisfies all constraint vocabulary by using a beam search-based grid beam search method, which is the same method as disclosed in Non-Patent Document 1, for example.

[0061] Specifically, at each processing time j, the search unit 240 groups the output candidates according to the number of constraint tokens each candidate satisfies, and then performs a search by retaining a predetermined number of candidates from the sequences with the highest generation probability within each group. This allows the most probable output sequence to be obtained under the condition that it must contain the constraint vocabulary.

[0062] Note that the search unit 240 performing the search using grid beam search is just one example. Any processing method that performs a constrained search, including the constraint vocabulary, may be used.

[0063] <Modification section 250 of the generating device 200> The modification unit 250 modifies the output sequence (also called the output sentence) output by the search unit 240, correcting the parts of the translated sentence that indicate constraint vocabulary so that there are no discrepancies with the given constraint vocabulary. More details are as follows.

[0064] During the pre-processing process, which prepares the constraint vocabulary for input into a machine translation model, information about whitespace may be lost, or the handling of multibyte character encodings may differ from the original. In addition, words that the machine translation model cannot handle may be replaced with special characters. As a result, when the processed "constraint vocabulary" is input into the machine translation model, a discrepancy may arise between the input "constraint vocabulary" and the original constraint vocabulary initially provided.

[0065] In that case, searching for exact matches using the original constraint vocabulary within the output sequence may no longer yield results, potentially resulting in an output sequence that does not include the original constraint vocabulary.

[0066] Such problems are not limited to the "input generation unit 220 + sequence conversion unit 230 + search unit 240" configuration of this embodiment, but can occur in conventional vocabulary-constrained machine translation models in general.

[0067] To solve the above problems, the modification unit 250 performs modification processing on the concatenated output sequence.

[0068] Specifically, the modification unit 250 retains the original output sequence (the output sentence without modification) and the original constraint vocabulary. For example, if the modification unit 250 successfully matches the normalized sequence with the normalized constraint vocabulary, it modifies the original output sequence with the original constraint vocabulary. Examples of normalization processes are listed below. However, these are just examples and are not limited to those listed below.

[0069] (1) Normalization related to character system factors Normalization related to character system factors includes, for example, normalization of multibyte characters. For instance, if full-width symbols are represented in hexadecimal, they are corrected to their correct, non-hexadecimal representation. This applies to languages ​​with multibyte characters, such as Japanese and Chinese, and normalization is performed only on the output sequence. However, normalization may also be applied to the constraint vocabulary.

[0070] (2) Normalization of tokenizer factors Normalization of tokenizer factors involves, for example, removing leading and trailing spaces from symbols. That is, since tokenization introduces spaces before and after symbols such as hyphens, these are removed. Note that this is a general normalization process and is applied only to the output sequence. However, normalization may also be applied to the constraint vocabulary.

[0071] (3) Normalization of predetermined character factors In the normalization of a specified character factor, the specified character is, for example, a space. For instance, for constraint vocabulary that does not match in the translated text using a simple exact match, a search is performed again, ignoring full-width and half-width spaces, and the matching locations are replaced with the original notation of the constraint vocabulary. Note that this is a general-purpose normalization process.

[0072] Both the constraints and the output statements are normalized (e.g., both full-width and half-width spaces are ignored) before matching. The character range of the original output statement corresponding to the string range of the matched output statement is then replaced with the original constraint vocabulary.

[0073] (4) Normalization of unknown word factors If the machine translation model encounters words it cannot handle (unknown words, OOV), it outputs a special token "??". Therefore, taking this into account, a modified version of the constraint vocabulary is created by replacing substrings in the constraint vocabulary with this special token, and then the original constraint vocabulary is used to replace the exact matches. For example, for the constraint vocabulary "multiple lacunar infarct cases," "multiple lacunar ?? cases" is created and used for the search.

[0074] Note that the special token "??" is just an example; any token that indicates an unknown word to the model will work.

[0075] Here, assuming the configuration shown in Figure 4, an example of the procedure for performing (4) above is shown below.

[0076] S10: The vocabulary-constrained sequence generator 270 outputs a sentence that contains "??". For example, it outputs "This case was a case of multiple lacunar infarcts". Searching this output sentence using the constraint vocabulary "cases of multiple lacunar infarcts" will not yield a match.

[0077] S20: The modification unit 250 assumes that the output sentence contains a string of constraint vocabulary and performs the process of searching for the constraint vocabulary, which is "multiple lacunar infarct cases". Specifically, it performs the following S20-1 and S20-2.

[0078] S20-1: Modification unit 250 searches for output sentences containing ?? using a string obtained by replacing any part of the constraint vocabulary with ?? as the query. For example, searches are performed for multiple lacunar ?? cases, multiple ?? infarct cases, ?? lacunar infarct cases, ... and so on. The search is performed with all possible patterns replaced with ??.

[0079] S20-2: Modification unit 250 replaces the matched character range in the output statement with the original constraint vocabulary (in the example above, "Multiple Lacunar Infarct Cases") when a matching query is found.

[0080] The above process allows us to obtain output sentences that can be confirmed to contain 100% of the constraint vocabulary through an exact match search.

[0081] Of the processes (1) to (4) described above, for example, (1) is performed first, and then (2), (3), and (4) are performed in order. However, this is just an example, and at least one of the processes (1) to (4) may be performed in any order.

[0082] An example of the processing procedure for the modification unit 250 will be explained with reference to the flowcharts in Figures 9 and 10.

[0083] In S1 of Figure 9, the modification unit 250 concatenates the words, which are the search results received from the search unit 240, to form an output sentence.

[0084] The modification unit 250 repeats the processing in S3 to S8 for each constraint vocabulary (S2). In S3, the modification unit 250 performs a search using the constraint vocabulary as the query and the output sentence as the search target.

[0085] In S4, the modification unit 250 determines whether the query does not match (i.e., whether there are constraint vocabulary words that do not exist in the output statement). If the determination in S4 is Yes (no match), proceed to S5; otherwise, proceed to S8.

[0086] In S5, the modification unit 250 normalizes at least one of the constraint vocabulary or the output sentence before performing matching. That is, after normalization, it performs a search using the constraint vocabulary as the query and the output sentence as the search target.

[0087] In S6, the modification unit 250 determines whether a match was found. If the determination in S6 is No, the process returns to S2 and processes the next constraint vocabulary.

[0088] If the decision in S6 is Yes, the process proceeds to S7. In S7, the modification unit 250 replaces the original output sentence (output sentence before normalization) with the original constraint vocabulary (constraint vocabulary before normalization) at the matching locations.

[0089] In S8, if all constraint vocabulary has been checked, the process ends. If not all constraint vocabulary has been checked, the process from S2 is executed for the next constraint vocabulary.

[0090] Next, the details of the process in S5 in Figure 9 will be explained with reference to Figure 10.

[0091] In S51, the modification unit 250, as a preprocessing step, copies the original constraint vocabulary to create a matching constraint vocabulary and copies the original output sentence to create a matching output sentence. The modification unit 250 then performs the following normalization and matching on the matching constraint vocabulary or the matching output sentence. The normalization process corresponds to (1) to (4) described above.

[0092] In S52, the correction unit 250 performs character system normalization. Specifically, for example, it normalizes multibyte characters in the matching output text. Normalization of multibyte characters means, for example, converting full-width symbols that are represented in hexadecimal to their correct characters.

[0093] In S53, the modification unit 250 performs normalization of the tokenizer factors. For example, the modification unit 250 removes leading and trailing spaces from symbols in the matching output sentence.

[0094] In S54, the modification unit 250 normalizes predetermined characters. Specifically, for example, the modification unit 250 performs a process to remove whitespace (full-width / half-width) from the matching constraint vocabulary and the matching output sentence.

[0095] In S55, the modification unit 250 normalizes the unknown word factor. For example, the modification unit 250 creates a version in which substrings in the matching constraint vocabulary are replaced with special tokens. Since there are multiple types of special tokens and multiple locations where replacement is possible, the modification unit creates versions for each of these multiple replacements and performs brute-force matching with the output sentence in S56. In S56, the modification unit 250 performs the matching.

[0096] Note that the procedure in Figure 10 is just one example. For example, a search (matching) of the output sentence using the constraint vocabulary may be performed after each step from S52 to S55. In this case, the matching constraint vocabulary and matching output sentence may be initialized (a copy of the original) each time a match fails, or the normalization in the next step may be performed while maintaining the normalization result from the previous step.

[0097] However, performing all normalization before matching increases the likelihood of successfully matching constraint vocabulary that does not match the output sentence due to multiple factors.

[0098] Furthermore, steps S52-56 do not necessarily have to be performed in this order. However, processing in this order can further improve the accuracy of matching unknown words.

[0099] (Experimental results) Experiments were conducted using the generation apparatus 200 in this embodiment. In the following description of the experimental results, "the generation apparatus 200 in this embodiment" will be referred to as the proposed system.

[0100] Figure 11 shows the base detailed settings and hyperparameters for each setting used in the experiment. Note that Figure 11 is only an example of detailed settings and hyperparameters.

[0101] Figure 12 shows the evaluation results for each setting (BLUE scores for English to Japanese and Japanese to English). (a) BASE shows the evaluation results when using a general transformer model. (b) BASE+LCD shows the evaluation results when using a model called LCD (lexically constrained decoder) that uses grid beam search, as disclosed in Non-Patent Document 1, "Post and Vilar (2018) proposed a grid beam search decoder. They guarantee to satisfy all constraints."

[0102] (c) LeCA shows the evaluation results using a model called LeCA (Lexical-Constraint-Aware NMT) disclosed in the reference "Chen et al. (2020) proposed a method that augment the input of the translation model." (d) LeCA+LCD shows the evaluation results of the proposed system. As shown in Figure 12, the proposed system yields the best evaluation results.

[0103] Figure 13 shows examples of translations by the "Base+LCD" model and the proposed system "LeCD+LCD". In Figure 13, "Source" shows the original language text, "Reference" shows the correct translation, and "Constraints" shows the constraint vocabulary.

[0104] The underlined parts in Figure 13 indicate the sections that matched the constraint vocabulary. As shown in Figure 13, although the translated sentences generated by the "Base+LCD" model contain all the constraint vocabulary, the same phrases are repeatedly generated, resulting in poor translation.

[0105] In contrast, the proposed system successfully generates translated sentences that include the constraint vocabulary. This is because the LeCA model in the proposed system assigns a higher score to the constraint vocabulary compared to the "Base+LCD" model, thereby enabling the generation of sentences that include the constraint vocabulary.

[0106] Figure 14 shows the BLEU scores for English-Japanese translation using various beam sizes for both the "Base+LCD" model and the proposed system. The "Base+LCD" model requires a beam size greater than 60 to generate a translation that includes all constraint vocabulary. In contrast, the proposed system can accurately generate a translation that includes all constraint vocabulary even with a smaller beam size.

[0107] In other words, the proposed system can shorten the search time and, as is clear from the experimental results, can improve translation accuracy.

[0108] Furthermore, experiments confirmed that the proposed system can ensure 100% inclusion of the constraint vocabulary in the translated text by implementing the normalization correction described above. Specifically, in the experiment, without normalization correction, only about 94% of the constraint vocabulary was included in an exact match, but by implementing normalization correction, 100% of the constraint vocabulary could be included.

[0109] Furthermore, in the proposed system that performs correction through normalization, when the BLUE of the translated text was compared with the BLUE of the translated text after replacing parts that do not contain the constraint vocabulary in an exact match with empty strings, it was confirmed that there was almost no difference between the two. This also indicates that all the constraint vocabulary is included in the translated text.

[0110] (Example hardware configuration) Any of the devices described in this embodiment (learning device 100, generation device 200) can be realized, for example, by having a computer execute a program. This computer may be a physical computer or a virtual machine on the cloud.

[0111] In other words, the device can be realized by using hardware resources such as the CPU and memory built into a computer to execute a program corresponding to the processing performed by the device. The program can be recorded on a computer-readable recording medium (such as portable memory), saved, and distributed. It can also be provided via a network, such as the Internet or email.

[0112] Figure 15 shows an example of the hardware configuration of the computer described above. The computer in Figure 15 has a drive device 1000, an auxiliary storage device 1002, a memory device 1003, a CPU 1004, an interface device 1005, a display device 1006, an input device 1007, an output device 1008, etc., all of which are interconnected by a bus BS. The computer may also be equipped with a GPU.

[0113] The program that enables processing on the computer is provided, for example, on a recording medium 1001 such as a CD-ROM or memory card. When the recording medium 1001 containing the program is set in the drive device 1000, the program is installed from the recording medium 1001 to the auxiliary storage device 1002 via the drive device 1000. However, the program does not necessarily have to be installed from the recording medium 1001; it may also be downloaded from another computer via a network. The auxiliary storage device 1002 stores the installed program as well as necessary files and data.

[0114] The memory device 1003 reads and stores a program from the auxiliary storage device 1002 when a program startup command is received. The CPU 1004 implements the functions related to the light touch maintenance device 100 according to the program stored in the memory device 1003. The interface device 1005 is used as an interface for connecting to a network, etc. The display device 1006 displays a GUI (Graphical User Interface) etc. generated by the program. The input device 1007 consists of a keyboard and mouse, buttons, or a touch panel etc., and is used to input various operation commands. The output device 1008 outputs the calculation results.

[0115] (Effects of the embodiment, etc.) As described above, in this embodiment, for machine translation with vocabulary constraints, input information is expanded according to the constraint vocabulary, the expanded information is input to the machine translation model, and a grid beam search is performed on the output from the machine translation model to ensure that all of the constraint vocabulary is included. This makes it possible to generate a translated sentence that includes all of the constraint vocabulary in a short search time. Furthermore, by appropriately modifying the output sentence, it is possible to generate a translated sentence that fully satisfies the constraints even if, for example, there are unknown words in the constraint vocabulary.

[0116] In summary, the generation device 200 shown in Figure 3 applied grid beam search to the output of a model that received a sequence of linked source language sentences and constraint vocabulary as input, resulting in shorter search times and improved processing speed. Furthermore, as can be seen from the experimental results, translation accuracy improved.

[0117] Furthermore, the generation device 200 shown in Figure 4 modifies the output sentence by referring to the original notation information before processing the constraint vocabulary and other data. Therefore, even when using an input with missing information as input to the model, all given constraint vocabulary will be included in the output sentence. Note that this effect can also be obtained with the generation device 200 shown in Figure 2.

[0118] The embodiments described above are further disclosed in the following appendices 1 and 2.

[0119] <Note 1> (Additional note 1) A generating device that generates an output sequence, which is a sequence of information, from an input sequence, which is a sequence of information, Memory and At least one processor connected to the memory, Includes, The aforementioned processor, Based on the aforementioned input sequence and constraint information, a constraint-equipped input sequence is generated. By inputting the aforementioned input sequence with constraint information into the sequence transformation model, output information is generated. The output sequence is generated by performing a constrained search using the output information such that the constraint information is included in the output sequence. generator. (Additional note 2) The processor modifies the output sequence obtained by the search unit so that the constraint information matches through a search of the output sequence. The generating apparatus described in Appendix 1. (Additional note 3) The processor, after normalizing at least one of the output sequence from the search unit and the constraint information, replaces the matched portion with the original, unprocessed control information if the constraint information matches the output sequence. The generating apparatus described in Appendix 2. (Additional note 4) A generation method performed by a computer that generates an output sequence, which is a sequence of information, from an input sequence, which is a sequence of information, Based on the aforementioned input sequence and constraint information, a constraint-equipped input sequence is generated. By inputting the aforementioned input sequence with constraint information into the sequence transformation model, output information is generated. The output sequence is generated by performing a constrained search using the output information such that the constraint information is included in the output sequence. Generation method. (Additional note 5) A non-temporary storage medium storing a program that causes a computer to function as a generating device as described in any one of the appendices 1 through 3.

[0120] <Note 2> (Additional note 1) A generating device that generates an output sequence, which is a sequence of information, from an input sequence, which is a sequence of information, Memory and At least one processor connected to the memory, Includes, The aforementioned processor, Using a sequence transformation model, the output sequence is generated from the input sequence and the given constraint information. The output sequence is modified so that the constraint information is included in the output sequence. generator. (Additional note 2) The processor uses the constraint information as a query, normalizes at least one of the output sequence from the generation unit and the query, searches the output sequence using the query, and replaces the portion of the output sequence that matches the query with the original constraint information corresponding to the query, which has not undergone the normalization process. The generating apparatus described in Appendix 1. (Additional note 3) The processor performs at least one of the following normalizations as the normalization: normalization of character system factors, normalization of tokenizer factors, normalization of predetermined characters, and normalization of unknown word factors. The generating apparatus described in Appendix 2. (Additional note 4) A generation method performed by a computer that generates an output sequence, which is a sequence of information, from an input sequence, which is a sequence of information, Using a sequence transformation model, the output sequence is generated from the input sequence and the given constraint information. The output sequence is modified so that the constraint information is included in the output sequence. Generation method. (Additional note 5) A non-temporary storage medium storing a program that causes a computer to function as a generating device as described in any one of the appendices 1 through 3.

[0121] Although this embodiment has been described above, the present invention is not limited to this specific embodiment, and various modifications and changes are possible within the scope of the gist of the invention as described in the claims. [Explanation of symbols]

[0122] 100 Learning Devices 110 Parallel Text Data Database 120 Input section 130 Vocabulary constraint generator 140 Input / Output Generation Unit 150 Training Data Database 160 Model Learning Department 170 Output section 200 generator 210 Input section 220 Input generation unit 230 Series Conversion Unit 240 Search Department 250 Correction section 260 Output section 270 Sequence generation unit with lexical constraints 1000 drive unit 1001 Recording media 1002 Auxiliary storage 1003 Memory device 1004 CPU 1005 Interface device 1006 Display device 1007 Input device 1008 Output device< / eos> < / sep> < / sep> < / eos> < / sep>

Claims

1. A generation device that generates an output sequence of another string from an input sequence of a string, A generation unit that generates the output sequence from the input sequence and given constraint information using a lexical-constrained sequence transformation model, The system includes a modification unit that modifies the output sequence obtained by the generation unit so that the constraint information is included in the output sequence, The modification unit uses the constraint information as a query, normalizes at least one of the output sequence from the generation unit and the query, searches the output sequence using the query, and replaces the portion of the output sequence that matches the query with the original constraint information corresponding to the query, which has not undergone the normalization process. generator.

2. The modification unit performs at least one of the following normalizations as the normalization: normalization of character system factors, normalization of tokenizer factors, normalization of predetermined characters, and normalization of unknown word factors. The generating apparatus according to claim 1.

3. A generation method performed by a computer that generates an output sequence, which is another string, from an input sequence, which is a string, Using a lexical-constrained sequence transformation model, the output sequence is generated from the input sequence and the given constraint information. A generation method for modifying the output sequence so that the constraint information is included in the output sequence. In modifying the output sequence, the computer uses the constraint information as a query, normalizes at least one of the output sequence and the query, searches the output sequence using the query, and replaces the portion of the output sequence that matches the query with the original constraint information corresponding to the query, which has not undergone the normalization process. Generation method.

4. A program for causing a computer to function as a component in the generating apparatus described in claim 1 or 2.