Learning device, inference device, learning method, inference method, and program

Abstract

To achieve series conversion in which a structure of text is used as a constraint.SOLUTION: A learning device includes: a generation unit that generates, from data about a pair of first text and second text, a structural constraint on the second text and a structured second text; and a learning unit that, based on the first text, the structural constraint, and the structured second text, performs training of a series conversion model that performs structurally constrained series conversion.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to the technical field of machine translation. [Background technology]

[0002] Constrained vocabulary machine translation is known as a conventional technique (e.g., Non-Patent Documents 1 to 3). Constrained vocabulary machine translation is a task that performs machine translation under the constraint of generating sentences that contain specified words as translations. By specifying translations, it is possible to ensure consistency of translations within a document, which is important in translations for patents, legal matters, etc.

[0003] Vocabulary-constrained machine translation can also be applied to interactive translation, where a human provides instructions for corrections while translating, such as post-editing. This task has been actively tackled in recent years, and it is becoming possible to generate high-quality translations that satisfy the constraints. [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] Matt Post and David Vilar.: Fast lexically constrained decoding with dynamic beam allocation for neural machine translation. In Marilyn Walker, Heng Ji, and Amanda Stent, editors, Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pp.1314-1324, New Orleans, Louisiana, June 2018. Association for Computational Linguistics. [Non-Patent Document 2] Guanhua Chen, Yun Chen, Yong Wang, and Victor O.K. Li.: Lexical-constraint-aware neural machine translation via data augmentation.In Proceedings of IJCAI 2020: Main track, pp.3587-3593, 7 2020. [Non-Patent Document 3] Katsuki Chousa and Makoto Morishita.: Input augmentation improves constrained beam search for neural machine translation: NTT at WAT 2021. In Proceedings of the 8th Workshop on Asian Translation (WAT2021), pp. 53-61, Online, August 2021.Association for Computational Linguistics. [Summary of the Invention] [Problem to be solved by the invention]

[0005] Expanding the units specified as constraints from phrases to text structures is expected to improve the operability of machine translation output. However, there is no prior art machine translation technology that uses text structures as constraints. Note that using text structures as constraints is not limited to translation, but can be applied to sequence conversion in general.

[0006] The present invention has been made in view of the above points, and an object of the present invention is to provide a technique for realizing sequence transformation using the structure of text as a constraint. [Means for solving the problem]

[0007] According to the disclosed technology, a generation unit generates a structure constraint for the second text and a structured second text from paired data of the first text and the second text; a learning unit that learns a sequence conversion model that performs sequence conversion with a structural constraint based on the first text, the structural constraint, and the structured second text; A learning device is provided, comprising: [Effects of the Invention]

[0008] The disclosed technology provides a technique for implementing sequence transformation that uses the structure of text as a constraint. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 10 is a diagram illustrating an example of linearization. [Figure 2] FIG. 10 is a diagram for explaining structurally constrained translation. [Figure 3] FIG. 1 is a configuration diagram of a learning device 100 and an inference device 200. [Figure 4] FIG. 10 is a diagram showing a processing flow during learning. [Figure 5]FIG. 10 is a diagram showing a processing flow during inference. [Figure 6] FIG. 10 is a diagram showing details of baseline settings and hyperparameters. [Figure 7] FIG. 10 is a diagram showing experimental results. [Figure 8] FIG. 1 is a configuration diagram of a learning device 100. [Figure 9] FIG. 2 is a diagram illustrating the configuration of an inference device 200. [Figure 10] FIG. 2 illustrates an example of a hardware configuration of the apparatus. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, an embodiment of the present invention (the present embodiment) will be described with reference to the drawings. The embodiment described below is merely an example, and the embodiment to which the present invention is applied is not limited to the following embodiment.

[0011] Text has an underlying structure (e.g., phrase structure, discourse structure), which expresses sentence patterns and relationships between words in the text. In this embodiment, structurally constrained machine translation using such structures will be described. Structurally constrained machine translation is a task of converting an input sentence into another language under the structural constraint that the output sentence must have a specified structure.

[0012] In this embodiment, as an example, the case where the technology according to the present invention is applied to translation into another language will be described, but the technology according to the present invention is not limited to translation into another language, and can also be applied to sequence conversion between the same languages ​​(e.g., sequence conversion into another domain of the same language, summarization, speech generation, etc.) as long as it is sequence conversion using an encoder-decoder model. In the description of this embodiment, "machine translation" may be replaced with "sequence conversion."

[0013] In the present embodiment described below, the unit of translation is a sentence, but the units in the source language and target language may be any units. In other words, it is not necessary to use units such as sentences or paragraphs.

[0014] Below, first, the problem will be described in more detail, and then the technology according to the present embodiment will be described.

[0015] (About the assignment) As mentioned above, when considering the syntactic structure and discourse structure of a translated sentence, being able to specify its substructure as a constraint makes it possible to manipulate the sentence structure of the translated sentence and the relationships between phrases.

[0016] In other words, by using subtrees of syntactic structures such as phrase structures and dependency structures as constraints, it becomes possible to select voice in translation, and to resolve differences in word order that are a problem in simultaneous interpretation between Japanese and English, etc. Furthermore, by using discourse structure, it is possible to ensure that the consistency of the parallel relationships of phrases, which is important in translating patent documents, etc., is maintained before and after translation.

[0017] However, no neural machine translation method with structural constraints has been studied to date. However, there is a task called vocabulary-constrained machine translation, in which phrases are specified as constraints, and there are several prior arts for this task (e.g., Non-Patent Documents 1 to 3).

[0018] (Outline of this embodiment) In this embodiment, a neural machine translation method with structural constraints is realized by the following techniques.

[0019] To provide structural constraints, two pieces of information are required: the translation sentence and its structure. To achieve this, we consider a structured translation sentence in which these two pieces of information are converted into a string (called "linearization"). Then, we use a structurally constrained machine translation model to generate the structured translation directly from the source sentence.

[0020] Linearization in natural language processing is a procedure for converting a graph or tree structure into a string of characters. In linearization, a method is generally used to convert a graph or tree structure into a string of characters in an order called the pre-order order of a depth-first search, and this method is also adopted in this embodiment.

[0021] An example of linearization is shown in Figure 1. Figure 1 shows an example of linearizing a phrase structure tree into a string. In the example in Figure 1, a format called an S-expression is used to describe the tree nodes in the order they would be found in a depth-first search that prioritizes the left child.

[0022] Each structural constraint is also linearized and converted into a single continuous string. This allows the structural constraint to appear as a substring of the structured translation output by the structurally constrained machine translation model, making it possible to handle structural constraints in the same framework as vocabulary constraints. In this embodiment, the vocabulary constrained machine translation method is extended to implement a translation model that takes structural constraints into account and structurally constrained decoding, thereby achieving structurally constrained machine translation.

[0023] Experiments on Japanese-English translation have shown that the technology according to this embodiment can generate translations that fully satisfy constraints without sacrificing accuracy. It has also been shown that even with only a translation model that takes structural constraints into account, highly accurate translations can be generated with a structural constraint satisfaction rate approaching 100%. Details of the experiment will be described later.

[0024] To facilitate understanding of the processing content according to this embodiment, an example of general translation and processing executed by inference device 200, which will be described later, is shown in FIG.

[0025] In general translation, an input sentence is converted into a translation sentence, as shown in Fig. 2. In contrast, inference device 200 according to this embodiment receives an input sentence and structural constraints, linearizes the specified structural constraints, and outputs a structured translation sentence as a linearized character string from the structural constraints and the input sentence (source language sentence) using a translation model with structural constraints.

[0026] Furthermore, the inference device 200 may further perform a structurally constrained search on the above output, thereby achieving high translation accuracy while satisfying the structural constraints.

[0027] (Device configuration example, operation overview) FIG. 3 shows an example configuration of learning device 100 and inference device 200 according to this embodiment. Learning device 100 is a device that learns a structurally constrained machine translation model. Inference device 200 is a device that executes structurally constrained machine translation using the structurally constrained machine translation model. Note that learning device 100 and inference device 200 may be separate devices, or the functions of learning device 100 and inference device 200 may be included in a single device.

[0028] <Learning device 100> 3, learning device 100 can access a bilingual text data DB (database) 110. Learning device 100 may include bilingual text data DB 110.

[0029] 3, the learning device 100 includes a structural constraint generation unit 120, an input / output generation unit 130, a learning data DB 140, and a model learning unit 150. The functions of each unit are outlined below.

[0030] The bilingual data DB 110 is a storage unit (which may be a database server) that stores bilingual data. The structural constraint generation unit 120 generates structural constraints from the bilingual data. The input / output generation unit 130 generates input / output (training data) for machine translation with structural constraints.

[0031] The training data DB 140 is a storage unit that stores the inputs and outputs of the structurally constrained machine translation generated by the input / output generation unit 130 as training data. The model training unit 150 uses the training data stored in the training data DB 140 to train the structurally constrained machine translation model. The trained structurally constrained machine translation model is stored in the machine translation model DB 300. Specifically, what is stored in the machine translation model DB 300 is model parameters, etc.

[0032] The operational flow of the learning device 100 will be described with reference to the flowchart in Fig. 4. In S101 (step 101), bilingual sentence data read from the bilingual sentence data DB 110 is input to the learning device 100. In S102, the structural constraint generation unit 120 generates linearized structural constraints.

[0033] In S103, the input / output generating unit 130 generates pairs of extended inputs and outputs as training data from the structural constraints and the bilingual sentences, and stores the generated training data in the training data DB 140.

[0034] In S104, the model learning unit 150 learns a structurally constrained machine translation model from the expanded input-output pairs, and stores the learned model in the machine translation model DB 150.

[0035] <Inference device 200> Next, we will explain the configuration and operation of inference device 200. As shown in Fig. 3, inference device 200 includes input generation unit 210, sequence conversion unit 220, and translation search unit 230. The functions of each unit are outlined below.

[0036] The input generation unit 210 generates input for a structurally constrained machine translation model from an input sentence and structural constraints. The sequence conversion unit 220 uses the trained structurally constrained machine translation model to perform sequence conversion prediction from the input generated by the input generation unit 210. The translation search unit 230 searches for a translation that satisfies the structural constraints from the prediction results.

[0037] The operational flow of the inference device 200 will be described with reference to the operational flow of Fig. 5. In S201, an input sentence and a structural constraint are input to the inference device 200.

[0038] In S202, the input generation unit 210 linearizes the structural constraints. Then, in S203, the input generation unit 210 extends the input sequence using the structural constraints. In S204, the sequence transformation unit 220 performs sequence transformation prediction on the extended input sequence using a trained machine translation model with structural constraints.

[0039] In S205, the translation search unit 230 searches for a translation that satisfies the structural constraints from the prediction results of the structurally constrained machine translation model. In S206, the translation search unit 230 outputs the translation obtained by the search. The processing content of each device will be explained in more detail below.

[0040] (formulation) First, the formulation in this embodiment will be explained. Structurally constrained machine translation is a process of generating an original text X=(x1,...,x I ) and a set of substructures in the target language C=(C1,...,C K ) is given as a structural constraint, the translation Y=(y1, ,y J In this case, the order in the list given as constraints does not necessarily match the order in which the substructures appear in the translation.

[0041] In conventional techniques, lexical constraints are directly included in the translation, whereas in the task of this embodiment, structural constraints are not directly included in the translation. Therefore, to determine whether the generated translation satisfies the constraints, it is necessary to separately acquire the structure of the translation. Furthermore, because conventional translation models do not have access to the structure of the translation during the generation process, it is difficult to apply existing constrained decoding methods to structural constraints.

[0042] Therefore, in this embodiment, in order to obtain a translation and its structure, the structure is linearized as an S-formula and expressed as a single string, and a structured translation Y with L tokens is obtained. str =(y1 str ,···,y L str ) is generated directly. k is Y str The sequence is given as a single continuous S-expression with the same structure as above. Then, structurally constrained machine translation is realized by searching for a sequence that maximizes the following conditional probability from among sequences that satisfy constraint C.

[0043]

number

[0044] The processing contents of each unit for realizing the machine translation formulated as above will be explained in more detail below.

[0045] (Structural constraint generation unit 120 of learning device 100) The bilingual data (data of pairs of source language sentences and translation sentences) read from the bilingual data DB 110 does not contain information on structural constraints. Therefore, the structural constraint generation unit 120 receives the bilingual sentences as input and generates structural constraints for the bilingual sentences. The structural constraint generation unit 120 outputs the generated structural constraints to the structural constraint-attached input / output generation unit 120.

[0046] That is, the structural constraint generating unit 120 creates structural constraints from the correct translations in the bilingual texts. These structural constraints are used as pseudo-correct data for the structural constraints during model training.

[0047] More specifically, the structural constraint generation unit 120 acquires structural information of the correct translation sentence from the bilingual corpus (parallel sentences) using an existing parser, and randomly extracts substructures from the structural information to create structural constraints. The structural constraint generation unit 120 randomly extracts K substructures so that there is no overlap between the structural constraints. At this time, the number K of constraints is randomly determined according to a probability distribution given as a hyperparameter.

[0048] (Input generation unit 210 of inference device 200 / input / output generation unit 130 of learning device 100) Next, we will explain the input generation unit 210 of the inference device 200 and the input / output generation unit 130 of the learning device 100. Hereinafter, for operations common to the input generation unit 210 and the input / output generation unit 130, the subject of the operation will be written as "input generation unit 210 / input / output generation unit 130."

[0049] The input generator 210 receives a source sentence and structural constraints as inputs. The input / output generator 130 receives a source sentence, a translation sentence, and structural constraints as inputs.

[0050] The input / output generating unit 130 acquires the structure of the correct answer sentence (translation sentence) using an existing parser, linearizes it, and creates a structured translation sentence. In this embodiment, linearization is performed using the S-formula described above.

[0051] The input generation unit 210 / input / output generation unit 130 linearizes the input structural constraints in the same way as the structured translation.

[0052] The input generation unit 210 / input / output generation unit 130 expands the source language sentence and creates an input sequence with added information on structural constraints.

[0053] Specifically, the input generation unit 210 / the input / output generation unit 130 uses a technique of combining a source language sentence with structural constraints to expand the input sequence. This input sequence is a delimiting symbol (a special character string). <sep>The source language sentence and each substructure C in the structural constraint set C k By concatenating these, it is created as follows:

[0054]

number

[0055] The input generation unit 210 outputs the expanded input sequence to the sequence conversion unit 220 , and outputs the structural constraints expressed as S-expressions to the translation search unit 230 .

[0056] The input / output generating unit 130 outputs pairs of the expanded input sequence and the structured translation as training data for the structurally constrained machine translation. The training data is stored in the training data DB 140.

[0057] (Model learning unit 150 of learning device 100) Next, the operation of the model learning unit 150 of the learning device 100 will be described. From pairs of input sequences and structured translations in the training data for structurally constrained machine translation, the model learning unit 150 calculates the loss between the output from the model for the input sequence and the correct target language sentence (structured translation), and updates the model parameters. For example, the model parameters are optimized to minimize the loss.

[0058] The model used is not limited to a specific one, but in this embodiment, an encoder-decoder model (a neural network model) that is commonly used as a sequence conversion model (for example, a sequence conversion model for translation) is used. The encoder may also be called an encoder, and the decoder may also be called a decoder.

[0059] In the encoder-decoder model, an input sequence is input to the encoder, and a sequence transformation prediction result is output from the decoder.

[0060] (Sequence transformation unit 220 of inference device 200) Next, we will explain sequence conversion unit 220 of inference device 200. Sequence conversion unit 220 uses a trained model (specifically, an encoder-decoder model) trained by model training unit 150 to generate a sentence using an extended input sequence. The processing by sequence conversion unit 220 is similar to general neural machine translation processing using an encoder-decoder model.

[0061] (Translation search unit 230 of inference device 200) Next, we will explain the translation search unit 230 of the inference device 200. The translation search unit 230 uses the output probability of the decoder in the sequence conversion unit 220 (trained model) to search for (an approximate solution to) the output sequence that maximizes the generation probability when an input sequence is given. In other words, as explained in the formulation section, it searches for a sequence that maximizes the conditional probability shown in equation (1).

[0062] The translation search unit 230 performs a search using, for example, a beam search technique, which is an existing technique. By using a search technique based on a beam search, it is possible to guarantee that the output sequence satisfies all structural constraints.

[0063] That is, the translation search unit 230 receives structural constraints from the input generation unit 210 and performs constrained decoding. In this embodiment, the method disclosed in Non-Patent Document 1 (Post and Vilar (2018)) is adopted. In this method, at each processing time j, output candidates are grouped according to the number of tokens of the constraints that the candidate satisfies, and a predetermined number of candidates with the highest generation probability are retained from each group for search. A more detailed explanation is provided below.

[0064] In a translation task executed by inference device 200, it is necessary to generate a translation that always satisfies given structural constraints, but processing by sequence conversion unit 220 using a trained model alone cannot guarantee that the generated translation will satisfy the constraints. Therefore, when decoding using a structurally constrained translation model, lexically constrained decoding (LCD) is performed using the method described in Non-Patent Document 1.

[0065] LCD is a method for selecting candidate translations by taking constraints into consideration when performing a beam search to find a translation, thereby ensuring that a translation that satisfies the constraints is obtained. This method was originally proposed as constrained decoding for vocabulary constraints, but by treating structural constraints as substrings of a structured translation, as in the technology according to this embodiment, structural constraints can be handled in a framework similar to vocabulary-constrained machine translation. Therefore, this method can be used as a constrained decoding method for structural constraints.

[0066] (About the experiment) In order to evaluate the effectiveness (effectiveness) of the technology according to this embodiment (herein referred to as the "proposed method"), we conducted an accuracy evaluation of structurally constrained machine translation using phrase structure as a structural constraint for Japanese-English translation. The details of the experiment are explained below.

[0067] <Dataset> We used the Asian scientific paper excerpt corpus (ASPEC) as the bilingual corpus for training and evaluation. Because the last 1 million sentence pairs of the ASPEC training data contained a lot of noise, we only used the first 2 million sentence pairs. Phrase structure was used to provide structural information for the text. Syntactic analysis was performed using the CRF model in SuPar (https: / / github.com / yzhangcs / parser) to create structured translations expressed as S-formulas. SentencePiece was used to segment the text into subwords. A shared vocabulary was used for both Japanese and English, with a size of 32,000. To avoid using excessively long sentences during training, we used the clean-corpus-n.perl utility in Moses (https: / / github.com / moses-smt / mosesdecoder / blob / master / scripts / training / clean-corpus-n.perl) to remove sentences longer than 512 from the training data.

[0068] When creating structural constraints for the training data, the maximum number of tokens for each constraint was set to 150, and the minimum tree depth was set to 2. In addition, the maximum number of structural constraints per sentence was set to 3, and the number of constraints K for each sentence was determined by sampling according to the following distribution p(K):

[0069]

number

[0070] <Translation model> In this experiment, the following three translation models were created and used for evaluation.

[0071] Baseline: A general translation model that generates a translation from a source text Structured translation model: A translation model that generates structured translations from source text. Structurally Constrained Translation Model: A translation model that generates structured translations from source text and structural constraints Transformer (big) was used as the baseline. Because using constrained decoding requires a larger beam size than usual, the beam size was set to 20 in this experiment. Details of the settings and hyperparameters for other baselines are shown in Figure 6. For other models, the settings and hyperparameters used are similar to those of the baseline unless otherwise specified. fairseq was used to implement the model.

[0072] <Evaluation scale> The evaluation of structurally constrained machine translation was based on the perspective of vocabulary-constrained machine translation, and was carried out from the perspective of translation accuracy of the translated sentences and the degree of fulfillment of structural constraints.

[0073] The translation accuracy was evaluated using BLEU, a de facto standard method for automatic evaluation, and sacrebleu was used for the calculation. Some of the models used in this experiment output structured translations, but when evaluating their translation accuracy, the structural information was removed from the output and only the translations were extracted and calculations were performed.

[0074] To measure the satisfaction rate of structural constraints, we used two metrics, Term% and Sent%, which are also used in constrained vocabulary machine translation (Non-Patent Document 3). Term% indicates the percentage of correctly generated structural constraints, and Sent% is defined as the percentage of translated sentences that satisfy all structural constraints.

[0075] <Experimental Results> Figure 7 shows the translation accuracy and structural constraint satisfaction rate of translated sentences for each method. First, we can see that the translation accuracy of the structured translation model did not decrease significantly, with a BLEU decrease of 0.8 points compared to the baseline. The structured translation model performs translation and syntactic analysis simultaneously by generating structured translation sentences expressed as S-expressions. This makes the task more difficult than the baseline, which only performs translation, and there was a possibility that the translation accuracy would be lower. In response to this concern, the results confirmed that the method of generating structured translation sentences can perform translation with the same level of accuracy as the baseline.

[0076] Next, when we look at the results of combining a structured translation model with constrained decoding, we see that translation accuracy improved by 1 point in BLEU compared to the structured translation model alone, and that the structural constraint fulfillment rate also achieved nearly 100%. In this case, even though constrained decoding, which should fulfill the hard constraints, is applied, Term% and Sent% do not reach 100%. This is due to orthographic variations such as character normalization during tokenization, and when this is addressed, both fulfillment rates reach 100%.

[0077] Furthermore, while the average number of words in the vocabulary constraints in the WAT vocabulary-constrained translation dataset (Proceedings of the 8th Workshop on Asian Translation (WAT2021), pp. 1-45) is 6.6 to 7.4 words, the structural constraints created in this experiment had an average of around 21 tokens, with the number of tokens increasing due to the vocabulary constraints being given as S-expressions. We confirmed that by using constrained decoding, it is possible to search for translations that satisfy the constraints, even for structural constraints consisting of such long tokens, without sacrificing accuracy.

[0078] Furthermore, focusing on the results of the structurally constrained translation model, we can see that the BLEU score was +16.3 points compared to the baseline, demonstrating a significant improvement in translation accuracy. Furthermore, the structural constraint fulfillment rate achieved scores close to 100% for both Term% and Sent%, despite not using constrained decoding, meaning that most structural constraints were satisfied. The reason for this high fulfillment rate is likely due to the extremely large number of tokens in the structural constraints. An increase in the number of tokens in the structural constraints means that an increased number of tokens are copied directly from the input to the output of the structurally constrained translation model. This may be due to the model's strong tendency to copy the input during the model training process, resulting in a very high fulfillment rate.

[0079] Finally, when we look at the results of combining a structurally constrained translation model with constrained decoding, we see that the constraint satisfaction rate is improved compared to a structurally constrained translation model alone. Meanwhile, while translation accuracy is improved compared to the baseline and methods using a structurally constrained translation model, it is still lower than a structurally constrained translation model alone. This is thought to be because constrained decoding selects translation candidates taking structural constraints into account, which prevents sufficient search compared to a typical beam search, resulting in lower translation accuracy.

[0080] (Other configuration examples) The configurations of the learning device 100 and the inference device 200 are not limited to those shown in Fig. 3. For example, the learning device 100 may have the configuration shown in Fig. 8. The learning device 100 in Fig. 8 includes a generation unit 300 and a learning unit 400. The generation unit 300 has, for example, the functions of the structural constraint generation unit 120 and the input / output generation unit 130. The learning unit 400 has, for example, the functions of the model learning unit 150.

[0081] Inference device 200 may also have the configuration shown in Fig. 9. Inference device 200 in Fig. 9 includes generation unit 500 and conversion unit 600. Generation unit 500 has, for example, the functions of input generation unit 210. Conversion unit 600 has, for example, the functions of sequence conversion unit 220 and translation search unit 230.

[0082] (Example of hardware configuration) Any of the devices described in this embodiment (learning device 100, inference device 200) can be realized, for example, by causing a computer to execute a program. This computer may be a physical computer or a virtual machine on the cloud.

[0083] That is, the device can be realized by executing a program corresponding to the processing performed by the device using hardware resources such as a CPU and memory built into a computer. The program can be recorded on a computer-readable recording medium (such as a portable memory) and stored or distributed. The program can also be provided via a network such as the Internet or email.

[0084] Fig. 10 is a diagram showing an example of the hardware configuration of the computer. The computer in Fig. 10 includes a drive device 1000, an auxiliary storage device 1002, a memory device 1003, a CPU 1004, an interface device 1005, a display device 1006, an input device 1007, an output device 1008, and the like, all of which are interconnected via a bus B. The computer may further include a GPU.

[0085] A program for realizing processing on the computer is provided by a recording medium 1001 such as a CD-ROM or a memory card. When the recording medium 1001 storing the program is set in the drive device 1000, the program is installed from the recording medium 1001 to the auxiliary storage device 1002 via the drive device 1000. However, the program does not necessarily have to be installed from the recording medium 1001, but may be downloaded from another computer via a network. The auxiliary storage device 1002 stores the installed program as well as necessary files, data, etc.

[0086] The memory device 1003 reads and stores a program from the auxiliary storage device 1002 when an instruction to start the program is received. The CPU 1004 realizes the functions related to the device in accordance with the program stored in the memory device 1003. The interface device 1005 is used as an interface for connecting to a network, etc. The display device 1006 displays a GUI (Graphical User Interface) or the like according to the program. The input device 1007 is composed of a keyboard, mouse, buttons, a touch panel, etc., and is used to input various operation instructions. The output device 1008 outputs the results of calculations.

[0087] The following additional notes are provided regarding the above-described embodiments.

[0088] <Additional Notes> (Additional note 1) a generation unit that generates a structural constraint for the second text and a structured second text from data of a pair of a first text and a second text; a learning unit that learns a sequence conversion model that performs sequence conversion with a structural constraint based on the first text, the structural constraint, and the structured second text; A learning device comprising: (Additional note 2) The generation unit generates an input sequence by adding information about the structural constraint to the first text, and the training unit trains the sequence transformation model using the input sequence as an input and the structured second text as an output. Item 1. A learning device according to item 1. (Additional note 3) a generation unit that receives an input text and an input structure constraint and generates an input sequence by adding information about the input structure constraint to the input text; a conversion unit that converts the input sequence into structured text using a sequence conversion model that has been trained to output structured text using structural constraints; An inference device comprising: (Additional note 4) The conversion unit searches for an output sequence that maximizes the generation probability when the input text and the input structure constraint are given. Item 3. An inference device according to item 3. (Additional note 5) A learning method executed by a learning device, a generating step of generating a structural constraint for the second text and a structured second text from data of a pair of a first text and a second text; a learning step of learning a sequence transformation model that performs sequence transformation with a structural constraint based on the first text, the structural constraint, and the structured second text; A learning method that includes: (Additional note 6) An inference method executed by an inference device, comprising: a generation step of receiving an input text and an input structure constraint and generating an input sequence by adding information about the input structure constraint to the input text; a transformation step of transforming the input sequence into structured text using a sequence transformation model trained to output structured text using structural constraints; An inference method comprising: (Additional note 7) A non-transitory storage medium storing a program for causing a computer to function as each part of the learning device described in appendix 1 or 2. (Additional note 8) A non-transitory storage medium storing a program for causing a computer to function as each part of the inference device described in appended claim 3 or 4.

[0089] Although the present embodiment has been described above, the present invention is not limited to such a specific embodiment, and various modifications and changes are possible within the scope of the gist of the present invention described in the claims. [Explanation of symbols]

[0090] 100 Learning Device 110 Bilingual Text Data DB 120 Structural constraint generator 130 Input / Output Generation Unit 140 Learning Data DB 150 Model Learning Department 200 Reasoning device 210 Input Generation Unit 220 Series Conversion Unit 230 Translation Search Unit 300 Generation part 400 Learning Department 500 generation section 600 conversion unit 1000 Drive Device 1001 Recording media 1002 Auxiliary storage 1003 Memory device 1004 CPU 1005 Interface device 1006 Display device 1007 Input Device 1008 Output Device< / eos> < / sep>

Claims

1. a generating unit that generates a structural constraint for the second text and a structured second text from data of a pair of a first text and a second text; a learning unit that learns a sequence transformation model that performs sequence transformation with a structural constraint based on the first text, the structural constraint, and the structured second text; A learning device comprising:

2. The generation unit generates an input sequence by adding information about the structural constraint to the first text, and the training unit trains the sequence transformation model using the input sequence as an input and the structured second text as an output. The learning device according to claim 1 .

3. a generation unit that receives an input text and an input structure constraint and generates an input sequence by adding information about the input structure constraint to the input text; a conversion unit that converts the input sequence into structured text using a sequence conversion model that has been trained to output structured text using structural constraints; An inference device comprising:

4. The conversion unit searches for an output sequence that maximizes the generation probability when the input text and the input structure constraint are given. The inference device according to claim 3 .

5. A learning method executed by a learning device, a generating step of generating a structural constraint of the second text and a structured second text from data of a pair of a first text and a second text; a learning step of learning a sequence transformation model that performs sequence transformation with a structural constraint based on the first text, the structural constraint, and the structured second text; A learning method that includes:

6. An inference method executed by an inference device, comprising: a generation step of receiving an input text and an input structure constraint and generating an input sequence by adding information about the input structure constraint to the input text; a transformation step of transforming the input sequence into structured text using a sequence transformation model trained to output structured text using structural constraints; An inference method comprising:

7. A program for causing a computer to function as each unit in the learning device according to claim 1 or 2.

8. A program for causing a computer to function as each unit in the inference device according to claim 3 or 4.

Images