Sentence generation model generating device, sentence generation model and sentence generation device

Abstract

This text generation model generating device is a device which employs machine learning to generate a text generation model for generating an output text in a second language in accordance with an input of an input text in a first language, and is provided with: an encoder input unit for inputting first data constituting the input text into an encoder; a decoder input unit for inputting a context for specifying a condition of the output text, a starting symbol, and second data constituting the output text into a decoder; an updating unit for updating weighting factors which are constituents of the encoder and the decoder, on the basis of a word-by-word error between a word array output from the decoder in a subsequent stage including and following the starting symbol, and a word array included in the second data; and a model output unit which outputs the text generation model of which the weighting factors have been updated.

Description

[Technical Field]

[0001] The present invention relates to a sentence generation model generating device, a sentence generation model, and a sentence generation device. [Background technology]

[0002] There is known a technology that uses machine learning to generate a model that generates an output sentence consisting of a translation in a second language, for example, in response to an input sentence in a first language, and that uses the generated model to configure a translation engine, a scoring engine, etc. For example, Patent Document 1 discloses a technology that uses a machine learning model to generate a document corresponding to an input document. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2020-135457 Summary of the Invention [Problem to be solved by the invention]

[0004] Conventional translation engines and scoring engines, which are constructed using models trained based on an input sentence and its translation, output translations that correspond in meaning, but are unable to output translations that comply with conditions such as the use of specific expressions and conformity to the context.

[0005] The present invention has been made in view of the above problems, and has as its object to obtain an output sentence in a second language that complies with specific conditions in response to an input sentence in a first language. [Means for solving the problem]

[0006] In order to solve the above problem, a sentence generation model generation device according to one embodiment of the present invention is a sentence generation model generation device that generates, by machine learning, a sentence generation model that generates an output sentence in a second language different from the first language in response to an input of an input sentence in a first language, wherein the sentence generation model is an encoder-decoder model including a neural network and configured with an encoder and a decoder, and learning data used for the machine learning of the sentence generation model includes first data, a context, and second data, wherein the first data includes an arrangement of multiple words that constitute the input sentence, the second data includes an arrangement of multiple words that constitute an output sentence corresponding to the input sentence, and the context includes one or more words in the second language that are related to the second data, and the sentence generation model generation device includes: an encoder input unit that inputs the first data to the encoder in accordance with the order of the words; a decoder input unit that inputs the context, a start symbol that is a predetermined symbol that indicates the start of output of the output sentence, and words that constitute the second data to the decoder in accordance with the order of the words; an update unit that updates weighting coefficients that constitute the encoder and decoder based on a word-by-word error between the arrangement of words output from the decoder in a stage subsequent to the input of the start symbol and the arrangement of words included in the second data; and a model output unit that outputs the sentence generation model whose weighting coefficients have been updated by the update unit.

[0007] According to the above aspect, the sentence generation model is configured by an encoder-decoder model including an encoder and a decoder. In training of the sentence generation model in which first data corresponding to an input sentence is input to the encoder and second data corresponding to an output sentence is input to the decoder, the second data, i.e., a context including words related to the output sentence, is input to the decoder together with the second data. Therefore, the sentence generation model learns the association between the context and the second data, and it is possible to obtain a sentence generation model that outputs an output sentence in accordance with the condition of the output sentence set in the context. [Effects of the Invention]

[0008] It is possible to obtain an output sentence in a second language according to specific conditions in response to an input sentence in a first language. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a block diagram showing a functional configuration of a sentence generation model generation device according to an embodiment of the present invention. [Figure 2] FIG. 1 is a block diagram showing a functional configuration of a sentence generation device according to an embodiment of the present invention. [Figure 3] FIG. 2 is a hardware block diagram of a sentence generation model generation device and a sentence generation device. [Figure 4] FIG. 2 is a diagram illustrating a configuration of a sentence generation model. [Figure 5] FIG. 10 is a diagram illustrating an example of generation of first data, second data, and a context based on a corpus. [Figure 6] FIG. 10 is a diagram illustrating an example of generation of first data, second data, and a context based on a corpus. [Figure 7] FIG. 2 is a diagram illustrating an example of first data, second data, and context used in model training. [Figure 8] FIG. 1 is a diagram illustrating a schematic configuration of a Transformer, which is an example of an encoder-decoder model. [Figure 9] FIG. 2 is a diagram illustrating a sentence generation process using a sentence generation model. [Figure 10] FIG. 10 is a diagram illustrating a process for evaluating a created sentence in an evaluation system configured by a sentence generation device. [Figure 11] 10 is a flowchart showing the processing content of a sentence generation model generation method in the sentence generation model generation device. [Figure 12] 10 is a flowchart showing the processing content of a sentence generation method in the sentence generation device. [Figure 13] FIG. 13 is a diagram showing the configuration of a sentence generation model generation program. [Figure 14] FIG. 14 is a diagram showing the structure of the sentence generation program. DETAILED DESCRIPTION OF THE INVENTION

[0010] The following describes embodiments of a sentence generation model generating device, a sentence generation device, and a sentence generation model according to the present invention with reference to the drawings. Where possible, identical parts are designated by the same reference numerals, and redundant explanations will be omitted.

[0011] The sentence generation model of this embodiment is a model constructed by machine learning to cause a computer to function and generate an output sentence in a second language different from the first language in response to an input sentence in a first language. The sentence generation model includes a neural network and is configured by an encoder-decoder model including an encoder and a decoder.

[0012] The sentence generation model generation device of this embodiment is a device that generates a sentence generation model by machine learning. The sentence generation device is a device that generates an output sentence in a second language in response to an input sentence in a first language, using the sentence generation model constructed by machine learning.

[0013] An example of a problem that can be solved by the sentence generation model generation device, sentence generation model, and sentence generation device of this embodiment will be described.

[0014] In school education and other settings, students may be required to write English compositions using specific expressions (e.g., gerunds, infinitives, etc.). For example, when learning the "so that" construction, students are required to translate the Japanese sentence "We study English so that we can talk to foreigners." into English "We study English so that we can talk to foreigners." However, conventional translation and scoring engines often output the English translation "We study English to become able to speak with foreigners." and give it a high rating.

[0015] Furthermore, the translation of the answer "Yes, I am" to the Japanese question "Are you interested in history?" is "Yes, I am" if the English question is "Are you interested in history?", and "Yes, I do" if the English question is "Do you have an interest in history?". Therefore, conventional translation engines, which do not take the question into consideration, cannot uniquely determine the translation of the answer.

[0016] In this embodiment, it is possible to generate output sentences that meet desired conditions, and also possible to appropriately evaluate the created sentences based on the output sentences that meet the desired conditions.

[0017] FIG. 1 is a diagram showing the functional configuration of a sentence generation model generation device according to this embodiment. The sentence generation model generation device 10 is a device that generates, by machine learning, a sentence generation model that generates an output sentence in a second language different from an input sentence in a first language in response to the input of an input sentence in the first language. As shown in FIG. 1, the sentence generation model generation device 10 functionally includes a context generation unit 11, an encoder input unit 12, a decoder input unit 13, an update unit 14, and a model output unit 15. These functional units 11 to 15 may be configured in a single device or may be distributed across multiple devices.

[0018] Furthermore, the sentence generation model generation device 10 is configured to be able to access storage means such as a model storage unit 30 and a corpus storage unit 40. The model storage unit 30 and the corpus storage unit 40 may be configured within the sentence generation model generation device 10, or may be configured as separate devices external to the sentence generation model generation device 10 and accessible from the sentence generation model generation device, as shown in FIG.

[0019] The model storage unit 30 is a storage means for storing a sentence generation model that has already been learned or is in the process of being learned, and can be configured with a storage device, a memory, and the like.

[0020] The corpus storage unit 40 is a storage means that stores training data used in machine learning of the sentence generation model and corpora for generating training data, and can be composed of storage, memory, etc.

[0021] FIG. 2 is a diagram showing the functional configuration of a sentence generation device according to this embodiment. The sentence generation device 20 is a device that generates an output sentence in a second language different from the first language in response to an input sentence in a first language, using a sentence generation model constructed by machine learning. As shown in FIG. 2, the sentence generation device 20 functionally comprises an input unit 21, a context input unit 22, a word input unit 23, and an output unit 24. The sentence generation device 20 may further comprise a prepared sentence acquisition unit 25, a prepared sentence input unit 26, and a prepared sentence evaluation unit 27. These functional units 21 to 27 may be configured in a single device or may be distributed across multiple devices.

[0022] The sentence generation device 20 is also configured to be able to access a model storage unit 30 that stores a trained sentence generation model. The model storage unit 30 may be configured within the sentence generation device 20 or may be configured in a separate external device.

[0023] Furthermore, in this embodiment, an example is shown in which the sentence generation model generation device 10 and the sentence generation device 20 are configured as separate devices (computers), but they may also be configured as an integrated device.

[0024] The block diagrams shown in FIGS. 1 and 2 show functional blocks. These functional blocks (components) are realized by any combination of at least one of hardware and software. Furthermore, there are no particular limitations on the method for realizing each functional block. That is, each functional block may be realized using a single device that is physically or logically coupled, or may be realized using two or more physically or logically separated devices that are connected directly or indirectly (for example, by wire, wirelessly, etc.) and these multiple devices. A functional block may also be realized by combining software with the single device or multiple devices.

[0025] Functions include, but are not limited to, judgment, determination, judgment, calculation, computation, processing, derivation, investigation, search, confirmation, reception, transmission, output, access, resolution, selection, election, establishment, comparison, assumption, expectation, consideration, broadcasting, notifying, communicating, forwarding, configuring, reconfiguring, allocation, mapping, and assignment. For example, a functional block (component) that performs transmission is called a transmitting unit or transmitter. As mentioned above, there are no particular limitations on how these functions are implemented.

[0026] For example, the sentence generation model generation device 10 and the sentence generation device 20 according to an embodiment of the present invention may function as computers. Fig. 3 is a diagram showing an example of the hardware configuration of the sentence generation model generation device 10 and the sentence generation device 20 according to this embodiment. The sentence generation model generation device 10 and the sentence generation device 20 may each be physically configured as a computer device including a processor 1001, a memory 1002, a storage 1003, a communication device 1004, an input device 1005, an output device 1006, a bus 1007, etc.

[0027] In the following description, the term "apparatus" can be read as a circuit, a device, a unit, etc. The hardware configuration of the sentence generation model generation device 10 and the sentence generation device 20 may be configured to include one or more of the devices shown in the figures, or may be configured to exclude some of the devices.

[0028] Each function of the sentence generation model generation device 10 and the sentence generation device 20 is realized by loading predetermined software (programs) onto hardware such as a processor 1001 and a memory 1002, causing the processor 1001 to perform calculations and control communication via a communication device 1004 and the reading and / or writing of data in the memory 1002 and storage 1003.

[0029] The processor 1001 controls the entire computer by running, for example, an operating system. The processor 1001 may be configured as a central processing unit (CPU) including an interface with peripheral devices, a control device, an arithmetic unit, a register, etc. For example, the functional units 11 to 15, 21 to 27, etc. shown in FIGS. 1 and 2 may be realized by the processor 1001.

[0030] Furthermore, the processor 1001 reads programs (program codes), software modules, and data from the storage 1003 and / or the communication device 1004 into the memory 1002, and executes various processes in accordance with these. The programs used are those that cause a computer to execute at least some of the operations described in the above-described embodiments. For example, the functional units 11 to 15 and 21 to 27 of the sentence generation model generation device 10 and the sentence generation device 20 may be implemented by a control program stored in the memory 1002 and running on the processor 1001. Although the above-described various processes have been described as being executed by one processor 1001, they may also be executed simultaneously or sequentially by two or more processors 1001. The processor 1001 may be implemented on one or more chips. The programs may also be transmitted from a network via a telecommunications line.

[0031] The memory 1002 is a computer-readable recording medium and may be composed of at least one of, for example, a read-only memory (ROM), an erasable programmable ROM (EPROM), an electrically erasable programmable ROM (EEPROM), a random access memory (RAM), etc. The memory 1002 may also be called a register, a cache, a main memory (primary storage device), etc. The memory 1002 can store executable programs (program codes), software modules, etc. for implementing the sentence generation model generation method and sentence generation method according to one embodiment of the present invention.

[0032] Storage 1003 is a computer-readable recording medium, and may be, for example, at least one of an optical disk such as a CD-ROM (Compact Disc ROM), a hard disk drive, a flexible disk, a magneto-optical disk (e.g., a compact disk, a digital versatile disk, a Blu-ray disc), a smart card, a flash memory (e.g., a card, a stick, a key drive), a floppy disk, a magnetic strip, etc. Storage 1003 may also be referred to as an auxiliary storage device. The above-mentioned storage medium may be, for example, a database, a server, or other suitable medium including memory 1002 and / or storage 1003.

[0033] The communication device 1004 is hardware (transmission / reception device) for performing communication between computers via a wired and / or wireless network, and is also called, for example, a network device, a network controller, a network card, or a communication module.

[0034] The input device 1005 is an input device (for example, a keyboard, a mouse, a microphone, a switch, a button, a sensor, etc.) that receives input from the outside. The output device 1006 is an output device (for example, a display, a speaker, an LED lamp, etc.) that outputs to the outside. The input device 1005 and the output device 1006 may be integrated into one device (for example, a touch panel).

[0035] Furthermore, each device such as the processor 1001 and the memory 1002 is connected by a bus 1007 for communicating information. The bus 1007 may be configured as a single bus, or may be configured as different buses between the devices.

[0036] Furthermore, the sentence generation model generation device 10 and the sentence generation device 20 may be configured to include hardware such as a microprocessor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a programmable logic device (PLD), or a field programmable gate array (FPGA), and some or all of the functional blocks may be realized by the hardware. For example, the processor 1001 may be implemented by at least one of these pieces of hardware.

[0037] Fig. 4 is a diagram showing the configuration of a sentence generation model of this embodiment. As shown in Fig. 4, the sentence generation model MD is an encoder-decoder model including a neural network and configured by an encoder en and a decoder de. The neural network that configures the encoder-decoder model is not limited, but may be, for example, a recurrent neural network (RNN). The sentence generation model MD may also be a neural network called a Transformer.

[0038] The learning data used for machine learning of the sentence generation model MD of this embodiment includes first data a, second data b, and context c. The first data a includes an arrangement of multiple words that make up an input sentence in a first language. The second data b includes an arrangement of multiple words that make up an output sentence in a second language that corresponds to the input sentence. The output sentence is, for example, a bilingual translation of the input sentence. The context c includes one or more words in the second language that are related to the second data b.

[0039] The encoder en receives input of first data a constituting an input sentence in a first language. Specifically, the first data a is divided into words, for example, by morphological analysis or the like. Each divided word is converted (embedded) into a corresponding word vector and input to the encoder en according to the order of the words in the first data a (input sentence). The encoder en outputs a vector indicating a calculation result based on the first data a (for example, an output of a hidden layer, a source-target attention, etc.) to the decoder de.

[0040] In a typical encoder-decoder model, the decoder sequentially outputs a sequence of words based on input of a vector from the encoder and a predetermined start symbol (vector) indicating the start of output. In contrast, a context c is input to the decoder de of the sentence generation model MD of this embodiment before input of the start symbol ss. The decoder de outputs a sequence of words (vectors) of the output sentence t based on input of the output from the encoder en, the context c, and the start symbol ss. When the decoder de outputs a terminal symbol es indicating the end of the output sentence, the output sentence t is composed of the sequence of words output up to that point. In the learning phase, second data b corresponding to the output sentence (a translation of the input sentence in a second language) corresponding to the first data a (input sentence) is input to the decoder de word by word in the order of arrangement after input of the start symbol ss.

[0041] The context c includes one or more words in the second language related to the second data b. The generation of the context c will be described in detail later, but the context c may be configured to include one or more words that form part of the second data b. The context c may also be a question sentence in the second language, the answer sentence of which is an output sentence composed of words included in the second data b.

[0042] 1 again, the functional units of the sentence generation model generation device 10 will be described. The context generation unit 11 generates a context based on a corpus. The generation of a context and learning data including a context will be described with reference to FIGS.

[0043] 5 is a diagram showing an example of generation of first data, second data, and a context based on a corpus. The context generation unit 11 acquires a corpus cp0 from, for example, the corpus storage unit 40. The corpus cp0 is composed of a first sentence cp01 written in a first language and a second sentence cp02 written in a second language. In the example of FIG. 5, the first sentence cp01 is a Japanese sentence, and the second sentence cp02 is an English translation of the first sentence cp01.

[0044] The context generation unit 11 extracts a word cx that constitutes a part of the second sentence cp02 as a context. The context generation unit 11 may randomly extract the word cx from a plurality of words that constitute the second sentence cp02. Alternatively, the context generation unit 11 may extract the word cx based on a specified input that indicates a characteristic part of the second sentence cp02.

[0045] The context generation unit 11 generates a context c01 based on the extracted word cx. The context generation unit 11 also generates first data a01 and second data b01 in the training data based on the first sentence cp01 and the second sentence cp02, respectively, and generates training data consisting of the first data a01, the context c01, the start symbol ss, and the second data b01.

[0046] The context generation unit 11 may include, in the context c01, information indicating a relationship with the second data b01. In the example shown in Fig. 5, the context generation unit 11 includes, in the context c01, a symbol cl01 indicating that the word cx is a word to be used in the syntax of the second data b01 (output sentence).

[0047] FIG. 6 is a diagram illustrating an example of generation of first data, second data, and a context based on a corpus. The context generation unit 11 acquires a corpus cp1 from, for example, the corpus storage unit 40. The corpus cp1 includes a first sentence cp11 written in a first language, "What is the man going to do? He is going to take pictures of birds.", and a second sentence cp12 written in a second language, "What is the man going to do? He is going to take pictures of birds." In the example of FIG. 6, the first sentence cp11 is a Japanese sentence, and the second sentence cp12 is an English translation of the first sentence cp11. The first sentence, cp11, consists of the question, cpq1, "What is the man going to do?" and the answer, cpa1, "He is going to take pictures of birds." The second sentence, cp12, consists of the question, cpq2, "What is the man going to do?" and the answer, cpa2, "He is going to take pictures of birds."

[0048] The context generation unit 11 extracts, from the corpus, an answer sentence in the first language and its translation in the second language as first data and second data. In the example shown in Fig. 6, the context generation unit 11 extracts an answer sentence cpa1 in the first language and an answer sentence cpa2 in the second language as first data a02cpa1 "Tori no shashin wo torutsumoridesu" and second data b02 "He is going to take pictures of birds.", respectively.

[0049] Furthermore, the context generation unit 11 extracts the question sentence as a context based on a corpus including question sentences written in the second language and answer sentences to the question sentence. In the example shown in Fig. 6, the context generation unit 11 extracts the question sentence cpq2 from the corpus cp12 including the question sentence cpq2 and the answer sentence cpa2 written in the second language, and sets the extracted question sentence cpq2 as the context c02. Then, the context generation unit 11 generates training data including the first data a02, the context c02, the start symbol ss, and the second data b02.

[0050] The context generation unit 11 may include, in the context c02, information indicating a relationship with the second data b02. In the example shown in Fig. 6, the context generation unit 11 includes, in the context c02, a symbol cl02 indicating that the context c02 is a question sentence with the second data b02 as an answer sentence.

[0051] As described with reference to FIGS. 5 and 6, a context can be easily generated based on a corpus, which prevents an increase in the cost of obtaining learning data that includes a context.

[0052] 7 is a diagram showing examples of first data, second data, and context used in training a sentence generation model. The training data for the sentence generation model MD may include, as the first data, arbitrary symbols that are predetermined symbols without linguistic meaning, instead of an arrangement of multiple words that make up an input sentence.

[0053] 7, the training data may be composed of first data consisting of an arbitrary symbol a03 having no semantic content, a context c03 consisting of a question sentence in the second language, and b03 consisting of an answer sentence in the second language. Based on a corpus consisting of question sentences and answer sentences in the second language, the context generation unit 11 may extract the question sentence as the context c03, extract the answer sentence as the second data, and further add the arbitrary symbol a03 to generate the training data.

[0054] The context generation unit 11 includes in the context c03 a symbol cl03 indicating that the context c03 is a question sentence with the second data b03 as an answer sentence. The context generation unit 11 also includes a start symbol ss03 at the beginning of the second data b03. The start symbol ss03 may indicate the start of an answer sentence and also indicate that the answer sentence is an answer to the question sentence that constitutes the context.

[0055] 7, even if there is no first data corresponding to the input sentence that is a translation of the output sentence, the decoder can learn the relationship between the context and the second data. This allows for the expansion of training data at low cost and improves the accuracy of the output sentence output by the decoder relative to the desired output.

[0056] The encoder input unit 12 inputs the first data a to the encoder en in accordance with the arrangement order of the words.

[0057] The decoder input unit 13 inputs the context c, a start symbol ss which is a predetermined symbol indicating the start of output of an output sentence, and the second data b to the decoder de for each word in accordance with the arrangement order.

[0058] The update unit 14 updates the weighting coefficients constituting the encoder en and the decoder de based on the error for each word between the sequence of words output from the decoder de at the stage after the input of the start symbol ss and the sequence of words included in the second data b.

[0059] When the sentence generation model MD is configured by, for example, a recurrent neural network (RNN), the encoder input unit 12 sequentially inputs word vectors of the words that make up the first data a to the input layer of the RNN that makes up the encoder en in word order. The output of the intermediate layer of the encoder en, based on the input of the last word vector of the first data a, is output to the decoder de.

[0060] Next, the decoder input unit 13 sequentially inputs the word vectors of the words (including the symbol cl indicating the relationship with the second data) that make up the context c to the input layer of the RNN that makes up the decoder de in word order. Furthermore, the decoder input unit 13 sequentially inputs the start symbol ss and the second data b to the decoder de in word order. When the start symbol ss is input to the decoder de, the decoder de sequentially outputs a sequence of word vectors of the output sentence t together with likelihoods (for example, calculated by a softmax function).

[0061] The update unit 14 calculates the error between the sequence of words output from the decoder de and the sequence of words in the second data b for each word, and updates the weighting coefficients that make up the neural network of the encoder en and the decoder de, for example, by the error backpropagation method.

[0062] Fig. 8 is a diagram illustrating the schematic configuration of a transformer, which is an example of an encoder-decoder model. As shown in Fig. 8, when the sentence generation model MD1 (MD) is configured by a transformer, the encoder input unit 12 inputs word vectors aw11 to aw14 of the words that make up the first data a1 "Hai, Ari masu" to the input layer ila of the encoder en1 in accordance with the order of the words. The transformer allows parallel processing of input data, rather than sequential input of words as in an RNN.

[0063] In the encoder en1, self-attention sa1 for the hidden layer mla is calculated from the input layer ila, and word vectors are converted into vectors according to self-attention sa1. Similarly, self-attention sa2 for the output layer ola is calculated from the hidden layer mla, and word vectors are further converted. Furthermore, source-target attention ta for the input layer ilb of the decoder de1 is calculated from the output layer ola of the encoder en1.

[0064] The decoder input unit 13 inputs the word vectors cw11 to cw12 of the words that make up the context c1, the start symbol ss, and the word vectors bw11 to bw13 of the words that make up the second data b1 “Yes There is” in parallel to the input layer ilb of the decoder de1 in the learning phase according to the order of the words.

[0065] In the decoder de1, similar to the encoder en1, the self-attention sa3 from the input layer ilb to the hidden layer mlb is calculated, and the vector is transformed according to the self-attention sa3. Similarly, the self-attention sa4 from the hidden layer mlb to the output layer olb is calculated, and the vector is transformed according to the self-attention sa4.

[0066] The update unit 14 calculates the error for each word between the word sequence t11 to t13 based on the word vector wv output in the subsequent stage after the input of the start symbol ss and the word sequence bw11 to bw13 of the words that make up the second data b1, and updates the weighting coefficients for calculating self-attention and source-target attention using the error backpropagation method.

[0067] 1, the model output unit 15 outputs the sentence generation model MD obtained after machine learning based on a required amount of learning data. The model output unit 15 may store the sentence generation model MD in the model storage unit 30.

[0068] Next, processing of a sentence generation phase using the functional units of the sentence generation device 20 and the trained sentence generation model will be described with reference to Fig. 2 and Fig. 9. Fig. 9 is a diagram schematically showing the sentence generation processing using the sentence generation model.

[0069] 9, the sentence generation model MD2 is a model that is learned and constructed by the sentence generation model generation device 10. The sentence generation model MD2 includes an encoder en2 and a decoder de2.

[0070] The sentence generation model MD (MD1, MD2), which is a model including a trained neural network, can be considered as a program that is read or referenced by a computer and causes the computer to execute predetermined processes and realize predetermined functions.

[0071] That is, the trained sentence generation model MD (MD1, MD2) of this embodiment is used in a computer equipped with a CPU and memory. Specifically, the CPU of the computer operates in accordance with instructions from the trained sentence generation model MD (MD1, MD2) stored in the memory to perform calculations on input data input to the input layer of the neural network based on trained weight coefficients (parameters) and functions corresponding to each layer, and to output results (likelihoods) from the output layer.

[0072] The input unit 21 inputs the words aw21 to aw24 that make up the input data a2 "Hai, Ari masu" that make up the input sentence to the encoder en2 in the order of arrangement. The encoder en2 outputs the operation result to the decoder de2.

[0073] The context input unit 22 inputs words ct2, cw21 to cw26 that make up an input context c2 consisting of one or more words in a second language related to the output sentence to the decoder de2 in the order of arrangement. The word ct2 is a symbol that indicates the relationship of the context to the output sentence. The input context c2 "Is he interested in history?" is information for specifying the conditions and context of the output sentence, and may be, for example, a question sentence with the words to be included in the output sentence and the output sentence as the answer.

[0074] The word input unit 23 inputs a start symbol ss to the decoder de2 in a stage following the input of the input context c2. The decoder de2 outputs the initial word tw21 of the output sentence t2 in accordance with the start symbol ss. In each stage after the input of the start symbol ss, the word input unit 23 sequentially inputs the words output from the decoder de2 in the previous stage to the decoder de2. The decoder de2 sequentially outputs a sequence of words tw21 to tw24 that make up the output sentence t2 "Yes, he is" in accordance with the sequentially input words.

[0075] When a terminal symbol es indicating the end of output of an output sentence is output, the output unit 24 arranges the words tw21 to tw24 output sequentially at each stage of the decoder de2 to generate an output sentence t2. The output unit 24 then outputs the generated output sentence t2. The manner in which the output sentence t2 is output is not limited, and may be, for example, stored in a predetermined storage means, displayed on a display means, output by voice, or the like.

[0076] 2 and 10, an evaluation system for evaluating a created sentence, which is configured by the sentence generation device 20, will be described. FIG. 10 is a diagram showing the process of evaluating a created sentence in the evaluation system configured by the sentence generation device 20.

[0077] The decoder de3 shown in FIG. 10 outputs a likelihood indicating the likelihood of each of the words tw31 to tw34 output at each stage after the input of the start symbol ss as a word constituting the output sentence t3 for each word. The evaluation system configured by the sentence generation device 20 evaluates a created sentence created and input by a user, assuming that the output sentence t3 is correct. In response to the presentation of an input sentence corresponding to the output sentence t3, the user inputs, for example, a translation of the input sentence in a second language as a created sentence. Note that while this embodiment assumes that a created sentence input by a user is evaluated, a created sentence created and input by a person, device, or the like other than the user may also be evaluated.

[0078] The created sentence acquisition unit 25 acquires a created sentence r3 that is created by a user in a second language and input to the evaluation system. The created sentence r3 is made up of an array of words rw31 to rw34.

[0079] At each stage after the input of the start symbol ss, the created sentence input unit 26 sequentially inputs to the decoder de3 the word vectors of the words rw31 to rw34 that make up the created sentence r3 created in the second language, instead of the words tw31 to tw34 output from the decoder de3 in the previous stage.

[0080] The created sentence evaluation unit 27 evaluates the created sentence r3 based on the input of the start symbol ss and the sequential input of each word rw31 to rw34 that constitutes the created sentence r3, by comparing the likelihood of each word rw31 to rw34 output from the decoder de3 at each stage after the input of the start symbol ss with the likelihood of each word tw31 to tw34 that constitutes the output sentence t3.

[0081] Specifically, at each output stage, the decoder de3 outputs the likelihood of each word in the entire vocabulary handled by the sentence generation model generation device 10 and the sentence generation device 20. In the phase of the sentence generation process, an output sentence t3 is constructed by arranging the words with the highest likelihood at each output stage.

[0082] At each stage of the decoder de3, the created sentence evaluation unit 27 obtains the likelihood associated with each word rw31 to rw34 and the terminal symbol es of the created sentence r3 from the likelihood of each vocabulary output in response to the input of the start symbol ss and the words (rw31 to rw34) output in the previous stage.

[0083] The created sentence evaluation unit 27 calculates and outputs an evaluation value for the created sentence r3 by comparing the likelihood of each word tw31 to tw34 that make up the output sentence t3 with the likelihood of each word rw31 to rw34 of the created sentence r3. The method for calculating the evaluation value is not limited, but may be based on, for example, the ratio of the likelihoods of each word in each sentence t3 and r3, or the sum or average of the likelihoods of each sentence t3 and r3.

[0084] 10, the evaluation system evaluates the created sentence based on the comparison between the likelihood of each word constituting the output sentence and the likelihood of each word obtained by sequentially inputting the words constituting the created and input created sentence into a decoder. This makes it possible to configure an evaluation system that evaluates the likelihood of the created sentence as a translation corresponding to the input sentence.

[0085] FIG. 11 is a flowchart showing the processing content of the sentence generation model generation method in the sentence generation model generation device 10.

[0086] In step S1, the sentence generation model generation device 10 acquires training data including first data a, second data b, and context c. The training data may be data that has been generated in advance based on a corpus and stored in the corpus storage unit 40, or may be data that has been generated by the context generation unit 11 based on a corpus.

[0087] In step S2, the first data a is input to the encoder en in accordance with the arrangement order of the words.

[0088] In step S3, the decoder input unit 13 inputs the context c to the decoder de. Subsequently, in step S4, the decoder input unit 13 inputs the start symbol ss to the decoder de. Furthermore, in step S5, the decoder input unit 13 inputs the second data b to the decoder de word by word in accordance with the arrangement order.

[0089] In step S6, the update unit 14 calculates the error for each word between the sequence of words output from the decoder de at the stage after the input of the start symbol ss and the sequence of words included in the second data b, and updates the weighting coefficients constituting the encoder en and decoder de using the error backpropagation method.

[0090] In step S7, the update unit 14 determines whether machine learning based on the required amount of learning data has been completed. If it is determined that learning has been completed, the process proceeds to step S8. On the other hand, if it is not determined that learning has been completed, the processes of steps S1 to S6 are repeated.

[0091] In step S8, the model output unit 15 outputs the trained sentence generation model MD.

[0092] FIG. 12 is a flowchart showing the processing steps of the sentence generation method using the trained sentence generation model MD in the sentence generation device 20.

[0093] In step S11, the input unit 21 inputs the words of the input data that make up the input sentence to the encoder of the sentence generation model in the order of arrangement for each word. In response to the input of the input data, the encoder outputs the calculation result to the decoder.

[0094] In step S12, the context input unit 22 inputs the input context for specifying the conditions of the output sentence to the decoder for each word in accordance with the arrangement order. Then, in step S13, the word input unit 23 inputs the start symbol ss to the decoder after the input of the input context.

[0095] In step S14, the output unit 24 acquires a word (or symbol) output from the output layer of the decoder. In step S15, the output unit 24 determines whether the output from the decoder is a terminal symbol indicating the end of the output of the output sentence. If it is determined that the output from the decoder is a terminal symbol, the process proceeds to step S17. On the other hand, if it is not determined that the output from the decoder is a terminal symbol, the process proceeds to step S16.

[0096] In step S16, the word input unit 23 inputs the word output from the previous output layer of the decoder to the current input layer of the decoder, and the process then returns to step S14.

[0097] In step S17, the output unit 24 arranges the words sequentially output from the output layer at each stage of the decoder to generate an output sentence, and then in step S18, the output unit 24 outputs the output sentence.

[0098] Next, with reference to FIG. 13, a sentence generation model generation program for causing a computer to function as the sentence generation model generation device 10 of this embodiment will be described.

[0099] 13 is a diagram showing the configuration of a sentence generation model generation program. The sentence generation model generation program P1 is configured to include a main module m10 that comprehensively controls the sentence generation model generation process in the sentence generation model generation device 10, a context generation module m11, an encoder input module m12, a decoder input module m13, an update module m14, and a model output module m15. The modules m11 to m15 realize the functions of the context generation unit 11, the encoder input unit 12, the decoder input unit 13, the update unit 14, and the model output unit 15.

[0100] The sentence generation model generation program P1 may be transmitted via a transmission medium such as a communication line, or may be stored in a recording medium M1 as shown in FIG.

[0101] Next, with reference to FIG. 14, a sentence generation program for causing a computer to function as the sentence generation device 20 of this embodiment will be described.

[0102] 14 is a diagram showing the configuration of a sentence generation program. The sentence generation program P2 is configured to include a main module m20 that controls the sentence generation process in the sentence generation device 20 overall, an input module m21, a context input module m22, a word input module m23, and an output module m24. The sentence generation program P2 may further include a prepared sentence acquisition module m25, a prepared sentence input module m26, and a prepared sentence evaluation module m27. The modules m21 to m27 realize the functions of the input unit 21, the context input unit 22, the word input unit 23, the output unit 24, the prepared sentence acquisition unit 25, the prepared sentence input unit 26, and the prepared sentence evaluation unit 27.

[0103] The sentence generation program P2 may be transmitted via a transmission medium such as a communication line, or may be stored in a recording medium M2 as shown in FIG.

[0104] According to the sentence generation model generation device 10, sentence generation model generation method, and sentence generation model generation program P1 of the present embodiment described above, the sentence generation model is configured as an encoder-decoder model including an encoder and a decoder. In training of the sentence generation model in which first data corresponding to an input sentence is input to the encoder and second data corresponding to an output sentence is input to the decoder, the second data, i.e., a context including words related to the output sentence, is input to the decoder together with the second data. Therefore, the sentence generation model learns the association between the context and the second data, and it is possible to obtain a sentence generation model that outputs an output sentence in accordance with the conditions for the output sentence set in the context.

[0105] In a sentence generation model generation device according to another aspect, the context may include one or more words that form part of the second data.

[0106] According to the above aspect, the context includes the words to be used in the output sentence, so that it is possible to generate a sentence generation model that can output an output sentence including the words to be used.

[0107] In addition, a sentence generation model generating device according to another embodiment may further include a context generation unit that extracts, as context, words that constitute part of a second sentence based on a corpus consisting of a first sentence written in a first language and a second sentence that is a translation of the first sentence written in a second language.

[0108] According to the above aspect, it is possible to obtain, as learning data, a context for specifying, as a condition, a word to be used in an output sentence, based on a corpus.

[0109] In a sentence generation model generation device according to another aspect, the context may be a question sentence in the second language, with an output sentence composed of words included in the second data as an answer sentence.

[0110] According to the above aspect, the context includes a question sentence with the output sentence as an answer sentence, so that it is possible to generate a sentence generation model that can output an answer sentence as an output sentence that is in line with the context of the question sentence.

[0111] In addition, a sentence generation model generating device according to another embodiment may further include a context generation unit that extracts a question sentence as a context based on a corpus including question sentences written in a second language and answer sentences to the question sentences.

[0112] According to the above aspect, it is possible to obtain, as learning data, a context for specifying, as a condition, a context to be followed in an output sentence, based on a corpus.

[0113] In addition, in another embodiment of the sentence generation model generating device, the first data may be an arbitrary symbol, which is a predetermined symbol that does not have linguistic meaning, instead of an arrangement of multiple words that make up the input sentence.

[0114] According to the above embodiment, even if there is no first data corresponding to the input sentence that is a translation of the output sentence, the decoder can learn the relationship between the context and the second data. Therefore, it is possible to expand the training data at low cost and improve the accuracy of the output sentence output by the decoder relative to the desired output.

[0115] In the sentence generation model generation device according to another aspect, the context may include information indicating a relationship with the second data.

[0116] According to the above aspect, the decoder can learn how to use the conditions specified in the context, thereby improving the accuracy of the output sentences output by the decoder relative to the desired output.

[0117] In order to solve the above problem, one embodiment of the present invention provides a sentence generation model that is trained by machine learning to cause a computer to function and generate an output sentence in a second language different from the first language in response to an input of an input sentence in a first language. The learning data used for the machine learning of the sentence generation model includes first data including an arrangement of multiple words that make up the input sentence, second data including an arrangement of multiple words that make up an output sentence corresponding to the input sentence, and a context including one or more words in the second language that are related to the second data. The sentence generation model is an encoder-decoder model that includes a neural network and is composed of an encoder and a decoder. The first data is input to the encoder in accordance with the order of the words, and the context, a start symbol that is a predetermined symbol that indicates the start of output of the output sentence, and the second data are input to the decoder in accordance with the order of the words and symbols in the context, the start symbol, and the second data. The sentence generation model is constructed by machine learning that updates weight coefficients that make up the encoder and decoder based on the error for each word between the word arrangement output from the decoder in a stage subsequent to the input of the start symbol and the word arrangement included in the second data.

[0118] According to the above-mentioned method, the sentence generation model is configured as an encoder-decoder model including an encoder and a decoder. In training the sentence generation model, first data corresponding to an input sentence is input to the encoder, second data corresponding to an output sentence is input to the decoder, and the second data, i.e., a context including words related to the output sentence, is input to the decoder together with the second data. Therefore, the association between the context and the second data is learned, and the sentence generation model can output an output sentence in accordance with the conditions for the output sentence set in the context.

[0119] In order to solve the above problem, a sentence generation device according to one aspect of the present invention is a sentence generation device that generates an output sentence in a second language different from the first language in response to an input of an input sentence in a first language using a sentence generation model constructed by machine learning, wherein learning data used for the machine learning of the sentence generation model includes first data including an arrangement of multiple words corresponding to the input sentence, second data including an arrangement of multiple words corresponding to an output sentence corresponding to the input sentence, and a context including one or more words in the second language related to the second data, and the sentence generation model is an encoder-decoder model including a neural network and configured by an encoder and a decoder, and the first data is input to the encoder in accordance with the order of the words, and the encoder generates an output sentence in accordance with the order of the words, a start symbol which is a predetermined symbol indicating the start of output of the output sentence, and the second data includes the context, the start symbol, and the second data. The sentence generation device is constructed using machine learning that updates weight coefficients that configure the encoder and decoder based on the error for each word between the word sequence output from the decoder in a subsequent stage after the input of a start symbol and the word sequence included in the second data, and is equipped with an input unit that inputs input data that configures the input sentence to the encoder in the word sequence, a context input unit that inputs an input context consisting of one or more words in the second language related to the output sentence to the decoder, a word input unit that inputs a start symbol to the decoder in a stage after the input of the input context, and at each stage after the input of the start symbol, sequentially inputs words that were output from the decoder in the previous stage to the decoder, and an output unit that generates an output sentence by arranging the words that were sequentially output at each stage of the decoder, and outputs the generated output sentence.

[0120] According to the above aspect, the sentence generation model is configured as an encoder-decoder model including an encoder and a decoder. In training the sentence generation model, first data corresponding to an input sentence is input to the encoder, second data corresponding to an output sentence is input to the decoder, and the second data, i.e., a context including words related to the output sentence, is input to the decoder together with the second data. As a result, the trained sentence generation model learns the association between the context and the second data. Therefore, by inputting input data constituting the input sentence to the encoder and inputting input context for specifying conditions for the output sentence to the decoder, it is possible to output an output sentence in accordance with desired conditions.

[0121] In another embodiment of the sentence generation device, the decoder outputs a likelihood indicating the likelihood of each word output at each stage after the input of the start symbol as a word constituting the output sentence, and the sentence generation device further includes a created sentence input unit that, at each stage after the input of the start symbol, sequentially inputs words constituting a created sentence created in the second language into the decoder in place of the words output from the decoder at the previous stage, and a created sentence evaluation unit that evaluates the created sentence based on the input of the start symbol and the sequential input of each word constituting the created sentence, comparing the likelihood of each word constituting the created sentence output from the decoder at each stage after the input of the start symbol with the likelihood of each word constituting the output sentence.

[0122] According to the above embodiment, the created sentence is evaluated based on a comparison between the likelihood of each word constituting the output sentence and the likelihood of each word obtained by sequentially inputting the words constituting the created and input created sentence into the decoder. This makes it possible to configure an evaluation system that evaluates the likelihood of the created sentence as a translation corresponding to the input sentence.

[0123] Although the present embodiment has been described in detail above, it is clear to those skilled in the art that the present embodiment is not limited to the embodiment described in this specification. The present embodiment can be implemented in modified and altered forms without departing from the spirit and scope of the present invention as defined by the claims. Therefore, the description in this specification is intended to be illustrative and does not have any limiting meaning on the present embodiment.

[0124] Each aspect / embodiment described herein may be applied to systems utilizing LTE (Long Term Evolution), LTE-Advanced (LTE-A), SUPER 3G, IMT-Advanced, 4G, 5G, FRA (Future Radio Access), W-CDMA (registered trademark), GSM (registered trademark), CDMA2000, UMB (Ultra Mobile Broadband), IEEE 802.11 (Wi-Fi), IEEE 802.16 (WiMAX), IEEE 802.20, UWB (Ultra-WideBand), Bluetooth (registered trademark), or other suitable systems and / or next generation systems enhanced thereon.

[0125] The order of the procedures, sequences, flowcharts, etc. of each aspect / embodiment described herein may be changed unless it is consistent. For example, the methods described herein present elements of various steps in an example order and are not limited to the particular order presented.

[0126] Input and output information may be stored in a specific location (for example, memory) or managed in a management table. Input and output information may be overwritten, updated, or added to. Output information may be deleted. Input information may be sent to another device.

[0127] The determination may be made based on a value represented by one bit (0 or 1), a Boolean value (true or false), or a numerical comparison (e.g., comparison with a predetermined value).

[0128] Each aspect / embodiment described in this specification may be used alone or in combination, or may be switched depending on the implementation. Furthermore, notification of predetermined information (e.g., notification that "X is true") is not limited to being done explicitly, but may be done implicitly (e.g., by not notifying the predetermined information).

[0129] Although the present disclosure has been described in detail above, it is clear to those skilled in the art that the present disclosure is not limited to the embodiments described herein. The present disclosure can be implemented in modified and altered forms without departing from the spirit and scope of the present disclosure as defined by the claims. Therefore, the description of the present disclosure is intended to be illustrative and does not have any limiting meaning on the present disclosure.

[0130] Software shall be construed broadly to mean instructions, instruction sets, code, code segments, program code, programs, subprograms, software modules, applications, software applications, software packages, routines, subroutines, objects, executable files, threads of execution, procedures, functions, etc., whether referred to as software, firmware, middleware, microcode, hardware description language, or otherwise.

[0131] Software, instructions, etc. may also be transmitted or received over a transmission medium. For example, if software is transmitted from a website, server, or other remote source using wired technologies such as coaxial cable, fiber optic cable, twisted pair, and Digital Subscriber Line (DSL), and / or wireless technologies such as infrared, radio, and microwave, these wired and / or wireless technologies are included within the definition of transmission media.

[0132] The information, signals, etc. described in this disclosure may be represented using any of a variety of different technologies. For example, data, instructions, commands, information, signals, bits, symbols, chips, etc. that may be referred to throughout the above description may be represented by voltages, currents, electromagnetic waves, magnetic fields or magnetic particles, optical fields or photons, or any combination thereof.

[0133] It should be noted that terms explained in this disclosure and / or terms necessary for understanding this specification may be replaced with terms having the same or similar meanings.

[0134] As used herein, the terms "system" and "network" are used interchangeably.

[0135] Furthermore, the information, parameters, etc. described in this specification may be expressed as absolute values, as relative values ​​from a predetermined value, or as corresponding other information.

[0136] As used in this disclosure, the terms "determining" and "determining" may encompass a wide variety of actions. "Determining" and "determining" may include, for example, judging, calculating, computing, processing, deriving, investigating, looking up, searching, inquiring (e.g., searching in a table, database, or other data structure), ascertaining, and the like. "Determining" and "determining" may also include receiving (e.g., receiving information), transmitting (e.g., sending information), input, output, accessing (e.g., accessing data in memory), and the like. Furthermore, "judgment" and "decision" can include regarding resolving, selecting, choosing, establishing, comparing, etc. as having been "judged" or "decided." In other words, "judgment" and "decision" can include regarding some action as having been "judged" or "decided." Furthermore, "judgment (decision)" can be interpreted as "assuming," "expecting," "considering," etc.

[0137] As used in this disclosure, the phrase "based on" does not mean "based only on," unless expressly specified otherwise. In other words, the phrase "based on" means both "based only on" and "based at least on."

[0138] When designations such as "first," "second," etc. are used herein, any reference to such elements does not generally limit the quantity or order of those elements. These designations may be used herein as a convenient method of distinguishing between two or more elements. Thus, a reference to a first and a second element does not imply that only two elements may be employed therein or that the first element must precede the second element in some way.

[0139] To the extent that the terms "include," "including," and variations thereof are used herein or in the claims, these terms are intended to be inclusive, similar to the term "comprising." Furthermore, the term "or," as used herein or in the claims, is not intended to be an exclusive or.

[0140] In this specification, plural devices are also included unless the context or the technology clearly indicates that only one device is present.

[0141] Throughout this disclosure, the plural is intended to be included unless the singular is clearly indicated by the context. [Explanation of symbols]

[0142] 10... sentence generation model generation device, 11... context generation unit, 12... encoder input unit, 13... decoder input unit, 14... update unit, 15... model output unit, 20... sentence generation device, 21... input unit, 22... context input unit, 23... word input unit, 24... output unit, 25... created sentence acquisition unit, 26... created sentence input unit, 27... created sentence evaluation unit, 30... model storage unit, 40... corpus storage unit, de, de1, de2, de3... decoders, en, en1, en2... encoders, M1... recording medium, m10... main module, m11... context generation model module, m12...encoder input module, m13...decoder input module, m14...update module, m15...model output module, M2...recording medium, m20...main module, m21...input module, m22...context input module, m23...word input module, m24...output module, m25...created sentence acquisition module, m26...created sentence input module, m27...created sentence evaluation module, MD, MD1, MD2...sentence generation model, P1...sentence generation model generation program, P2...sentence generation program.

Claims

1. 1. A sentence generation model generation device that generates, by machine learning, a sentence generation model that generates an output sentence in a second language different from an input sentence in a first language in response to the input sentence in the first language, the sentence generation model is an encoder-decoder model including a neural network and configured with an encoder and a decoder, learning data used in machine learning of the sentence generation model includes first data, a context, and second data; the first data includes an arrangement of a plurality of words that constitute the input sentence; the second data includes an arrangement of a plurality of words constituting the output sentence corresponding to the input sentence, the context includes one or more words in a second language associated with the second data; The sentence generation model generation device an encoder input unit that inputs the first data to the encoder in accordance with the order of words; a decoder input unit that inputs the context, a start symbol that is a predetermined symbol indicating the start of output of the output sentence, and each word and symbol that constitutes the second data to the decoder in an order that follows the arrangement order of the context, the start symbol, and the words and symbols of the second data; an update unit that updates weighting coefficients constituting the encoder and decoder based on a word-by-word error between a word sequence output from the decoder at a stage subsequent to the input of the start symbol and a word sequence included in the second data; a model output unit that outputs the sentence generation model in which the weight coefficients have been updated by the update unit; A sentence generation model generating device comprising:

2. the context includes one or more words that form part of the second data; The sentence generation model generating device according to claim 1 .

3. a context generation unit that extracts, as the context, words that constitute a part of the second sentence based on a corpus including a first sentence written in the first language and a second sentence that is a translation of the first sentence written in a second language. The sentence generation model generating device according to claim 2 .

4. the context is a question sentence in the second language, the answer sentence of which is the output sentence configured with words included in the second data; The sentence generation model generating device according to claim 1 .

5. a context generation unit that extracts the question sentence as a context based on a corpus that includes question sentences written in the second language and answer sentences to the question sentences, The sentence generation model generating device according to claim 4.

6. the first data is an arbitrary symbol that is a predetermined symbol that does not have linguistic meaning, instead of an arrangement of a plurality of words that constitute the input sentence; The sentence generation model generating device according to any one of claims 1 to 5.

7. the context includes information indicating a relationship with the second data; The sentence generation model generating device according to any one of claims 1 to 6.

8. A machine learning trained sentence generation model for causing a computer to function and generate an output sentence in a second language different from an input sentence in a first language, the machine learning trained sentence generation model comprising: the learning data used for the machine learning of the sentence generation model includes first data including an arrangement of a plurality of words constituting the input sentence, second data including an arrangement of a plurality of words constituting the output sentence corresponding to the input sentence, and a context including one or more words in a second language related to the second data; The sentence generation model is An encoder-decoder model including a neural network and configured with an encoder and a decoder, the first data is input to the encoder in accordance with the order of words; the context, a start symbol which is a predetermined symbol indicating the start of output of the output sentence, and each word and symbol constituting the second data are input to the decoder in an order that follows the arrangement order of the context, the start symbol, and the words and symbols of the second data; constructed by machine learning that updates weight coefficients constituting the encoder and decoder based on an error for each word between a sequence of words output from the decoder at a later stage after input of the start symbol and a sequence of words included in the second data; A trained sentence generation model.

9. 1. A sentence generation device that generates an output sentence in a second language different from a first language in response to an input sentence in a first language, using a sentence generation model constructed by machine learning, the learning data used for the machine learning of the sentence generation model includes first data including a sequence of a plurality of words corresponding to the input sentence, second data including a sequence of a plurality of words corresponding to the output sentence corresponding to the input sentence, and a context including one or more words in a second language related to the second data; The sentence generation model is An encoder-decoder model including a neural network and configured with an encoder and a decoder, the first data is input to the encoder in accordance with the order of words; the context, a start symbol which is a predetermined symbol indicating the start of output of the output sentence, and each word and symbol constituting the second data are input to the decoder in an order that follows the arrangement order of the context, the start symbol, and the words and symbols of the second data; constructed by machine learning that updates weight coefficients constituting the encoder and decoder based on an error for each word between a sequence of words output from the decoder at a later stage after input of the start symbol and a sequence of words included in the second data, The sentence generation device an input unit that inputs input data constituting the input sentence to the encoder in accordance with the order of words; a context input unit for inputting an input context to the decoder, the input context comprising one or more words in the second language that are related to the output sentence; a word input unit that inputs the start symbol to the decoder at a stage after the input of the input context, and at each stage after the input of the start symbol, sequentially inputs words output from the decoder at the previous stage to the decoder; an output unit that arranges the words sequentially output at each stage of the decoder to generate the output sentence and outputs the generated output sentence; A sentence generation device comprising:

10. the decoder outputs a likelihood indicating the likelihood of each word being a word constituting the output sentence for each word output at each stage after the input of the start symbol; The sentence generation device a created sentence input unit that, in each stage after the input of the start symbol, sequentially inputs words constituting a created sentence created in the second language into the decoder, instead of the words output from the decoder in the previous stage; and a created sentence evaluation unit that evaluates the created sentence based on input of the start symbol and sequential input of each word that constitutes the created sentence, by comparing the likelihood of each word that constitutes the created sentence output from the decoder at each stage after input of the start symbol with the likelihood of each word that constitutes the output sentence. The sentence generation device according to claim 9.

Images