Sentence generation device, sentence generation learning device, sentence generation method, sentence generation learning method, and program

Abstract

To enable information in need of consideration when a sentence is generated to be added as a text.SOLUTION: A sentence generation device includes a content selection unit and a generation unit. The content selection unit extracts a set of words based on an input sentence, a degree of importance of each word included in the input sentence, and an output length. The generation unit generates an output sentence based on the input sentence and the set of words by, with the input sentence and the set of words as input, inputting information corresponding to the input sentence and the set of words into a machine learning model in which supervised learning has been performed so as to generate the output sentence based on the input sentence and the set of words.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a sentence generation device, a sentence generation learning device, a sentence generation method, a sentence generation learning method, and a program. [Background technology]

[0002] Neural network-based sentence generation technology is advancing. This technology takes text as input and generates a predetermined sentence (e.g., a summary sentence) based on a pre-trained neural network model.

[0003] For example, Non-Patent Document 1 proposes a method of reflecting the importance of a word level, which is calculated by multiplying the attention of a sentence included in an input document by the importance of a word, when generating a sentence. [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] Wan-Ting Hsu, Chieh-Kai Lin, Ming-Ying Lee, Kerui Min, Jing Tang, Min Sun. 2018. A Unified Model for Extractive and Abstractive Summarization using Inconsistency Loss. Summary of the Invention [Problem to be solved by the invention]

[0005] However, in Non-Patent Document 1, the information to be considered when generating a sentence is given in the form of an importance score, length embedding, or the like, and cannot be given as text.

[0006] The present invention has been made in view of the above points, and has as its object to make it possible to add information that should be taken into consideration when generating a sentence as text. [Means for solving the problem]

[0007] In order to solve the above problem, the sentence generation device has a content selection unit that extracts a set of words based on an input sentence, the importance of each word included in the input sentence, and an output length, and a generation unit that generates an output sentence based on the input sentence and the set of words by inputting the input sentence and the set of words and inputting information corresponding to the input sentence and the set of words into a machine learning model that has been supervised trained to generate an output sentence based on the input sentence and the set of words. [Effects of the Invention]

[0008] It is possible to add information to be taken into consideration when generating a sentence as text. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a diagram illustrating an example of a hardware configuration of a sentence generation device 10 according to a first embodiment. [Figure 2] 1 is a diagram illustrating an example of a functional configuration of a sentence generation device 10 according to a first embodiment. [Figure 3] FIG. 2 is a diagram illustrating an example of the configuration of a generation unit 12 according to the first embodiment. [Figure 4] 4 is a flowchart illustrating an example of a processing procedure executed by the sentence generation device 10 according to the first embodiment. [Figure 5] FIG. 10 is a diagram for explaining estimation of importance for each word. [Figure 6] FIG. 2 is a diagram for explaining processing by a generation unit 12 in the first embodiment. [Figure 7] FIG. 2 is a diagram illustrating an example of a functional configuration of the sentence generation device 10 during learning in the first embodiment. [Figure 8] FIG. 10 is a diagram illustrating an example of the configuration of a generation unit 12 according to a second embodiment. [Figure 9]FIG. 10 is a diagram for explaining processing by a generation unit 12 in the second embodiment. [Figure 10] FIG. 11 is a diagram illustrating an example of a functional configuration of a sentence generation device 10 according to a third embodiment. [Figure 11] 10 is a flowchart illustrating an example of a processing procedure executed by the sentence generation device 10 according to the third embodiment. [Figure 12] FIG. 2 is a diagram illustrating an example of the configuration of a knowledge source DB 20. [Figure 13] FIG. 2 is a diagram for explaining a first example of a degree of association calculation model. [Figure 14] FIG. 10 is a diagram for explaining a second example of the relevance calculation model. [Figure 15] FIG. 11 is a diagram illustrating an example of a functional configuration of a sentence generation device 10 during learning in a third embodiment. [Figure 16] FIG. 13 is a diagram illustrating an example of a functional configuration of a sentence generation device 10 according to a fourth embodiment. [Figure 17] FIG. 13 is a diagram illustrating an example of the configuration of a generation unit 12 according to a fourth embodiment. [Figure 18] FIG. 13 is a diagram illustrating an example of a model configuration according to a fourth embodiment. [Figure 19] FIG. 13 is a diagram illustrating an example of a functional configuration of a sentence generation device 10 during learning in a fourth embodiment. [Figure 20] FIG. 13 is a diagram illustrating an example of a functional configuration of a sentence generation device 10 according to a fifth embodiment. [Figure 21] FIG. 13 is a diagram illustrating an example of the configuration of a generation unit 12 according to a fifth embodiment. [Figure 22] FIG. 13 is a diagram illustrating an example of a model configuration according to a fifth embodiment. [Figure 23] FIG. 20 is a diagram illustrating an example of a functional configuration of a sentence generation device 10 according to a sixth embodiment. [Figure 24] FIG. 20 is a diagram illustrating an example of a model configuration according to the sixth embodiment. [Figure 25] FIG. 22 is a diagram illustrating an example of a functional configuration of a sentence generation device 10 during learning in a sixth embodiment. [Figure 26]FIG. 20 is a diagram illustrating an example of a functional configuration of a sentence generation device 10 according to a seventh embodiment. [Figure 27] FIG. 22 is a diagram illustrating an example of a functional configuration of a sentence generation device 10 during learning in a seventh embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments of the present invention will be described with reference to the drawings. Fig. 1 is a diagram showing an example of the hardware configuration of a sentence generation device 10 in a first embodiment. The sentence generation device 10 in Fig. 1 has a drive device 100, an auxiliary storage device 102, a memory device 103, a CPU 104, an interface device 105, etc., which are connected to each other via a bus B.

[0011] The program that realizes the processing in the sentence generation device 10 is provided by a recording medium 101 such as a CD-ROM. When the recording medium 101 storing the program is set in the drive device 100, the program is installed from the recording medium 101 to the auxiliary storage device 102 via the drive device 100. However, the program does not necessarily have to be installed from the recording medium 101, but may be downloaded from another computer via a network. The auxiliary storage device 102 stores the installed program as well as necessary files, data, etc.

[0012] When an instruction to start a program is received, the memory device 103 reads the program from the auxiliary storage device 102 and stores it. The CPU 104 executes functions related to the sentence generation device 10 in accordance with the program stored in the memory device 103. The interface device 105 is used as an interface for connecting to a network.

[0013] The sentence generation device 10 may include a GPU (Graphics Processing Unit) instead of the CPU 104 or in addition to the CPU 104.

[0014] Fig. 2 is a diagram showing an example of the functional configuration of the sentence generation device 10 in the first embodiment. In Fig. 2, the sentence generation device 10 has a content selection unit 11 and a generation unit 12. These units are realized by processing in which one or more programs installed in the sentence generation device 10 are executed by the CPU 104 or the GPU.

[0015] A source text (input sentence) and information different from the input sentence (conditions or information to be considered in summarizing the source text (hereinafter referred to as "consideration information")) are input as text to the sentence generation device 10. In the first embodiment, an example will be described in which the length (number of words) K of the sentence (summary sentence) generated by the sentence generation device 10 based on the source text (hereinafter referred to as "output length K") is adopted as the consideration information.

[0016] The content selection unit 11 estimates the importance [0,1] for each word that makes up the source text. The content selection unit 11 extracts a predetermined number of words (those with the top K importance) based on the output length K, and outputs the result of concatenating the extracted words as reference text. Note that importance refers to the probability that a word will be included in the summary.

[0017] The generation unit 12 generates a target text (summary) based on the source text and the reference text output from the content selection unit 11.

[0018] The content selection unit 11 and the generation unit 12 are based on neural networks that perform a sentence generation task (summarization in this embodiment). Specifically, the content selection unit 11 is based on BERT (Bidirectional Encoder Representations from Transformers), and the generation unit 12 is based on the Transformer-based pointer generator model "Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems 30, pages 5998-6008." (hereinafter referred to as "Reference 1"). Therefore, the processing by the content selection unit 11 and the generation unit 12 is performed based on learned values ​​(learned parameters) of learning parameters for the neural network.

[0019] Fig. 3 is a diagram showing an example of the configuration of the generation unit 12 in the first embodiment. As shown in Fig. 3, the generation unit 12 includes a source text encoding unit 121, a reference text encoding unit 122, a decoding unit 123, and a synthesis unit 124. The function of each unit will be described later.

[0020] The following describes the processing procedure executed by the sentence generation device 10. Fig. 4 is a flowchart for explaining an example of the processing procedure executed by the sentence generation device 10 in the first embodiment.

[0021] In step S101, the content selection unit 11 selects the source text X C For each word in the list, estimate (calculate) the importance.

[0022] In this embodiment, the content selector 11 uses BERT (Bidirectional Encoder Representations from Transformers). BERT has achieved SOTA (State-of-the-Art) in many sequence tagging tasks. In this embodiment, the content selector 11 segments the source text into words using a BERT tokenizer, a fine-tuned BERT model, and a task-specific feed forward network. The content selector 11 segments the source text into words using the following formula: C n Importance of p ext n Calculate p ext n is the source text X C The nth word in x C n Indicates the importance of.

[0023]

number

[0024]

number

[0025] 5 is a diagram for explaining the estimation of the importance of each word. In FIG. 5, C In this example, the content selection unit 11 calculates the importance p of each of the N words. ext n Calculate.

[0026] Next, the content selection unit 11 selects the importance p extn A set of K words (word string) is extracted in descending order of their length (S102), where K is the output length as described above. The extracted word string is output to the generation unit 12 as a reference text.

[0027] In step S101, the importance is calculated by p in the following formula (2): extw n In this case, p extw n A set of K words (word strings) is extracted as reference text in descending order of the word size.

[0028]

number

[0029] Regardless of whether formula (1) or formula (2) is used, this embodiment makes it possible to control the length of the summary sentence depending on the number of words in the reference text.

[0030] Next, the generation unit 12 generates a reference text and a source text X C A summary is generated based on the above (S103).

[0031] Details of step S103 will be explained below. Fig. 6 is a diagram for explaining the process by the generation unit 12 in the first embodiment.

[0032] [Source text encoding unit 121] The source text encoding unit 121 encodes the source text X C Received

[0033]

number

[0034] The embedding layer of the source text encoder 121 embeds each word x C n The one-hot vectors (of size V) are fed to a pre-trained weight matrix such as Glove (Jeffrey Pennington, Richard Socher, and Christopher D. Manning. 2014. Glove: Global vectors for word representation. In EMNLP. (hereinafter referred to as "Reference 2")).

[0035]

number

[0036] Then the embedding layer uses a fully connected layer to word Let d be the dimension of the word embedding. model The embedding layer maps the word embeddings to vectors of dimensions and passes the mapped embeddings to a ReLU function. The embedding layer also adds a positional encoding to the word embeddings (Reference 1).

[0037] The Transformer Endcoder Block of the source text encoder 121 has the same structure as that in Reference 1. The Transformer Endcoder Block consists of a multi-head self-attention network and a fully connected feed forward network. Each network applies residual connections.

[0038] [Reference text coding part 122] The reference text encoding unit 122 encodes the reference text X, which is a string of the top K words along with their respective importance. p Reference text X p The order of words in is rearranged to the order in which they appear in the source text. The output from the reference text coding unit 122 is as follows:

[0039]

number

[0040] The Transformer Decoder Block of the reference text encoder 122 is almost the same as that in Reference 1. In addition to two sublayers for each encoder layer, the reference text encoder 122 has an interactive alignment layer that performs multi-head attention on the output of the encoder stack. Residual connections are applied in the same way as the Transformer Endcoder Block of the source text encoder 121.

[0041] [Decoding unit 123] The decoding unit 123 p and the word sequence of the summary sentence Y generated as an autoregressive process. p t is used as a guide vector for generating a summary sentence. The output from the decoding unit 123 is as follows:

[0042]

number

[0043] The embedding layer of the decoding unit 123 uses a pre-trained weight matrix W e tUsing the above, we can find the t-th word y in the summary sentence Y. t M y t The embedding layer maps to M y t and M p t and then hand it over to a highway network (Rupesh Kumar Srivastava, Klaus Greff, and Jurgen Schmidhuber. 2015. Highway networks. CoRR, 1505.00387.). The concatenated embedding is thus

[0044]

number

[0045] The Transformer Decoder Block of the decoding unit 123 has the same structure as that in Reference 1. This component is used in stages during testing, so subsequent masks are used.

[0046] [Synthesis section 124] The synthesis unit 124 uses a pointer generator to select either the information from the source text or the information from the decoding unit 123 based on the distribution of copies, and generates a summary based on the selected information.

[0047] In this embodiment, the copy distribution is the first attention head of the decoding unit 123. Therefore, the final vocabulary distribution is as follows:

[0048]

number

[0049]

number

[0050]

number

[0051] Next, learning will be described. Fig. 7 is a diagram showing an example of the functional configuration of the sentence generation device 10 in the first embodiment during learning. In Fig. 7, the same parts as in Fig. 3 are given the same reference numerals, and their description will be omitted.

[0052] During learning, the sentence generation device 10 further includes a parameter learning unit 13. The parameter learning unit 13 is realized by processing in which one or more programs installed in the sentence generation device 10 are executed by the CPU 104 or the GPU.

[0053] [Learning data for content selection section 11] For example, pseudo-training data such as "Sebastian Gehrmann, Yuntian Deng, and Alexander Rush. 2018. Bottom-up abstractive summarization. In EMNLP, pages 4098-4109." (hereinafter referred to as "Reference 3") is used as training data. The training data is a set of all source texts X. C n The word x C n and label r n Pair (x C n ,r n ) in the summary C n If is selected, r n is 1. To automatically generate this pair of data, first, we generate the source sentence S of the oracle that maximizes the ROUGE-R score in the same way as in "Wan-Ting Hsu, Chieh-Kai Lin, Ming-Ying Lee, Kerui Min, Jing Tang, and Min Sun. 2018. A unified model for extractive and abstractive summarization using inconsistency loss. In ACL (1), pages 132-141." oracle Then, a dynamic programming algorithm is used to extract the reference summary and S oracle Finally, all aligned words are labeled as 1, and all other words are labeled as 0.

[0054] [Learning data for generation unit 12] For the training of the generator 12, three sets of data (X C ,X p , Y) is generated. Specifically, the content selection unit 11 is used to generate the oracle statement S oracle Select and press S oracleAll words in x C n For p ext n Then, p ext n Select the top K words according to the original word order X p K is calculated using the reference summary length T. To obtain natural-sounding summaries close to the desired length, the reference summary length T is quantized into a discrete size range. In this embodiment, the size range is set to 5.

[0055] [Loss function of content selection part 11] The process performed by the content selector 11 is a simple binary classification task, so binary cross-entropy loss is used.

[0056]

number

[0057] [Loss function of generator 12] The main loss of the generator 12 is the cross-entropy loss.

[0058]

number

[0059]

number

[0060] The overall loss of the generator 12 is a linear combination of the above three losses.

[0061]

number

[0062] As described above, the parameter learning unit 13 evaluates the processing results of the content selection unit 11 and the generation unit 12 based on the learning data described above using the loss function, and updates the learning parameters of the content selection unit 11 and the generation unit 12 until the loss function converges. The values ​​of the learning parameters when the loss function converges are used as learned parameters.

[0063] [experiment] An experiment conducted on the first embodiment will be described.

[0064] <Dataset> We used the CNN-DM dataset (Karl Moritz Hermann, Tomas Kocisky, Edward Grefenstette, Lasse Espeholt, Will Kay, Mustafa Suleyman, and Phil Blunsom. 2015. Teaching machines to read and comprehend. In Advances in Neural Information Processing Systems 28, pages 1693-1701. (hereafter referred to as "Reference 4")), a standard corpus for news summaries. The summaries are bullet points of articles displayed on each website. Following Abigail See, Peter J. Liu, and Christopher D. Manning. 2017. Get to the point: Summarization with pointer generator networks. In ACL (1), pages 1073-1083. (2017), we used the unanonymized version of the corpus and truncated the source documents to 400 tokens and the target summaries to 120 tokens. The dataset contains 286817 training pairs, 13368 validation pairs, and 11487 test pairs.To evaluate the domain transfer ability of the model, we also used the Newsroom dataset (Max Grusky, Mor Naaman, and Yoav Artzi. 2018. Newsroom: A dataset of 1.3 million summaries with diverse extractive strategies. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 708-719. Association for Computational Linguistics.).

[0065] We trained the content selector 11 on the Newsroom dataset (Reference 3), using the generator 12 trained on the CNN / DM dataset. Newsroom contains a variety of news sources (38 different news sites). To train the content selector 11, we sampled 300,000 training pairs from all training data. The size of the test pairs is 106,349.

[0066] <Model configuration> The same configuration was used for both datasets. The content selector 11 used a pre-trained BERT large model (Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. CoRR.). We fine-tuned BERT for two epochs. Default settings were used for other parameters for fine-tuning. The content selector 11 and generator 12 used pre-trained 300-dimensional GloVe embeddings. Transformer d model The model size was set to 512. The Transformer includes four Transformer blocks for the source text encoding unit 121, the reference text encoding unit 122, and the decoding unit 123. The number of heads was 8, and the number of dimensions of the feed forward network was 2048. The dropout rate was set to 0.2. The optimization was performed using β1=0.9, β2=0.98, and ε=e -9 We used the Adam optimizer (Diederik P. Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In International Conference on Learning Representations (ICLR)). We varied the learning rate during training according to Reference 1. We set the warm-up step to 8000. We set the input vocabulary size to 100000 and the output vocabulary size to 1000.

[0067] <Experimental Results> Table 1 shows the ROUGE scores of Non-Patent Document 1 and the first embodiment.

[0068] [Table 1] According to Table 1, it can be seen that the first embodiment is superior to Non-Patent Document 1 in all aspects of ROUGE-1 (R-1), ROUGE-2 (R-2) and ROUGE-L (RL).

[0069] As described above, according to the first embodiment, it is possible to add information (output length) to be considered when generating a sentence as text. As a result, it is possible to treat the source text (input sentence) and the feature amount of the information to be considered equally.

[0070] In "Yuta Kikuchi, Graham Neubig, Ryohei Sasano, Hiroya Takamura, and Manabu Okumura. 2016. Controlling output length in neural encoder-decoders. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1328-1338. Association for Computational Linguistics," length control is performed using length embedding. This method does not explicitly consider the importance of words according to their length, and is therefore unable to appropriately control the information to be included in the output sentence when controlling the length. On the other hand, this embodiment makes it possible to generate highly accurate summaries while more directly controlling the importance of information according to the output length K without using length embedding.

[0071] Next, a second embodiment will be described. In the second embodiment, differences from the first embodiment will be described. Points not specifically mentioned in the second embodiment may be the same as those in the first embodiment.

[0072] Fig. 8 is a diagram showing an example of the configuration of the generation unit 12 in the second embodiment. In Fig. 8, the same parts as in Fig. 3 are given the same reference numerals, and the description thereof will be omitted.

[0073] Figure 8 differs from Figure 3 in that the source text encoder 121 and the reference text encoder 122 refer to each other, and this cross-referencing occurs when the source text and the reference text are encoded.

[0074] In this way, the second embodiment differs in the configuration of the generation unit 12. Therefore, in the second embodiment, the reference text and the source text X C The procedure for generating a summary based on the above also differs from that of the first embodiment.

[0075] 9 is a diagram for explaining the processing by the generation unit 12 in the second embodiment. As shown in FIG. 9, in the second embodiment, the source text coding unit 121 and the reference text coding unit 122 are collectively referred to as a joint coding unit 125.

[0076] [Joint Encoding Unit 125] First, the embedding layer of the joint encoding unit 125 embeds the word x C l Each one-hot vector (of size V) is trained using a pre-trained weight matrix such as GloVe.

[0077]

number

[0078] Then the embedding layer uses a fully connected layer to word dimensional word embeddings model The embedding layer maps the word embeddings to vectors of dimensions and passes the mapped embeddings to a ReLU function. The embedding layer also adds a positional encoding to the word embeddings (Reference 1).

[0079] The Transformer Endcoder Block of the joint encoding unit 125 encodes the embedded source text and reference text in a stack of Transformer blocks. This block has the same architecture as that of Reference 1. It consists of two subcomponents: a multi-head self-attention network and a fully connected feed forward network. Each network applies residual connections. In this model, both the source text and the reference text are encoded separately in the encoder stack. Their outputs are respectively

[0080]

number

[0081] The Transformer dual encoder blocks in the joint encoding unit 125 calculate the interactive attention between the encoded source text and the reference text. Specifically, the source text and the reference text are first encoded, and then the other outputs of the encoder stack (i.e., E C s and E P S ) and perform multi-head attention on the source text and the reference text.

[0082]

number

[0083] [Decoding unit 123] The embedding layer of the decoder 123 receives the word sequence of the summary sentence Y generated as an autoregressive process. At each decoding step t, the decoder 123 embeds the word y tProject each one-hot vector of

[0084] The Transformer Decoder Block of the decoder 123 has the same architecture as that of Reference 1. This component is used in stages during testing, so subsequent masks are used. The decoder 123 converts the reference text into an encoded representation M p The decoder 123 uses a stack of decoder blocks that perform multi-head attention on the source text encoded representation M C We use another stack of decoder blocks that perform multi-head attention. The first performs the rewriting of the reference text, and the second complements the rewritten reference text with the original source information. The output of the stack is

[0085]

number

[0086] [Synthesis section 124] The synthesis unit 124 uses a pointer generator to select information from the source text, the reference text, or the decoding unit 123 based on the distribution of copies, and generates a summary based on the selected information.

[0087] The copy distribution of the source text and the reference text is as follows:

[0088]

number

[0089] The final vocabulary distribution is as follows:

[0090]

number

[0091] [Learning data for the content selection unit 11 and the generation unit 12] The learning data for each of the content selection unit 11 and the generation unit 12 may be the same as in the first embodiment.

[0092] [Loss function of content selection part 11] The process performed by the content selector 11 is a simple binary classification task, so binary cross-entropy loss is used.

[0093]

number

[0094] [Loss function of generator 12] The main loss of the generator 12 is the cross-entropy loss.

[0095]

number

[0096]

number

[0097] The overall loss of the generator 12 is a linear combination of the above three losses.

[0098]

number

[0099] As described above, the parameter learning unit 13 evaluates the processing results of the content selection unit 11 and the generation unit 12 based on the learning data described above using the loss function, and updates the learning parameters of the content selection unit 11 and the generation unit 12 until the loss function converges. The values ​​of the learning parameters when the loss function converges are used as learned parameters.

[0100] [experiment] An experiment conducted for the second embodiment will now be described. The data set used for the experiment for the second embodiment is the same as that used for the first embodiment.

[0101] <Experimental Results> Table 2 shows the ROUGE scores of Non-Patent Document 1 and the second embodiment.

[0102] [Table 2] According to Table 2, it can be seen that the second embodiment is superior to Non-Patent Document 1 in all aspects of ROUGE-1 (R-1), ROUGE-2 (R-2) and ROUGE-L (RL).

[0103] As described above, according to the second embodiment, it is possible to obtain the same effects as those of the first embodiment.

[0104] Furthermore, according to the second embodiment, words contained in the reference text can also be used to generate a summary.

[0105] Next, a third embodiment will be described. In the third embodiment, differences from the first embodiment will be described. Points not specifically mentioned in the third embodiment may be the same as those in the first embodiment.

[0106] In the third embodiment, an example will be described in which information similar to the source text is searched for from a knowledge source DB20 storing external knowledge, which is a text document (a collection of sentences), and the K text sentences in the information that are highly relevant to the source text and the relevance indicating the degree of relevance are used as reference texts, thereby enabling summarization that takes external knowledge into consideration and making it possible to directly control important information in an input sentence according to the external knowledge.

[0107] Fig. 10 is a diagram showing an example of the functional configuration of a sentence generation device 10 according to the third embodiment. In Fig. 10, parts that are the same as or correspond to those in Fig. 2 are given the same reference numerals, and their explanation will be omitted as appropriate.

[0108] 10, the sentence generation device 10 further includes a search unit 14. The search unit 14 searches for information from the knowledge source DB 20 using the source text as a query. The information searched by the search unit 14 corresponds to the consideration information in each of the above embodiments. That is, in the third embodiment, the consideration information is external knowledge (from which reference text is created based on the relevance with the source text).

[0109] FIG. 11 is a flowchart illustrating an example of a processing procedure executed by the sentence generation device 10 according to the third embodiment.

[0110] In step S201, the search unit 14 searches the knowledge source DB 20 using the source text as a query.

[0111] Fig. 12 is a diagram showing an example of the configuration of the knowledge source DB 20. Fig. 12 shows two examples (1) and (2).

[0112] (1) shows an example in which pairs of documents that serve as input and output sentences for a task executed by the sentence generation device 10 are stored in the knowledge source DB 20. Fig. 12(1) shows an example in which pairs of a news article and a headline (or a summary) are stored as an example in which the task is generating a title or a summary.

[0113] (2) shows an example in which one of the documents in the pair (only the headline in the example of FIG. 12) is stored in the knowledge source DB 20.

[0114] In either case, it is assumed that the knowledge source DB 20 stores a large amount of knowledge (information).

[0115] In step S201, the search unit 14 searches the knowledge source DB 20 for a document group of a number K' (approximately 30 to 1000) that can be reranked, as described below, using a high-speed search module such as elasticsearch.

[0116] When the knowledge source DB 20 has a configuration as shown in (1), the following search methods are possible: searching by the similarity between the source text and the headline, searching by the similarity between the source text and the news article, or searching by the similarity between the source text and the news article + headline.

[0117] On the other hand, if the knowledge source DB 20 has the configuration shown in (2), it is possible to search based on the similarity between the source text and the headline. The similarity is a known index for evaluating the similarity between documents, such as the number of times the same word is included or cosine similarity.

[0118] In this embodiment, in either case (1) or (2), the search results are assumed to be K' headlines with similarity, and each headline is assumed to be a single sentence. Hereinafter, each of the K' sentences (headlines) in the search results will be referred to as a "knowledge source text."

[0119] Next, the content selection unit 11 calculates the relevance at the sentence level (for each knowledge source text) for each knowledge source text using a relevance calculation model, which is a pre-trained neural network (S202). The relevance calculation model may constitute a part of the content selection unit 11. The relevance is an index showing the degree of relevance, similarity or correlation with the source text, and corresponds to the importance in the first or second embodiment.

[0120] FIG. 13 is a diagram illustrating a first example of a relevance calculation model. As shown in FIG. 13, a source text and a knowledge source text are input to an LSTM, respectively. Each LSTM converts each word constituting each text into a vector of a predetermined dimension. As a result, each text becomes an array of vectors of a predetermined dimension. The number of vectors (i.e., the length of the vector array) I is determined based on the number of words. For example, I is set to 300, and if the number of words is less than 300, a predetermined word such as "PAD" is used to adjust the number to 300. Here, for convenience, the number of words = the number of vectors. Therefore, the length of the vector array that is the conversion result of text containing I words is I.

[0121] The matching network receives the vector sequence of the source text and the vector sequence of the knowledge source text as input and calculates the sentence-level relevance β (0≦β≦1) for the knowledge source text. Note that the matching network may use, for example, a co-attention network ("Caiming Xiong, Victor Zhong, Richard Socher, DYNAMIC COATTENTION NETWORKS FOR QUESTION ANSWERING, Published as a conference paper at ICLR 2017") or the like.

[0122] Fig. 14 is a diagram for explaining a second example of the relevance calculation model. In Fig. 14, only the differences from Fig. 13 will be explained.

[0123] In FIG. 14, the matching network calculates the word-level relevance p for each word i included in the knowledge source text (i.e., for each element of the vector array). i (0≦p i ≦1). Such a matching network may also be realized using a co-attention network.

[0124] The relevance calculation model is based on the word-level relevance p i The sentence-level relevance β is calculated by the weighted sum of β = Σw i p i (i=1,...,number of words). Note that w i are the learnable parameters of the neural network.

[0125] 13 or 14 is executed for K' knowledge source texts, and therefore the relevance β is calculated for each knowledge source text.

[0126] Next, the content selection unit 11 extracts, as reference text, the result of concatenating a predetermined number (K) of knowledge source texts, two or more, in descending order of the relevance β calculated by the method shown in FIG. 13 or FIG. 14 (S203).

[0127] Next, the generation unit 12 generates a summary based on the reference text and the source text (S204). The processing executed by the generation unit 12 may be basically the same as that in the first or second embodiment. However, the attention probability α p tk may be weighted using the word-level relevance or the sentence-level relevance as follows: p tk of 、 We defined it as the attention head, but here, α p tk Since we are talking about the value of α p tkcorresponds to the attention probability. In the following, the sentence-level relevance or the word-level relevance will be denoted by β for convenience. Either the word-level relevance or the sentence-level relevance may be used, or both may be used.

[0128] When using sentence-level relevance, for example, attention probability α p tk is updated as follows:

[0129]

number

[0130] If you use word-level relevance, use β in the above formula. S For (k), the word-level relevance p corresponding to word k i When both are used, it is possible to weight the relevance at the word level with the relevance at the sentence level, as in the above formula (2). i is calculated for each sentence S. i has the same role as the importance (equation (1)) in the first embodiment. Furthermore, k is a word number assigned to a word in the reference text (a set of extracted sentences S).

[0131] Next, learning will be described. Fig. 15 is a diagram showing an example of the functional configuration of the sentence generation device 10 during learning in the third embodiment. In Fig. 15, parts that are the same as or correspond to those in Fig. 7 or Fig. 10 are given the same reference numerals, and their description will be omitted as appropriate.

[0132] In the third embodiment, the learning of the content selection unit 11 and the generation unit 12 may be basically the same as in the above embodiments. Here, two methods for learning the relevance calculation model used in the third embodiment will be described.

[0133] The first method is to define correct information for the relevance β at the sentence level using the calculation result of the relevance β and a score such as Rouge from the correct target text.

[0134] The second method is to determine the correct answer information of the relevance at the word level as 1 / 0 depending on whether or not the word is included in the correct sentence (for example, the target text such as a summary sentence).

[0135] In the above, an example has been shown in which the sentence generation device 10 has the search unit 14, but if the external knowledge included in the knowledge source DB 20 has been narrowed down in advance, all of the knowledge source texts included in the external knowledge may be input to the content selection unit 11. In this case, the sentence generation device 10 does not need to have the search unit 14.

[0136] As described above, according to the third embodiment, by using external knowledge, it is possible to efficiently generate a summary that includes words that are not present in the source text. The words are contained in the knowledge source text, which is, as the name suggests, text. Therefore, according to the third embodiment, it is also possible to add information that should be considered when generating a sentence as text.

[0137] The technology disclosed in "Ziqiang Cao, Wenjie Li, Sujian Li, and FuruWei. 2018. Retrieve, rerank and rewrite: Soft template based neural summarization. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 152-161. Association for Computational Linguistics." can generate target text taking external knowledge into consideration, but (1) it cannot use the words contained in the external knowledge directly for sentence generation. Also, (2) it cannot take into consideration the importance of each content of the external knowledge. On the other hand, the third embodiment (1) takes into consideration the relevance of the external knowledge at the sentence and word levels, and (2) uses a CopyNetwork (synthesis unit 124) to enable important parts of the external knowledge to be included in the output sentence.

[0138] Next, a fourth embodiment will be described. In the fourth embodiment, differences from the first embodiment will be described. Points not specifically mentioned in the fourth embodiment may be the same as those in the first embodiment.

[0139] Fig. 16 is a diagram showing an example of the functional configuration of the sentence generation device 10 in the fourth embodiment. In Fig. 16, parts that are the same as or correspond to those in Fig. 2 are given the same reference numerals. As shown in Fig. 16, in the fourth embodiment, the content selection unit 11 does not input consideration information.

[0140] The content selection unit 11 is a TransformerEncoder block (Encoder sal ) and l linear transformation layers. The content selection unit 11 selects the source text X C The nth word in x C n Importance of p ext nis calculated based on the following formula:

[0141]

number

[0142] The content selection unit 11 selects the source text X C Encoder sal The importance p obtained by inputting ext n K words in descending order of their value are taken from the reference text X. p As X C Extract from the reference text X p The order of words in the source text X C The content selection unit 11 maintains the order of the reference text X p is input to the generation unit 12. That is, in the fourth embodiment, a word string extracted based on the predicted value of the importance of the word by the content selection unit 11 is explicitly provided to the generation unit 12 as additional text information.

[0143] Fig. 17 is a diagram showing an example of the configuration of the generation unit 12 in the fourth embodiment. In Fig. 17, parts that are the same as or correspond to those in Fig. 3 are given the same reference numerals. In Fig. 17, the generation unit 12 includes an encoding unit 126 and a decoding unit 123. The encoding unit 126 and the decoding unit 123 will be described in detail with reference to Fig. 18.

[0144] Fig. 18 is a diagram showing an example of a model configuration in the fourth embodiment. In Fig. 18, parts that are the same as or correspond to those in Fig. 16 or 17 are given the same reference numerals. For convenience, the model shown in Fig. 18 is called a CIT (Conditional summarization model with Important Tokens) model.

[0145] The encoding unit 126 is composed of M layers of Transformer Encoder blocks. C +X p The input is X C +X p is X C and X p This is a character string in which a special token representing a separator is inserted between the characters. sal The RoBERTa (Y. Liu et al. arXiv, 1907.11692, 2019.) is used as the initial value for the input X C +X p and expresses the M-layer blocks.

[0146]

number

[0147] The decoding unit 123 may be the same as that in the first embodiment. That is, the decoding unit 123 is made up of M-layer Transformer Decoder blocks. The output H M e and the sequence {y1,...,y t-1} is input and the expression is generated by applying the M-layer blocks.

[0148]

number

[0149] All attention processing in the Multi-headAttentionTransformer block uses Multi-headAttention (A. Vaswani et al. In NIPS, pages 5998-6008, 2017). This processing consists of concatenating k attention heads, Multihead(Q,K,V)=Concat(head1,...,headk)W o Each head is represented as head i =Attention(QW Q i ,KW K i ,VW V i ) where, in the m-th layer self-attention of the encoding unit 126 and the decoding unit 123, the same vector representation H m In the context-attention of the decoding unit 123, H is added to Q. m d , K and V to H M e are given respectively.

[0150] Attention for each head

[0151]

number

[0152]

number

[0153]

number

[0154] Next, learning will be described. Fig. 19 is a diagram showing an example of the functional configuration of the sentence generation device 10 in the fourth embodiment during learning. In Fig. 19, the same parts as in Fig. 7 are given the same reference numerals, and their description will be omitted.

[0155] [Learning data and loss function of the generation unit 12] The training data for the generation unit 12 may be the same as that in the first embodiment. The loss function for the generation unit 12 is defined as follows using cross entropy: M represents the number of training data.

[0156]

number

[0157] The loss function of the content selection unit 11 is defined as follows using binary cross entropy:

[0158]

number

[0159] [Parameter Learning Part 13] In the fourth embodiment, the parameter learning unit 13 trains the content selection unit 11 using learning data on importance, and trains the generation unit 12 using correct answer information on the summary sentence Y. In other words, the content selection unit 11 and the generation unit 12 are trained independently.

[0160] The parameter learning unit 13 calculates the loss function L gen =L encdec +L sal As, L gen Training is performed so that is minimized.

[0161] In the first embodiment, the length of the target text (summary) in the training data is aligned with the output length, which is the consideration information, to learn the "importance of words according to the output length." On the other hand, in the fourth embodiment, the training data does not include the output length as consideration information. Therefore, the content selection unit 11 does not take the output length into particular consideration when calculating the importance of words, and calculates the importance only from the perspective of "whether the word is important for summarizing."

[0162] However, in the fourth embodiment as well, the output length is input as consideration information, and the digest length can be controlled by determining the number of tokens to be extracted based on the output length.

[0163] Next, a fifth embodiment will be described. In the fifth embodiment, differences from the fourth embodiment will be described. Points not specifically mentioned in the fifth embodiment may be the same as those in the fourth embodiment.

[0164] Fig. 20 is a diagram showing an example of the functional configuration of the sentence generation device 10 in the fifth embodiment. In Fig. 20, the same components as those in Fig. 16 are given the same reference numerals.

[0165] The content selection unit 11 in the fifth embodiment selects an individually trained encoder. sal Word-level importance p obtained from ext nUsing sentence-level importance p ext Sj Calculate the source text X C In p ext Sj The input text X is a combination of the top P sentences. s The sentence level importance p ext Sj can be calculated based on Equation 3.

[0166] Fig. 21 is a diagram showing an example of the configuration of the generation unit 12 in the fifth embodiment. In Fig. 21, the same parts as in Fig. 17 are given the same reference numerals. Fig. 22 is a diagram showing an example of the model configuration in the fifth embodiment. In Fig. 22, the same parts as in Fig. 18 are given the same reference numerals.

[0167] As shown in FIGS. 21 and 22, the encoding unit 126 in the fifth embodiment encodes an input text X s The model shown in Fig. 22 is referred to as a Sentence Extraction then Generation (SEG) model for convenience.

[0168] Other points are the same as those in the fourth embodiment.

[0169] Next, a sixth embodiment will be described. In the sixth embodiment, differences from the fourth embodiment will be described. Points not specifically mentioned in the sixth embodiment may be the same as those in the fourth embodiment.

[0170] FIG. 23 is a diagram showing an example of the functional configuration of the sentence generation device 10 in the sixth embodiment. In FIG. 23, parts that are the same as or correspond to those in FIG. 16 or 17 are given the same reference numerals. In the sixth embodiment, the sentence generation device 10 does not have a content selection unit 11. On the other hand, the generation unit 12 has a content selection encoding unit 127 instead of the encoding unit 126. The content selection encoding unit 127 combines the functions of the encoding unit 126 (encoder) and the content selection unit 11. In other words, the encoding unit 126, which also serves as the content selection unit 11, corresponds to the content selection encoding unit 127.

[0171] Fig. 24 is a diagram showing an example of a model configuration in the sixth embodiment. In Fig. 24, parts that are the same as or correspond to those in Fig. 23 or Fig. 18 are given the same reference numerals. Fig. 24 shows three example models (a) to (c). For convenience, (a) is called the MT (Multi-Task) model, (b) is called the SE (Selective Encoding) model, and (c) is called the SA (Selective Attention) model.

[0172] The MT model in (a) is based on the importance p ext n The content selection encoder 127 and the decoder 123 (i.e., the generator 12) are trained simultaneously by additionally using correct answer data for the sentences. In other words, the importance model and the sentence generation model are trained simultaneously. This point is common to the SE model, the SA model, and each model described below (i.e., other than the CIT model and the SEG model). In the MT model, the Encoder of the content selection encoder 127 sal shares the parameters of the encoding unit 126 in the fourth embodiment. As is clear from FIG. 24, in the MT model, the encoding result (H M e ) is input to the decoding unit 123.

[0173] The decoding unit 123 may be the same as that in the fourth embodiment.

[0174] The SE model (b) shows the importance p ext n(Q. Zhou et al., In ACL, pages 1095-1104, 2017.) Specifically, the decoding unit 123 biases the encoding result h M en The following weighting is given according to the importance:

[0175]

number

[0176] Unlike the SE model, the SA model (c) weights attention on the decoding unit 123 side. Specifically, the decoding unit 123 weights the context-attention weight matrix of the i-th head

[0177]

number

[0178]

number

[0179]

number

[0180] A model that combines the MT model with the SE model or the MT model is also effective as the sixth embodiment.

[0181] The SE+MT model has importance p in learning the SE model. ext n We use additional correct answer data for the above to simultaneously learn the summaries.

[0182] SA+MT model uses the importance p ext n We use additional correct answer data for the above to simultaneously learn the summaries.

[0183] Fig. 25 is a diagram showing an example of the functional configuration of the sentence generation device 10 during learning in the sixth embodiment. In Fig. 25, parts that are the same as or correspond to those in Fig. 23 or Fig. 19 are given the same reference numerals.

[0184] As described above, the parameter learning unit 13 of the sixth embodiment learns the content selection encoding unit 127 and the decoding unit 123 simultaneously.

[0185] In addition, in the case of the SE model and the SA model, the parameter learning unit 13 calculates the importance P ext n The generator 12 (the content selection encoder 127 and the decoder 123) is trained using only the correct answer information of the summary sentence Y without providing the correct answer information of the summary sentence Y. In this case, the parameter trainer 13 trains the loss function L gen =L encdec And L gen Training is performed so that is minimized.

[0186] On the other hand, in the case of the MT model, the parameter learning unit 13 performs multitask learning of the content selection coding unit 127 and the generation unit 12 (i.e., the content selection coding unit 127 and the decoding unit 123) using the correct answer information of the importance S and the correct answer information of the summary sentence Y. That is, when the content selection coding unit 127 is trained as an importance model (content selection unit 11), the importance P ext n The correct answer information (whether it is important or not) is used as training data, and X c From importance P ext n In addition, when the generation unit 12 (content selection encoding unit 127+decoding unit 123) is trained as a Seq2Seq (Encoder-Decoder) model, the input sentence X C The correct summary sentence for X is used as training data. C During this multitask learning, the parameters of the content selection coding unit 127 are shared between both tasks. ext nThe correct answer information (pseudo correct answer) is as explained in the fourth embodiment. In this case, the parameter learning unit 13 calculates the loss function L gen =L encdec +L sal As, L gen Training is performed so that is minimized.

[0187] Next, a seventh embodiment will be described. In the seventh embodiment, differences from the fourth embodiment will be described. Points not specifically mentioned in the seventh embodiment may be the same as those in the fourth embodiment.

[0188] FIG. 26 is a diagram showing an example of the functional configuration of a sentence generation device 10 according to the seventh embodiment. In FIG. 26, parts that are the same as or correspond to those in FIG. 16 or 23 are given the same reference numerals. The sentence generation device 10 in FIG. 26 has a content selection unit 11 and a generation unit 12. The generation unit 12 includes a content selection encoding unit 127 and a decoding unit 123. That is, the sentence generation device 10 according to the seventh embodiment has both the content selection unit 11 and the content selection encoding unit 127.

[0189] Such a configuration can be realized by combining the above CIT model (or SEG model) with the SE model or SA model (or MT model). Below, we will explain the combination of the CIT model and the SE model (CIT+SE model) and the combination of the CIT model and the SA model (CIT+SA model).

[0190] In the CIT+SE model, the X of the CIT model C +X p The importance of the SE model is

[0191]

number

[0192]

number

[0193] In the CIT+SA model, as in the CIT+SE model,

[0194]

number

[0195]

number

[0196] The processing executed by the parameter learning unit 13 in the seventh embodiment may be the same as that in the sixth embodiment.

[0197] [experiment] Experiments conducted on the fourth to seventh embodiments will be described.

[0198] <Dataset> We used two representative summary datasets: CNN / DM (K.M. Hermann et al., In NIPS, pages 1693-1701, 2015) and XSum (S. Narayan et al., In EMNLP, pages 1797-1807, 2018). CNN / DM uses summary data with a high extraction rate of approximately three sentences, while XSum uses summary data with a low extraction rate of approximately one sentence. Evaluation was performed using the ROUGE score, which is a standard measure of automatic summary evaluation. An overview of each data set is shown in Table 3. The average summary length was calculated by dividing the dev set of each data set into subwords using byte-level BPE in fairseq1 (A. Radford et al., Language models are unsupervised multi-task learners. Technical report, OpenAI, 2019).

[0199] [Table 3] <Study settings> Each model was implemented using fairseq. Training was performed using seven NVIDIA V100 32GB GPUs. For CNN / DM training, the same settings as in "M. Lewis et al. arXiv, 1910.13461, 2019." For XSum training, we consulted with the authors and changed the mini-batch gradient accumulation parameter UPDATEFREQ to 2 in the CNN / DM settings. Furthermore, the number of extracted words K in the content selection section11 of the CIT model during training was determined by checking the accuracy of the dev set. For CNN / DM, the correct summary length was divided into bins of 5, while for XSum, it was set to a fixed length of 30. For evaluation, for CNN / DM, K was set to the average summary length of the dev set, while for XSum, it was set to 30, the same as during training. For XSum, we also set K so that the important word sequences did not contain duplicates. Regarding the setting of K during training, there are several methods: setting it to an arbitrary fixed value, setting it to a value equal to or greater than an arbitrary threshold, or making it dependent on the correct summary length L. When learning is performed by setting K depending on the length L of the correct summary, the summary length can be controlled by changing the length of K during testing.

[0200] <Experimental Results> Tables 4 and 5 show the experimental results (ROUGE values ​​for each model) regarding the question, "Does combining importance models improve summary accuracy?"

[0201] [Table 4]

[0202] [Table 5] As shown in Tables 4 and 5, CIT+SE produced the best results for both datasets. First, the accuracy improved even with CIT alone, which shows that combining important words is an excellent method for providing important information to the sentence generation model (generation unit 12). Furthermore, further improvements in accuracy were confirmed by combining CIT with the SE and SA models, which suggests that although noise is included among the important words, the combination of soft weighting contributes to the improvement.

[0203] SE and SA improved accuracy for all data. MT, SE+MT, and SA+MT, which simultaneously learn importance-based answers, improved accuracy for CNN / DM, but decreased accuracy for XSum. This is thought to be due to the quality of the pseudo-ground truth for importance. CNN / DM has relatively long summaries with strong extraction elements, making it easy to align words when creating pseudo-ground truths. On the other hand, XSum data has short summaries and is often written in expressions other than the source text, making alignment difficult and resulting in the pseudo-ground truth acting as noise. This effect also meant that the improvement in accuracy for XSum was smaller than for CNN / DM in CIT, but since it showed the highest performance on data with different properties, it can be said to be a robust method.

[0204] Note that "our fine-tuning" represents the fine-tuning results (baseline) by the inventors of the present application. Also, the underlined results in Table 5 represent the highest accuracy among the improvements to the baseline model.

[0205] Table 6 shows the experimental results regarding the question, "What is the token extraction accuracy of the importance model alone?"

[0206] Table 6 shows the importance of CIT. ext This is the result of evaluating the ROUGE values ​​of the top selected token sequences and summary texts.

[0207] [Table 6] It can be seen that CNN / DM is able to properly identify important words. Compared to Presum (Y. Liu et al. In EMNLP-IJCNLP, pages 3728-3738, 2019), a conventional SOTA extractive summarization method, it achieves high accuracy in R1 and R2, demonstrating that it is able to extract important elements at the word level. ext While there was not much difference in the importance of each word in [2], this model can explicitly take important words into account. On the other hand, XSum uses data with a low extraction rate, so the accuracy of the importance model is low overall. We believe this is the reason why the improvement in summary accuracy is lower than that of CNN / DM. The method of providing pseudo-correct answers tested here was particularly effective for data with a high extraction rate, but further improvements in accuracy can be expected by improving the accuracy of the importance model even for data with a low extraction rate.

[0208] Although the above embodiments have been described using the summary generation task as an example, the above embodiments may be applied to various other sentence generation tasks.

[0209] In each of the above embodiments, the sentence generation device 10 during learning is an example of a sentence generation learning device.

[0210] Although the embodiments of the present invention have been described in detail above, the present invention is not limited to such specific embodiments, and various modifications and variations are possible within the scope of the gist of the present invention as described in the claims. [Explanation of symbols]

[0211] 10 Sentence generator 11 Content selection section 12 Generation part 13 Parameter learning section 14 Search section 15 Content Selection Encoding Unit 20 Knowledge source DB 100 Drive device 101 Recording media 102 Auxiliary storage device 103 Memory Device 104 CPU 105 Interface Device 121 Source Text Encoding Unit 122 Reference Text Coding Section 123 Decoding Unit 124 Synthesis Section 125 Joint Encoding Unit 126 Encoding section B Bus

Claims

1. a content selection unit that extracts a set of words based on an input sentence, the importance of each word included in the input sentence, and an output length; a generation unit that receives the input sentence and the set of words, and generates an output sentence based on the input sentence and the set of words by inputting information corresponding to the input sentence and the set of words into a machine learning model that has been supervised-trained to generate an output sentence based on the input sentence and the set of words; A sentence generation device comprising:

2. a content selection unit that extracts a set of words based on an input sentence, the importance of each word included in the input sentence, and an output length; a generation unit that receives the input sentence and the set of words, and generates an output sentence based on the input sentence and the set of words by inputting information corresponding to the input sentence and the set of words into a machine learning model that has been supervised-trained to generate an output sentence based on the input sentence and the set of words; a parameter learning unit that updates parameters of the machine learning model based on an output from the machine learning model to which a first text and a set of words extracted from the first text are input as input sentences, and a second text that is an output sentence corresponding to the first text; A sentence generation learning device comprising:

3. a content selection unit that extracts a set of sentences based on an input sentence and at least one of the importance of each sentence and the importance of each word of a plurality of sentences included in the external knowledge; a generation unit that receives the input sentence and the set of sentences, and generates an output sentence based on the input sentence and the set of sentences by inputting information corresponding to the input sentence and the set of sentences into a machine learning model that has been supervised-trained to generate an output sentence based on the input sentence and the set of sentences; A sentence generation device comprising:

4. a content selection unit that extracts a set of sentences based on an input sentence and at least one of the importance of each sentence and the importance of each word of a plurality of sentences included in the external knowledge; a generation unit that receives the input sentence and the set of sentences, and generates an output sentence based on the input sentence and the set of sentences by inputting information corresponding to the input sentence and the set of sentences into a machine learning model that has been supervised-trained to generate an output sentence based on the input sentence and the set of sentences; a parameter learning unit that updates parameters of the machine learning model based on an output from the machine learning model to which a first text and a set of sentences extracted from external knowledge are input as input sentences, and based on a second text that is an output sentence corresponding to the first text; A sentence generation learning device comprising:

5. a content selection unit that extracts a set of words based on an input sentence, the importance of each word included in the input sentence, and an output length; a generation unit that receives the set of words as input and inputs information corresponding to the set of words into a machine learning model that has been supervised-trained to generate an output sentence based on the set of words, thereby generating an output sentence based on the set of words; A sentence generation device comprising:

6. a content selection unit that extracts a set of words based on an input sentence, the importance of each word included in the input sentence, and an output length; a generation unit that receives the set of words as input and inputs information corresponding to the set of words into a machine learning model that has been supervised-trained to generate an output sentence based on the set of words, thereby generating an output sentence based on the set of words; a parameter learning unit that updates parameters of the machine learning model based on an output from the machine learning model to which a set of words extracted from a first text is input as an input sentence and a second text that is an output sentence corresponding to the first text; A sentence generation learning device comprising:

7. a content selection unit that extracts a set of sentences based on a plurality of input sentences, the importance of each sentence included in the plurality of input sentences, and an output length; a generation unit that receives the set of sentences as input and inputs information corresponding to the set of sentences into a machine learning model that has been supervised-trained to generate output sentences based on the set of sentences, thereby generating output sentences based on the set of sentences; A sentence generation device comprising:

8. a content selection unit that extracts a set of sentences based on a plurality of input sentences, the importance of each sentence included in the plurality of input sentences, and an output length; a generation unit that receives the set of sentences as input and inputs information corresponding to the set of sentences into a machine learning model that has been supervised-trained to generate output sentences based on the set of sentences, thereby generating output sentences based on the set of sentences; a parameter learning unit that updates parameters of the machine learning model based on an output from the machine learning model to which a first text and a set of sentences extracted from the first text are input as a plurality of input sentences, and based on a second text that is an output sentence corresponding to the first text; A sentence generation learning device comprising:

9. a content selection unit that extracts a set of words based on an input sentence, the importance of each word included in the input sentence, and an output length; a generation unit that receives the input sentence and the set of words, and generates an output sentence based on the input sentence and the set of words by inputting information corresponding to the input sentence and the set of words into a machine learning model that has been supervised-trained to generate an output sentence based on the input sentence and the set of words; A sentence generation device comprising:

10. a content selection unit that extracts a set of words based on an input sentence, the importance of each word included in the input sentence, and an output length; a generation unit that receives the input sentence and the set of words, and generates an output sentence based on the input sentence and the set of words by inputting information corresponding to the input sentence and the set of words into a machine learning model that has been supervised-trained to generate an output sentence based on the input sentence and the set of words; a parameter learning unit that updates parameters of the machine learning model based on an output from the machine learning model to which a first text and a set of words extracted from the first text are input as input sentences, and a second text that is an output sentence corresponding to the first text; A sentence generation learning device comprising:

11. a content selection step for extracting a set of words based on an input sentence, the importance of each word in the input sentence, and an output length; a generation step of inputting the input sentence and the set of words, and generating an output sentence based on the input sentence and the set of words by inputting information corresponding to the input sentence and the set of words into a machine learning model that has been supervised-trained to generate an output sentence based on the input sentence and the set of words; A sentence generation method characterized by being executed by a computer.

12. a content selection step for extracting a set of words based on an input sentence, the importance of each word in the input sentence, and an output length; a generation step of inputting the input sentence and the set of words, and generating an output sentence based on the input sentence and the set of words by inputting information corresponding to the input sentence and the set of words into a machine learning model that has been supervised-trained to generate an output sentence based on the input sentence and the set of words; a parameter learning procedure for updating parameters of the machine learning model based on an output from the machine learning model to which a first text and a set of words extracted from the first text are input as input sentences, and a second text that is an output sentence corresponding to the first text; A sentence generation learning method characterized in that the above is executed by a computer.

13. A program for causing a computer to function as the sentence generation device according to any one of claims 1, 3, 5, 7 and 9.

14. A program for causing a computer to function as the sentence generation learning device according to any one of claims 2, 4, 6, 8 and 10.

Images