Hypothesis generation device and hypothesis generation method

Abstract

To provide a hypothesis generation device and a hypothesis generation method by which an interesting hypothesis which is not easily think though related to content of input is obtained, and which can generate the hypothesis with high reliability.SOLUTION: A hypothesis generation device 100 comprises: a related text creation unit 114 which receives input of a text 110 to create a related text related to the text 110; and a hypothesis generation model 122 preliminarily learned to receive the text 110 and the related text as the input to generate a hypothesis from the text 110.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] This application relates to hypothesis generation technology, and more particularly to a hypothesis generation apparatus and a hypothesis generation method capable of generating various hypotheses.

Background Art

[0002] Predicting future events has great significance in fields such as academia, business, and politics. Based on such predictions, it becomes possible to make appropriate decisions based on future opportunities and risks.

[0003] In this specification, what is formed by putting such a prediction into a form understandable to humans, such as a sentence or a passage, is called a "hypothesis". The technology for generating hypotheses is roughly classified into two types. The first is a technology that extracts knowledge written by humans from a large amount of text collected from the web (hereinafter referred to as "web text") and concatenates them to form a hypothesis. The second is an automatic hypothesis generation technology that utilizes the generation technology of deep learning.

[0004] As the first technology, there is the technology disclosed in Patent Document 1 below. The technology disclosed in Patent Document 1 collects word sequences that are considered to represent causal relationships represented by pairs of texts of "noun + particle + predicate" from a large amount of Japanese web text. Among these word sequences, the preceding text represents the cause, and the subsequent text represents the result. If the result part of the text of the first causal relationship matches the cause part of another causal relationship, a new word sequence is created by chaining the cause part of the first causal relationship and the result part of the second causal relationship. In Patent Document 1, this word sequence is called a "social scenario". By repeating the process of chaining causal relationships in this way, a large number of social scenarios can be generated.

[0005] As the second technology, a technology called ChatGPT is known. This technology relates to a language model composed of a neural network learned by so-called deep learning. Specifically, based on the input text, a hypothesis is generated by a neural network and output. [Prior art documents] [Patent Documents]

[0006] [Patent Document 1] Japanese Patent Publication No. 2017-37544 [Non-patent literature]

[0007] [Non-Patent Document 1] OpenAI, “Introducing ChatGPT”, [online], OpenAI homepage, [searched March 5, 2023], Internet<URL:https: / / openai.com / blog / chatgpt> [Overview of the project] [Problems that the invention aims to solve]

[0008] Of the conventional technologies described above, the first technology utilizes existing data to generate hypotheses. Therefore, it is difficult to generate hypotheses that do not exist anywhere, and the reliability of the generated hypotheses is high. However, on the other hand, there is a problem in that it cannot obtain hypotheses that do not exist anywhere.

[0009] In the second technique, conversely, a hypothesis is obtained as the output of the generation process, meaning that some output can be obtained for any input, and that output may even be completely new. However, in the second technique, there is a problem in that there are no restrictions on the content that can be obtained, so it is not possible to know what content will be generated until the results are actually obtained. Also, the output can be clearly wrong, so the reliability of the output is low, and it is difficult to verify whether the output is valid or not. Question It also has a title.

[0010] Therefore, the object of this invention is to provide a hypothesis generation device and a hypothesis generation method that can generate interesting hypotheses that are related to the input content but are not easily conceived, and moreover, hypotheses that are highly reliable. [Means for solving the problem]

[0011] A hypothesis generation device according to the first aspect of the present invention includes a related text creation unit that receives text as input and creates related text related to said text, and a hypothesis generation model that receives text and related text as input and is pre-trained to generate hypotheses from text.

[0012] Preferably, the related text creation unit includes a question generation unit that generates one or more questions based on the text, and a question answering unit that searches an existing text archive for passages containing answers to the one or more questions generated by the question generation unit, and outputs them as related text.

[0013] More preferably, the hypothesis generation device further includes a hypothesis generation unit that inputs hypotheses generated by a hypothesis generation model into a related text creation unit instead of input text, and causes the hypothesis generation model to generate new hypotheses.

[0014] More preferably, the hypothesis generation device further includes a first selector that selects either the input text or a hypothesis generated by the hypothesis generation model and inputs it into the related text creation unit, and a second selector that selects either the input text or a hypothesis generated by the hypothesis generation model as input to the hypothesis generation model, inputs it into the hypothesis generation model together with the related text, and causes the hypothesis generation model to generate a new hypothesis.

[0015] Preferably, the hypothesis generation device further includes an input selector that selects either the input text or the related text generated by the related text creation unit and inputs it to the related text creation unit.

[0016] The hypothesis generation method according to the second aspect of the present invention includes a step in which a computer receives an input of text and creates related text related to the text, and a hypothesis generation step in which the computer receives the text and the related text as inputs and inputs them into a hypothesis generation model that has been previously learned to generate a hypothesis from the text to generate a new hypothesis.

[0017] The above and other objects, features, aspects and advantages of the present invention will become apparent from the following detailed description of the present invention, which is to be understood in connection with the accompanying drawings.

Effects of the Invention

[0018] According to the present invention, it is possible to provide a hypothesis generation device and a hypothesis generation method that can obtain interesting hypotheses that are related to the content of the input but are not easily thought of, and moreover, can generate hypotheses with high reliability. In addition, by presenting the related text used in the process of deriving the hypothesis together with the hypothesis, the user can infer the reasons and grounds for which the hypothesis was derived. A hypothesis for which the reasons and grounds can be inferred in this way is much more useful than a hypothesis for which no grounds are known at all.

Brief Description of the Drawings

[0019] [Figure 1] is a block diagram showing the functional configuration of a hypothesis generation system according to a first embodiment of the present invention. [Figure 2] is a block diagram showing the functional configuration of the related text creation unit shown in FIG. 1. [Figure 3] is a schematic diagram showing the relationship between the input and output of the hypothesis generation neural network shown in FIG. 1. [Figure 4] is a block diagram showing the functional configuration of a learning data generation system for generating learning data for the question generation unit shown in FIG. 2. [Figure 5] is a block diagram showing the functional configuration of a learning data generation system for generating learning data for the hypothesis generation neural network shown in FIG. 1. [Figure 6]is a schematic diagram showing a process of generating an inter-sentence semantic relationship DB (DataBase) shown in FIG. 5. [Figure 7] is a schematic diagram showing a generation process of learning data generated by the learning data assembly unit shown in FIG. 5. [Figure 8] is a schematic diagram showing the configuration of learning data and its generation process in a modification of the first embodiment. [Figure 9] is a schematic diagram showing another example of the learning data generation process in a modification of the first embodiment. [Figure 10] is a block diagram showing the functional configuration of a learning data generation system for generating a modification example of learning data in a modification of the first embodiment. [Figure 11] is a block diagram showing the functional configuration of an associated text creation unit used in the second embodiment. [Figure 12] is a block diagram showing the functional configuration of an associated text creation unit used in the third embodiment. [Figure 13] is a block diagram showing the functional configuration of a hypothesis generation system according to the fourth embodiment. [Figure 14] is a block diagram showing the functional configuration of the associated text creation unit shown in FIG. 13. [Figure 15] is a block diagram showing the functional configuration of a hypothesis generation system according to the fifth embodiment. [Figure 16] is a block diagram showing the functional configuration of a learning data generation system 620 for generating learning data of the associated text creation unit shown in FIG. 15. [Figure 17] is a block diagram showing the functional configuration of a learning data generation system for generating learning data of a hypothesis generation neural network according to the fifth embodiment. <000011�> [Figure 18] is a block diagram showing the functional configuration of a hypothesis generation system according to the sixth embodiment. [Figure 19] is a block diagram showing the functional configuration of a hypothesis generation system according to the seventh embodiment. [Figure 20] This is a block diagram showing the functional configuration of a training data generation system for generating training data for a hypothesis-generating neural network according to the eighth embodiment. [Figure 21] This is a block diagram showing the functional configuration of a training data generation system for generating training data for a hypothesis-generating neural network according to the ninth embodiment. [Figure 22] This is a block diagram showing the functional configuration of a training data generation system that generates training data for a hypothesis-generating neural network according to the 10th embodiment. [Figure 23] This is a block diagram showing the functional configuration of a training data generation system for generating training data for a hypothesis-generating neural network according to the 11th embodiment. [Figure 24] This is a schematic diagram showing the structure of training data for a hypothesis-generating neural network according to the 12th embodiment. [Figure 25] This is a block diagram showing the functional configuration of the learning system for the similarity calculation model used in the eighth, ninth, and tenth embodiments. [Figure 26] This is a schematic diagram illustrating the model training method for similarity calculation in the eighth, ninth, and tenth embodiments. [Figure 27] This is a block diagram showing the functional configuration of the similarity calculation model training data generation unit, as shown in Figure 25. [Figure 28] This is a schematic diagram illustrating the calculation method for similarity in the eighth, ninth, and tenth embodiments. [Figure 29] This is an external view of a computer for realizing a hypothesis generation system and a learning data generation system, etc., according to each embodiment of this invention. [Figure 30] This is a block diagram showing the hardware configuration of the computer shown in Figure 29. [Modes for carrying out the invention]

[0020] In the following descriptions and drawings, identical parts are given the same reference number. Therefore, detailed descriptions of them will not be repeated.

[0021] 1. First Embodiment [composition] Referring to Figures 1 to 7, a hypothesis generation system 100 according to the first embodiment of this invention will be described. Referring to Figure 1, the hypothesis generation system 100 receives input text 110 and keywords 112 as input. In this example, keywords 112 are shown as separate from the input text 110, but as will be described later, keywords 112 may be extracted from the input text 110.

[0022] The hypothesis generation system 100 receives a first input, which is the input text 110, and a second input, which will be described later. like A selector 1 has a second input that receives the text of a hypothesis that has already been created, and initially selects and outputs the input text 110, and in subsequent processing, selects and outputs the hypothesis text. 18 and selector 1 18 The output of and keyword 112 are taken as input, and based on these, selector 1 18 It includes a related text creation unit 114 for generating related text from input text, and a related text storage unit 116 for storing the related text generated by the related text creation unit 114.

[0023] The hypothesis generation system 100 further includes a hypothesis generation neural network 122 that is pre-trained to generate and output hypotheses based on either the input text or the related text stored in the related text storage unit 116, a hypothesis storage unit 124 for storing the hypotheses output by the hypothesis generation neural network 122, and related text Storage section 116The system includes a selector 120 for selecting one of several related texts stored in the system and outputting it as part of the input to the hypothesis-generating neural network 122. The second input of the selector 118 is one of the hypotheses stored in the hypothesis memory unit 124. The output of the selector 118 is connected to the input of the hypothesis-generating neural network 122, along with the input of the related text creation unit 114.

[0024] The hypothesis generation system 100 further includes a hypothesis generation control unit 12 for controlling the selectors 118, 120 and the hypothesis generation neural network 122, with the input of input text 110 as the first stage, and after a hypothesis is output from the hypothesis generation neural network 122 as the second and subsequent stages, in which the hypothesis generation system 100 inputs one of the hypotheses stored in the input text 110 or hypothesis storage unit 124 and one of the related texts selected by the selector 120 to the hypothesis generation neural network 122 to generate a new hypothesis. 6 include.

[0025] In this example, the related text generation unit 114 outputs multiple related texts using the keyword 112 along with the text output by the selector 118 in each of the multi-stage repetitions. However, the invention is not limited to such embodiments. For example, the first stage may take the input text 110 and the keyword 112 as input, and from the second stage onward, multiple related texts may be generated using the hypothesis selected by the selector 118 and the keyword extracted from the hypothesis. Alternatively, from the second stage onward, the hypothesis selected by the selector 118, the initially input keyword 112, and further keywords extracted from the hypothesis may be added as input.

[0026] Figure 2 shows the functional configuration of the related text generation unit 114 in Figure 1. Referring to Figure 2, the related text generation unit 114 includes a question generation unit 140 for generating one or more question sentences 142 using the text output by the selector 118 and the keyword 112, a large corpus 144 for storing a large amount of text, for example, collected in advance from the web, and a question answering unit 146 for each question sentence 142, which searches the large corpus 144 to extract answer passages from the large corpus 144 that consist of one or more sentences containing the answer to that question sentence, ranks them, filters them to retain the passages containing the appropriate answer, and outputs them. The answer passages extracted from the large corpus 144 by the question answering unit 146 become related text 148 for the text input to the related text generation unit 114.

[0027] Figure 3 shows the input-output relationship of the hypothesis-generating neural network 122. Referring to Figure 3, the hypothesis-generating neural network 122 is given input 170, which consists of a string formed by concatenating the [CLS] token, which indicates the beginning of the input, the input text (output of selector 118), [SEP], and the related text and the [SEP] token. Each word constituting these strings is assumed to have been converted to a word ID beforehand. The hypothesis-generating neural network 122 is pre-trained to output a hypothetical output word sequence 172 in response to this input 170. The beginning of the output word sequence 172 is a vector corresponding to the [CLS] token. In this embodiment, this vector is not used.

[0028] The question generation unit 140 shown in Figure 2 could be, for example, a rule-based system or one that uses a neural network generation model. One rule-based method involves replacing parts of the input text with question words. This method also varies depending on the language. In the case of Japanese, for example, questions can be generated by changing nouns in the input text to "who" or "what," or by excluding some nouns and noun phrases and adding strings such as "why," "how," or "when." Alternatively, questions can be generated by adding question expressions to nouns in the input text. Examples of question expressions include "What does ~ want to do?", "What will ~ do?", "What will ~ use?", "What will ~ do?", "What is ~ in?", "What is ~ good about?", "What is ~ bad about?", "How will ~ be done?", "How will ~ be used?", "Why will ~ be done?", and "Why will ~ be used?". Questions can be generated by adding these to nouns.

[0029] Conversely, you can add phrases like "why," "how," or "using what" before the entered text, or add strings like "and what happens," "what happens as a result," or "what kind of trouble occurs as a result" after the entered text.

[0030] Furthermore, to generate the question, it is also possible to delete a portion of the input text and then apply one of the rules described above. In this case, since a portion of the input text is deleted, the resulting question will be more abstract than one generated from the input text alone.

[0031] In this embodiment, a generative model consisting of a neural network is used as the question generation unit 140. The training data for this model can be a set of inputs and questions derived from those inputs that are related to the input in some way. For example, one could have "I like Nagoya." as the input and "What kind of tourist spots in Nagoya do you like?" as the output.

[0032] It is conceivable to create the training data manually. However, in this embodiment, the training data is created mechanically on a large scale from web text or the like. In this embodiment, the training data for the question generation unit 140 is generated as follows.

[0033] Figure 4 is a block diagram showing the functional configuration of the learning data generation system 200 for generating learning vectors for the neural network of the question generation unit 220. Referring to Figure 4, the learning data generation system 200 includes a large corpus 210 and a question sentence + pre-context extraction unit 212 that focuses on question sentences among the texts stored in the large corpus 210 and extracts the question sentence and the pre-context consisting of a predetermined number of sentences immediately preceding the question sentence. The method for discovering question sentences differs depending on the target language. For example, in Japanese, sentences ending with "ka." can be treated as question sentences. In English, sentences beginning with an interrogative word, or sentences beginning with "Do" or "Does," can be treated as question sentences.

[0034] The learning data generation system 200 further includes a learning data generation unit 214 for generating learning data from each pair of question sentences and previous contexts extracted by the question sentence + previous context extraction unit 212, a learning data storage unit 216 for storing the learning data generated by the learning data generation unit 214, and a learning processing unit 218 for performing learning on the question generation unit 220 using the learning data stored in the learning data storage unit 216.

[0035] For example, if the text contains the sentence, "What does it mean when I feel feverish with cold-like symptoms, but I don't have a fever?", and immediately before that, the preceding sentence is, "I thought it was strange, so I took my temperature several times, but it stayed between 36.5 and 37 degrees. I feel extremely unwell, I have no appetite, and I'm very fatigued, so I can't go to work. It's even difficult to get out of bed, so I haven't been able to go to the hospital," then the former, i.e. the question, will be the output, and the latter, i.e., the preceding sentence, will be the input, and training data will be generated.

[0036] The training data structure for the question generation unit 220 consists of pairs of the preceding context of a question and the question itself. In other words, the parameters of the question generation unit 220 are trained so that when the preceding context is input to the question generation unit 220, the output of the question generation unit 220 generates the question that follows that preceding context.

[0037] There are various other methods for automatically generating question sentences using text that includes questions. In addition to the training data mentioned above, it is also possible to automatically extract some of the expressions (such as nouns) included in the output and add them to the input. By doing so, the neural network can be trained to generate questions that include the expressions given in the input. Alternatively, it is also possible to pre-identify the type of question (why type, what type, how type, etc.) and add information specifying the type of question to the input. By doing so, the question generation unit 220 can be trained to generate questions of a specific type. In this case, inputs for the question generation unit 220 may be created for each of the multiple question types so that different types of questions are generated for a single input text.

[0038] In addition to those mentioned above, other possible question types include those related to writing style (standard language, honorific language), tense (past tense or not), whether or not personal pronouns are included, and whether or not it is a request. These question types may be added to the input along with specific keywords. Details of this process will be discussed later, referring to Figure 16.

[0039] Figure 5 shows the configuration of the learning data generation system 250 that generates training data for the hypothesis-generating neural network 122. Referring to Figure 5, the learning data generation system 250 includes a web text archive 262 containing a large amount of web text collected in advance from the web, and an inter-sentence semantic relationship DB 260 for storing a large number of text pairs that represent predetermined inter-sentence semantic relationships, which are generated from pairs of word sequences representing causal relationships extracted in advance from the web text archive 262 and the text passages containing them, and obtained after processing such as formatting, omission and completion. An inter-sentence semantic relationship is, for example, a specific relationship between two text pairs, similar to the causal relationships described earlier. Examples include solution relationships (the first text is a problem, and the second text is a solution to that problem), and purpose-situation / action / state relationships (the first text is a purpose, and the second text is a situation, action, or state for that purpose).

[0040] Each text pair included in the Inter-Sentence Semantic Relationship DB260 is extracted from the Web Text Archive 262, and each text pair in the Inter-Sentence Semantic Relationship DB260 is associated with access information to the corresponding portion of the Web Text Archive 262.

[0041] The learning data generation system 250 further extracts the passage containing each text pair from the web text archive 262 based on the access information assigned to that text pair, for each text pair included in the inter-sentence semantic relationship DB 260, and generates learning data for the hypothesis generation neural network 122 from the original text pair and the extracted passage. 4 The training data generated by the training data generation unit 264 is stored in the training data storage unit 266. The learning unit 268 reads this training data from the training data storage unit 266 and trains the hypothesis generation neural network 122.

[0042] The learning data generation unit 264 includes a record reading unit 290 for reading records containing each text pair from the inter-sentence semantic relationship DB 260, a record splitting unit 292 for splitting the records read by the record reading unit 290 into a first text, a second text, and access information, a corresponding text reading unit 294 for reading a passage containing that text pair from the web text archive 262 using the access information output by the record splitting unit 292, and a learning data assembly unit 296 for assembling and outputting learning data using the first text and second text output by the record splitting unit 292 and the passage output by the corresponding text reading unit 294.

[0043] Figure 6 shows a concrete example of the records in the inter-sentence semantic relationship DB 260 and the web text in the web text archive 262 described above. Referring to Figure 6, let's assume, for example, that there is a passage 300 in the web text archive 262. By performing an inter-sentence semantic relationship determination and generation process 302 using, for example, a pre-trained neural network, inter-sentence semantic relationship knowledge 304 can be obtained.

[0044] Inter-sentence semantic relationship knowledge 304 is obtained, for example, by extracting causal relationship pairs of texts from passage 300. Inter-sentence semantic relationship knowledge 304 concerning causal relationships, consisting of a cause part 320 and a consequence part 322, is obtained by performing omissions and completions on each of these texts, for example, manually or automatically, such as adding a string 310 present in passage 300 as the subject. The same applies to other inter-sentence relationships.

[0045] Figure 7 schematically illustrates the process of obtaining training data 330 from inter-sentence semantic relationship knowledge 304 and passages 300 extracted from web text. Referring to Figure 7, the cause part 320 of the inter-sentence semantic relationship knowledge 304 becomes the input 340 of the training data 330. The passage 300 becomes the related text 342 of the training data 330. And the consequence part 322 of the inter-sentence semantic relationship knowledge 304 becomes the output 344 of the training data 330.

[0046] [Operation] The hypothesis generation system 100 according to the first embodiment described above operates as follows. Referring to Figure 1, the hypothesis continuous generation control unit 126 controls the selector 118 so that it first selects an input to the first input. As a result, the input text 110 and the keyword 112 are input to the hypothesis generation system 100. The keyword 112 is input to the related text creation unit 114, and the input text 110 is input to the related text creation unit and the first input of the selector 118.

[0047] Referring to Figure 2, the question generation unit 140 generates one or more question sentences 142 based on the input text 110 and keywords 112 and provides them to the question answering unit 146. For each of the given one or more question sentences 142, the question answering unit 146 extracts one or more answer passages containing the answer to that question from the large corpus 144 and outputs them as related text 148.

[0048] The hypothesis continuous generation control unit 126 controls the selector 120 so that it selects the first related text from the related text storage unit 116, and a word sequence formed by concatenating the outputs of both selector 118 and selector 120 is input to the hypothesis generation neural network 122. At this time, a [CLS] token is added to the beginning of the input word sequence, a [SEP] token for separation is inserted between the output of selector 118 and the output of selector 120, and the same [SEP] token is added to the end of the output of selector 120.

[0049] The word sequences output by the hypothesis-generating neural network 122 in response to this input are stored in the hypothesis memory unit 124. Multiple outputs may be selected from the hypothesis-generating neural network 122. The same process is repeated while switching the related text selected by the selector 120. The first stage of processing ends when the above processing is completed for all related texts stored in the related text memory unit 116.

[0050] In the second stage, the hypothesis continuous generation control unit 126 controls selectors 118 and 120 so that selector 118 selects the second input and selector 120 selects the second related text. Selector 118 selects the first hypothesis stored in the hypothesis storage unit 124 and inputs it to the related text creation unit 114. The related text creation unit 114 then processes this hypothesis. Theory Based on the text and keyword 112, multiple related texts are created and output to the related text storage unit 116, similar to the first stage. Selector 120 selects and outputs the first related text from among the related texts stored in the related text storage unit 116. The word sequence formed by concatenating the hypothesis text output by selector 118 and the related text output by selector 120 is input to the hypothesis generation neural network 122, and the output of the hypothesis generation neural network 122 is stored in the hypothesis storage unit 124. The above process is then repeated to use all related texts stored in the related text storage unit 116.

[0051] When all related texts stored in the related text storage unit 116 have been used, the hypothesis continuous generation control unit 126 controls the selector 118 to select the second hypothesis stored in the hypothesis storage unit 124. The text of the second hypothesis is input to the related text creation unit 114, and multiple related texts are created and stored in the related text storage unit 116. The hypothesis continuous generation control unit 126 controls the selector 120 to select and output the first related text from the related text storage unit 116. The outputs of selector 118 and selector 120 are concatenated and given to the hypothesis generation neural network 122. The output of the hypothesis generation neural network 122 is stored in the hypothesis storage unit 124 as a new hypothesis. Subsequently, the hypothesis continuous generation control unit 126 controls the selector 120 to select the second related text and executes the above-described process.

[0052] In this way, for each hypothesis stored in the hypothesis memory unit 124, related text is created from that hypothesis and the keyword 112. Then, that hypothesis and all the related texts stored in the related text memory unit 116 are combined to generate a new hypothesis. This process is repeated.

[0053] This process is performed, and hypothesis generation is terminated when a predetermined termination condition is met. The termination condition is, for example, when the number of generated hypotheses exceeds a certain number.

[0054] As described above, according to this embodiment, keywords 112 are added to the input text 110 and provided to the hypothesis generation system 100, and the related text creation unit 114 generates multiple related texts based on the keywords 112. These related texts are added to the input text 110 and input to the hypothesis generation neural network 122. Through such experiments, it was found that interesting hypotheses that could not be easily predicted from the input could be obtained. It was also found that by inputting a specific keyword, there was a tendency to obtain hypotheses that included that keyword. Furthermore, the basis for the generation of these hypotheses can be understood. That is, in order to generate each hypothesis, related text is obtained from the web via question answering and used to generate the hypothesis. As a result, it is possible to check why the final generated hypothesis turned out the way it did by looking at the related text used in the creation of the hypothesis. As a result, the reliability of each hypothesis is also increased.

[0055] 1A. First variation In the first embodiment described above, the training of the hypothesis-generating neural network 122 uses training data consisting of an input word sequence, a related text word sequence, and an output word sequence, as shown in Figure 7. However, the present invention is not limited to such embodiments. The training data may also be modified to include information indicating the type of inter-sentence semantic relationship knowledge 304 obtained from the input 340 and output 344 (e.g., causal relationship, solution relationship, purpose-situation / action / state relationship, etc.). Figure 8 shows the process of generating training data for this purpose.

[0056] Referring to Figure 8, in this modified example, the training data 360 includes an input 340 consisting of the causal part 320 of inter-sentence semantic relation knowledge 304, related text 342 and inter-sentence semantic relation type 370, and output 344. The combination of related text 342 and inter-sentence semantic relation type 370 functions as related text.

[0057] When training the hypothesis-generating neural network 122 in this manner, the input for hypothesis generation must include, as in the first embodiment, the input text 110 shown in Figure 1, related text, and words or symbols representing semantic relationships. In this case, the semantic relationship can be defined by the user during hypothesis generation and always assigned the same value (for example, a symbol representing a causal relationship) to the input, or an arbitrary value representing a semantic relationship can be assigned to the input during hypothesis generation. In the former case, hypotheses based on relationships according to the values ​​defined during hypothesis generation will be obtained, while in the latter case, there is a higher possibility of obtaining a wider variety of hypotheses.

[0058] 1B. Second variation The method for generating training data for the hypothesis-generating neural network is not limited to the first embodiment and the first modified example described above. For example, the method shown in Figure 9 is also possible. Referring to Figure 9, it is assumed that a passage 380 exists in the web text archive, which includes a cause section 394 and a consequence section 396 as causal relationships. The passage 380 contains text 390 corresponding to the cause section 394 and text 392 corresponding to the consequence section 396.

[0059] In generating training data, this passage 380 may be used as the related text for the cause section 394 and the consequence section 396. When using such training data, the process of taking the cause section 394 and passage 380 as input and extracting the consequence section 396 can be learned. As a result, if the passage contains both the cause section and the consequence section, inputting that passage along with the cause section as related text and generating a hypothesis increases the probability of obtaining the consequence section as output.

[0060] Figure 17 shows an example of a system for generating training data to perform such processing on a neural network for hypothesis generation. Referring to Figure 17, this training data generation system 680 includes a large corpus 210 that stores a large amount of text, such as a web text archive, and a text pair extraction unit 690 that extracts text pairs that have a specific relationship with each other and passages (passage 380 shown in Figure 9) that contain those text pairs (for example, texts 390 and 392 shown in Figure 9), ranks them, and filters and outputs the passages that contain the appropriate answers. The specific relationship here includes causal relationships, solution relationships, purpose-situation / action / state relationships, etc.

[0061] The learning data generation system 680 further includes a semantic relationship estimation unit 694 that estimates the semantic relationship between the two texts constituting the text pair extracted by the text pair extraction unit 690, and a related text creation unit 692 for generating related text using the passage 380 extracted by the text pair extraction unit 690. Although the related text creation unit 692 generates related text, in this example it uses the extracted passage 380 as is.

[0062] The learning data generation system 680 further includes a learning data generation unit 696 that generates learning data by taking the first sentence (cause in the case of a causal relationship) from the text pairs extracted by the text pair extraction unit 690 as input, the extracted passage as related text, and the second sentence (consequence in the case of a causal relationship) as output, and concatenating these together. The learning data output by the learning data generation unit 696 is stored in the learning data storage unit 698, and the learning processing unit 700 uses this learning data to train the hypothesis generation neural network 604, so that the hypothesis generation neural network 604 learns to generate a sentence corresponding to the output contained in the related text in response to the input.

[0063] Furthermore, returning to Figure 9, the text 400, consisting of texts 390 and 392 corresponding to the cause section 394 and the consequence section 396, may be deleted from the passage 380. The resulting passage 382 is then used as the related text for training the hypothesis-generating neural network 604. By training the data in this way, the hypothesis-generating neural network 604 can be trained to output content not mentioned in either (consequences to the input) based on the input and related text.

[0064] 1C. Third variation The training data for the hypothesis-generating neural network is not limited to those described above. For example, content words or phrases included in the output section of the training data may be added to the input as related text. Figure 10 shows the functional configuration of a training data generation system 440 that generates training data for a hypothesis-generating neural network 452 that performs such training.

[0065] Referring to Figure 10, the learning data generation system 440 includes a web text archive 262, an inter-sentence semantic relationship DB 260, and a learning data generation unit 450 for generating learning data for a hypothesis-generating neural network 452 using the contents stored therein.

[0066] The learning data generation unit 450 includes a record reading unit 290, a record splitting unit 460 for splitting records representing inter-sentence semantic relationships read from the inter-sentence semantic relationship DB 260 by the record reading unit 290 into a cause unit, a consequence unit, and access information, a corresponding text reading unit 294 for reading corresponding text, which is a passage containing the representation of the causal relationship being processed, from the web text archive 262 using the access information output by the record splitting unit 460, a content word or phrase extraction unit 462 for extracting content words or phrases contained in the second text from the second text output by the record splitting unit 460, and a learning data assembly unit 464 for assembling and outputting learning data using the first text and second text output by the record splitting unit 460, the words and phrases output by the content word and phrase extraction unit 462, and the passage output by the corresponding text reading unit 294.

[0067] Like this to The generated training data is stored in the training data storage unit 266. The learning unit 268 uses this training data to train the hypothesis generating neural network 452. When the hypothesis generating neural network 452 generates hypotheses, it receives input text 110 and keywords 112, similar to those shown in Figure 1, generates related text, and then inputs the input text 110, related text, and keywords 112 into the hypothesis generating neural network 452. As a result, hypotheses are obtained as output to the hypothesis generating neural network 452. In this case, the user may be asked to input a different keyword than keyword 112.

[0068] According to this modification, for example, if you specifically want to obtain hypothesis generation results related to "fisheries," you can train the hypothesis generation neural network 452 to include the keyword "fisheries." As a result, when you specifically want to obtain hypotheses in a particular field, you can obtain hypotheses in the desired field with high accuracy.

[0069] 2. Second Embodiment In the first embodiment described above, the related text creation unit 114 shown in Figure 1 generates the question sentence 142 using a question generation unit 140 that uses a neural network, as shown in Figure 2. However, this invention is not limited to such embodiments.

[0070] Figure 11 shows the functional configuration of a related text generation unit 470 that can be used in place of the related text generation unit 114 shown in Figure 2 in the hypothesis generation system according to this second embodiment. Referring to Figure 11, the related sentence generation unit 470 includes a web search unit 480 that performs a web search on the internet 472 using the input keyword 112 as a key and collects snippets 482 consisting of multiple sentences containing the keyword 112. In this second embodiment, these snippets 482 are used as related text 474.

[0071] These embodiments also provide the same effects and benefits as the first embodiment.

[0072] 3. Third Embodiment Figure 12 shows yet another example of the related text creation unit 490 used in the hypothesis generation system according to the third embodiment of this invention. The hypothesis generation system according to this invention can also be realized by using this related text creation unit 490 instead of the related text creation unit 114 shown in Figure 1.

[0073] Referring to Figure 12, the related text creation unit 490 includes a keyword extraction unit 500 for extracting keywords from the input text 110 (see Figure 1), a question generation unit 140 for generating one or more question sentences 142 based on the keywords extracted by the keyword extraction unit 500, a question answering unit 146 for extracting passages containing answers to the question sentences from the large corpus 144 using each of the question sentences 142, and a word extraction unit 502 for extracting content words or phrases from each of the passages containing answers extracted by the question answering unit 146. The words or phrases extracted by the word extraction unit 502 are used as related text 492.

[0074] In this embodiment, as in the first embodiment, words or phrases as related text can be added to the input and fed into the hypothesis-generating neural network. As a result, the same effects as in the first embodiment can be obtained.

[0075] 4. Fourth Embodiment Figure 13 shows the functional configuration of a hypothesis generation system 510 according to the fourth embodiment of the present invention. Referring to Figure 13, the hypothesis generation system 510 includes a related text creation unit 520 that receives input text 110 and generates and outputs related text; a related text storage unit 116 for storing the related text generated by the related text creation unit 520; a selector 522 for sequentially selecting one of a plurality of related texts stored in the related text storage unit 116; and a pre-trained hypothesis generation neural network 524 that receives input concatenating the input text 110 and the output of the selector 522 and generates and outputs a hypothesis for the input text 110.

[0076] The hypothesis generation system 510 further includes a hypothesis memory unit 124 for storing multiple hypotheses output by the hypothesis generation neural network 524, and a hypothesis continuous generation control unit 526 for controlling the generation of hypotheses by the hypothesis generation neural network 524, the generation of related text by the related text creation unit 520, and the selection of related text by the selector 522. This control generates related text from only the input text 110, from each of the multiple hypotheses stored in the hypothesis memory unit 124 and the input text 110, or from each of the hypotheses, and combines these related texts with the input text 110 to input to the hypothesis generation neural network 524 to generate even newer hypotheses. This control unit executes control 530 of the related text creation unit 520 and control 528 of the selector 522.

[0077] Referring to Figure 14, the related text generation unit 520 includes a question generation unit 560 that receives the hypotheses generated so far and the input text 110 as input and combines them to generate a new question sentence 562, and a question answering unit 146 that searches the large corpus 144 for passages containing answers to the question sentence 562, extracts answer passages containing answers to the question sentence 562 from the large corpus 144, ranks them, filters them to retain passages containing appropriate answers, and outputs them as related text 540.

[0078] There are various ways to generate the question sentence 562 in the question generation unit 560. For example, suppose the input text 110 is "Global warming is progressing," and one of the hypotheses generated by the processing up to the previous stage is "Sea surface temperature is rising." One example of a question sentence combining the two is, "Why is global warming progressing and sea surface temperature rising?" Such a question sentence can be generated simply by combining the initial input and one of the hypotheses from the previous stage, and finally adding the string "why?". Other types of question sentences can be generated in a similar manner.

[0079] The related text generation unit 520 generates a question using only the input text 110, for example, when no hypothesis has been generated yet. After a hypothesis has been generated, it generates a question by combining the newly generated hypothesis with the input text 110.

[0080] By generating the question in this way, the question 562 provided to the question answering unit 146 becomes a question with a clearer target. As a result, the related text that the question answering unit 146 searches and extracts from the large corpus 144 is more likely to be highly relevant to the input.

[0081] 5. Fifth Embodiment As described above, the hypothesis generation system of this invention allows for the generation of multiple hypotheses based on a single input. In the above embodiment, the relationship between these hypotheses cannot be specifically defined. Of course, there may be cases where a subsequent hypothesis is generated based on a preceding hypothesis. However, when looking at the series of hypotheses together, there is no particularly clear connection to be found.

[0082] However, if there is a narrative structure, such as a line of reasoning or argument, between the series of hypotheses, it is thought that the persuasiveness of the series of hypotheses will increase. For this purpose, in this fifth embodiment, the concept of categories is introduced.

[0083] In this embodiment, "category" is used to indicate the progression of hypothesis content between the preceding and succeeding hypotheses, specifying the semantic relationship, line of reasoning, and topical progression between the preceding and succeeding hypotheses. For example, "continuation," "contrast," and "exemplification" are possible. By specifying a progression of hypotheses such as "continuation → contrast → exemplification" across a series of hypotheses and generating them in order, it becomes possible to give the series of hypotheses a narrative structure. As a result, it may be possible to generate more interesting hypotheses with a compelling narrative.

[0084] Figure 15 shows the functional configuration of the hypothesis generation system 590 according to the fifth embodiment. Referring to Figure 15, the hypothesis generation system 590 includes a selector 600 having a first input that receives input text 110 and a second input that receives another sequence of words different from the input text 110 (a hypothesis generated by the processing up to the previous stage), an association text creation unit 570 connected to receive the output of the selector 600 and for generating association text for the input based on the input, an association text storage unit 572 for storing the association text generated by the association text creation unit 570, a hypothesis generation neural network 604, and a hypothesis storage unit 124 for storing the hypotheses output by the hypothesis generation neural network 604. Note that keyword input is not used in this embodiment. However, the invention is not limited to such embodiments, and, similar to the first embodiment, keyword input may be received in addition to the input text 110 and used in the creation of association text in the association text creation unit 570.

[0085] The configuration of the related text generation unit 570 is the same as that of the related text generation unit 114 shown in Figure 2, but this embodiment differs from the first embodiment in the configuration of the training data for the neural network that constitutes the question generation unit 140. This will be explained later.

[0086] The hypothesis generation system 590 further includes a selector 602 for selecting either a first input that receives input text 110 or one of the newly stored hypotheses in the hypothesis storage unit 124 and providing it as input to the hypothesis generation neural network 604; a category setting storage unit 608 for storing a sequence of categories to be sequentially set for the generated hypotheses; and a hypothesis continuous generation control unit 606 for controlling selectors 600 and 602 to select an input according to the processing stage each time a new hypothesis is stored in the hypothesis storage unit 124, and for providing information indicating the corresponding category (category information) from the category sequence stored in the category setting storage unit 608 as input to the hypothesis generation neural network 604.

[0087] The hypothesis-generating neural network 604 is provided with the output of the selector 602 (input text 110 or a newly generated hypothesis), one of the related texts stored in the related text storage unit 572, and category information from the hypothesis continuous generation control unit 606.

[0088] The related text storage unit 572 stores multiple related texts, and although not shown in Figure 15, the output from the related text storage unit 572 includes a selector for selecting one of the related texts stored in the related text storage unit 572 and providing it to the hypothesis generation neural network 604. This selector is controlled by the hypothesis continuous generation control unit 606, and for example, when the hypothesis selected by selector 600 is switched, the related texts stored in the related text storage unit 572 are selected in order and provided to the hypothesis generation neural network 604. Through this process, when a new hypothesis is generated, at least the same number of hypotheses as the related texts stored in the related text storage unit 572 are newly generated and stored in the hypothesis storage unit 124.

[0089] Figure 16 shows the functional configuration of the learning data generation system 620, which generates training data for a neural network for question generation, and is included in the related text creation unit 570 shown in Figure 15. Referring to Figure 16, the learning data generation system 620 includes a question sentence + pre-context extraction unit 212 for extracting question sentences and their pre-context from a large corpus 210, and a learning data generation unit 630 for generating training data for a question sentence generation neural network from the question sentences and their pre-context extracted by the question sentence + pre-context extraction unit 212. The configuration of the large corpus 210 and the question sentence + pre-context extraction unit 212 is the same as that shown in Figure 4.

[0090] The learning data generation unit 630 includes a text separation unit 640 for separating the text of the question and the preceding context extracted by the question + preceding context extraction unit 212 from each other, a partial expression extraction unit 642 that automatically extracts a part of the expression from the text separated by the text separation unit 640 upon receiving the question, and outputs an expression in which that part has been replaced with a variable, a style identification unit 644 that identifies the style of the text expression of the question (e.g., everyday language, honorific language) and outputs information indicating that style, and the tense of the question The system includes a tense identification unit 646 that identifies and outputs information representing tense, a question type identification unit 648 that identifies the type of question sentence (why type, what type, how type, etc.) and outputs information indicating the type of question sentence, and a concatenation unit 650 that separates the output of the partial expression extraction unit 642, the output of the style identification unit 644, the output of the tense identification unit 646, and the output of the question type identification unit 648 from each other using the [SEP] token, concatenates them with the previous context, and further concatenates the concatenated previous context with the question sentence to output it as learning data 634.

[0091] When various types of information are added to the preceding context in this way, during actual question generation, by adding desired values ​​as the above-mentioned information to the input, it becomes possible to generate questions containing intended expressions, questions of intended style, questions of intended type, and questions of intended tense using a neural network. Figure 16 shows an example of combining values ​​that specify each category. However, these categories are independent of each other. Therefore, when creating training data, it is also possible to specify only some of the values ​​that represent these categories.

[0092] Furthermore, nouns and other elements can be extracted from the question text extracted by the question text + pre-context extraction unit 212 and added to the pre-context. By doing so, questions containing nouns included in the input can be generated by the neural network for question generation. In addition to the information described above, information indicating whether or not to include personal pronouns in the question, or whether or not to make the question a request, can also be added to the pre-context.

[0093] 6. Sixth Embodiment Figure 18 shows the functional configuration of the hypothesis generation system 730 according to the sixth embodiment of this invention. This hypothesis generation system 730 receives input text 110, first generates related text, and then repeatedly performs the process of generating new related text based on the generated related text. After a predetermined termination condition is met, for example, when a predetermined number of related texts have been generated, the input text 110 is used to generate new related texts. and team These are combined and input into the hypothesis-generating neural network 122. Through this process, longer hypotheses are generated.

[0094] Referring to Figure 18, the hypothesis generation system 730 includes a selector 740 having a first input and a second input that receive input text 110, an association text creation unit 570 that receives the output of the selector 740 and generates one or more association texts based on the output of the selector 740, and an association text storage unit 572 for storing the association texts generated by the association text creation unit 570. The configuration of the association text creation unit 570 and the association text storage unit 572 is the same as that shown in Figure 15.

[0095] The hypothesis generation system 730 further includes a selector 742 that selects one of the related texts stored in the related text storage unit 572 and provides it to the second input of the selector 740 to cause the related text creation unit 570 to generate a new related text; a selector 120 that, after the related text creation unit 570 has finished generating related texts, sequentially selects and outputs the related texts stored in the related text storage unit 572; a pre-trained hypothesis generation neural network 122 that generates hypotheses upon receiving an input which is a concatenation of the input text 110 and the output of the selector 120; a hypothesis storage unit 124 for storing hypotheses generated by the hypothesis generation neural network 122; and a hypothesis generation control unit 744 that controls the selectors 740 and 120 to select appropriate inputs in the first hypothesis generation cycle and subsequent cycles, thereby causing the hypothesis generation neural network 122 to repeatedly generate hypotheses. In this embodiment, the process of obtaining multiple related texts from one related text is repeated. Through this process, these related texts expand in a tree-like relationship. As will be described later, in hypothesis generation, a series of related texts that exist along a path from the root of this tree toward the leaves are linked together, and a long hypothesis is generated from them. Therefore, the related text storage unit 572 also stores information for each related text, which input the related text was obtained from, that is, whether it was input text 110 or one of several related texts.

[0096] In this embodiment, during the first cycle of hypothesis generation, the selector 740 selects the input text 110 and inputs it to the related text creation unit 570. The related text creation unit 570 responds to the input text 110 by generating one or more related texts. The related text storage unit 572 stores these one or more related texts. Once the generation of related texts from the input text 110 is complete, the selector 742 selects these related texts one by one and provides them sequentially to the second input of the selector 740. The selector 740 inputs these related texts to the related text creation unit 570. As a result, one or more related texts are newly stored in the related text storage unit 572.

[0097] As the process of creating new related text using the related text stored in the related text storage unit 572 is repeated, the number of related texts stored in the related text storage unit 572 will eventually exceed a predetermined number. At that point, the selector 742 will terminate the selection of new related text.

[0098] Once the selector 742 has finished selecting new related texts, the selector 120 sequentially selects paths from the root to each leaf in the tree described above using the related texts stored in the related text memory 572, and then sequentially selects related texts along those paths, that is, it chains the related texts along each path and inputs them to the hypothesis generation neural network 122 along with the input text 110. The hypothesis generation neural network 122 takes the input text 110 and the related texts output sequentially by the selector 120 as input and generates new hypotheses. The hypothesis memory 124 stores these generated hypotheses in the hypothesis memory 124. These new hypotheses become long hypotheses obtained based on the linked related texts.

[0099] When the hypothesis generation neural network 122 generates a hypothesis using all the interconnected related texts stored in the related text memory unit 572, the hypothesis generation by the hypothesis generation system 730 is completed.

[0100] According to this hypothesis generation system 730, a hypothesis is generated by first generating multiple related texts, and then combining each of these related texts with the input text 110. As a result, it has the effect of generating long hypotheses with a simple process.

[0101] 7. Seventh Embodiment Figure 19 shows the functional configuration of the hypothesis generation system 770 according to the seventh embodiment of the present invention. Referring to Figure 19, the hypothesis generation system 770 includes an association text creation unit 114, an association text storage unit 116, selectors 118 and 120, and a hypothesis storage unit 124, similar to those shown in Figure 1; an input formatting unit 780 that combines the output of selector 118 and the output of selector 120 to form a sentence; a hypothesis generation neural network 782 that has the same function as the hypothesis generation neural network 122 shown in Figure 1, but receives a sentence formatted by the input formatting unit 780 as input to generate hypotheses; and a hypothesis continuous generation control unit 784 that controls each part of the hypothesis generation system 770, including selectors 118 and 120, to generate multiple hypotheses in response to the input of input text 110 and keywords 112.

[0102] The difference between this hypothesis generation system 770 and the hypothesis generation system 100 shown in Figure 1 is that the input format of the hypothesis generation neural network 782 is different from that of the hypothesis generation neural network 122 in Figure 1, and therefore an input formatting unit 780 is provided. In other respects, the hypothesis generation system 770 has the same configuration as the hypothesis generation system 100.

[0103] The input to the hypothesis-generating neural network 122 shown in Figure 1 is [CLS] + word sequence from input text 110 + [SEP] + word sequence from related text + [SEP]. In contrast, the input formatting unit 780 concatenates the input text 110 and the related text from selector 120 to form a natural sentence which is then input to the hypothesis-generating neural network 782. For example, the input formatting unit 780 combines the input text 110 and the related text from selector 120, formats it to generate the word sequence "[CLS] + word sequence from related text + "ga," + word sequence from input text 110 + [SEP]", and inputs it to the hypothesis-generating neural network 782.

[0104] For example, suppose the input text 110 is "We will develop a dialogue system," and the keyword 112 is "elderly people," and the related text obtained is "Elderly people have difficulty securing means of transportation." The input formatting unit 780 concatenates and formats these two sentences to generate the sentence "Elderly people have difficulty securing means of transportation, but we will develop a dialogue system," which is then input to the hypothesis generation neural network 782.

[0105] In order for the hypothesis-generating neural network 782 to generate hypotheses based on these inputs, the training data must also be in the same format as the output of the input formatting unit 780 described above. In other words, during training, the related text must also be in the same format as the output of the input formatting unit 780. To It is necessary to remember this and concatenate it with input text 110 to form a single sentence, which will then be used as input for the hypothesis-generating neural network 782.

[0106] 8. Eighth Embodiment Figure 20 shows the functional configuration of a training data generation system 810 for training a hypothesis-generating neural network 826 according to the eighth embodiment of this invention. Here, the hypothesis-generating neural network 826 to be trained is not particularly limited, but it is assumed to have the same configuration as the hypothesis-generating neural network 122 shown in the embodiment of Figure 1. The system for generating hypotheses using the hypothesis-generating neural network 826 may also be the same as the hypothesis generation system 100 shown in Figure 1.

[0107] This embodiment is characterized by a method for generating training data for the hypothesis-generating neural network 826.

[0108] The learning data generation system 810 includes an inter-sentence semantic relationship DB 260 and a learning data generation unit 820 that generates learning data for a hypothesis-generating neural network 826 based on text pairs stored in the inter-sentence semantic relationship DB 260 and a web text archive 262. The learning data generated by the learning data generation unit 820 is stored in a learning data storage unit 822. The learning unit 824 trains the hypothesis-generating neural network 826 using the learning data stored in the learning data storage unit 822, so that the hypothesis-generating neural network 826 generates hypotheses in response to input.

[0109] In this embodiment, the data generation process for training data differs from that in the first embodiment. As a result, the hypothesis-generating neural network 826 is thought to generate different hypotheses from the hypothesis-generating neural network 122 shown in Figure 1.

[0110] The learning data generation unit 820 includes a record reading unit 830 that reads text pairs contained in each record of the inter-sentence semantic relationship DB 260, and a record splitting unit 832 that splits the text of the record read by the record reading unit 830 into a first sentence (cause part in the case of a causal relationship) and a second sentence (consequence part in the case of a causal relationship).

[0111] The learning data generation unit 820 further includes a question generation unit 834 for generating one or more question sentences using the cause unit output by the record splitting unit 832, a question answering unit 836 for searching the web text archive 262 for multiple answer passages, each containing an answer to one or more questions generated by the question generation unit 834, extracting them from the web text archive 262, filtering them to retain passages containing appropriate answers, and outputting them, and a response storage unit 838 for storing the multiple answer passages output by the question answering unit 836.

[0112] The learning data generation unit 820 further includes a similarity calculation unit 840 for calculating the similarity between each of the multiple answer passages stored in the response memory unit 838 and the consequence section (corresponding to the output in hypothesis generation) output from the record splitting unit 832; an associated text selection unit 842 for selecting the answer passage corresponding to the highest similarity as the associated text for the cause section (input) based on the similarity calculated for each of the multiple answer passages by the similarity calculation unit 840; and a learning data assembly unit 844 for assembling learning data for the hypothesis generation neural network 826 by concatenating the cause section output by the record splitting unit 832 as input, the answer passage selected by the associated text selection unit 842 as associated text, and the consequence section output by the record splitting unit 832 as output.

[0113] The method for calculating similarity by the similarity calculation unit 840 will be described later with reference to Figures 25 to 28.

[0114] In this embodiment, the question generation unit 834 generates a question sentence from the cause unit, and the question answering unit 836 searches the web text archive 262 for multiple answer passages for that question sentence, extracts them from the web text archive 262, and filters them to retain passages containing appropriate answers. Then, the similarity calculation unit 840 and the related text selection unit 842 select the answer passage that is most similar to the conclusion unit. As a result, training data for the hypothesis generation neural network 826 is created by inputting the cause unit, the related text that is most closely related to the conclusion unit of the causal relationship having that cause unit, and the conclusion unit of that causal relationship.

[0115] 9. Ninth Embodiment Figure 21 shows the functional configuration of the learning data generation system 850 according to the ninth embodiment of this invention. In the ninth embodiment, the hypothesis-generating neural network 866 to be trained is not particularly limited, but it is assumed to have the same configuration as the hypothesis-generating neural network 122 shown in the embodiment of Figure 1. The system for generating hypotheses using the hypothesis-generating neural network 866 may also be the same as the hypothesis-generating system 100 shown in Figure 1.

[0116] This embodiment is characterized by a method for generating training data for the hypothesis-generating neural network 866.

[0117] The learning data generation system 850 includes an inter-sentence semantic relationship DB 260 similar to that of the eighth embodiment, and a learning data generation unit 860 that generates learning data for the hypothesis-generating neural network 866 based on text pairs stored in the inter-sentence semantic relationship DB 260 and the web text archive 262. The learning data generated by the learning data generation unit 860 is stored in the learning data storage unit 862. The learning unit 864 trains the hypothesis-generating neural network 866 using the learning data stored in the learning data storage unit 862, so that the hypothesis-generating neural network 866 generates hypotheses in response to input.

[0118] In this embodiment, the data generation process for training data differs from both the generation process in the first embodiment and the eighth embodiment. As a result, it is believed that the hypothesis-generating neural network 866 generates hypotheses that are different from those of the hypothesis-generating neural network 122 shown in Figure 1 and the hypothesis-generating neural network 826 shown in Figure 20.

[0119] The learning data generation unit 860, as shown in Figure 20, includes a record reading unit 830 that reads text pairs contained in each record of the inter-sentence semantic relationship DB 260, and a record splitting unit 832 that splits the text of the record read by the record reading unit 830 into a first sentence (cause part in the case of a causal relationship) and a second sentence (consequence part in the case of a causal relationship).

[0120] The learning data generation unit 860 further includes a question generation unit 870 for generating one or more question sentences using the outcome unit output by the record splitting unit 832, a question answering unit 872 for extracting multiple answer passages from the web text archive 262, each containing an answer to one or more questions generated by the question generation unit 870, and a response storage unit 874 for storing the multiple answer passages extracted by the question answering unit 872.

[0121] The learning data generation unit 860 further includes a similarity calculation unit 876 for calculating the similarity between each of the multiple answer passages stored in the response memory unit 874 and the cause section (corresponding to the input in hypothesis generation) output from the record splitting unit 832; a related text selection unit 878 for selecting the answer passage corresponding to the highest similarity as the related text based on the similarity calculated for each of the multiple answer passages by the similarity calculation unit 876; and a learning data assembly unit 880 for assembling learning data for the hypothesis generation neural network 866 by concatenating the cause section output from the record splitting unit 832 as input, the answer passage selected by the related text selection unit 878 as the related text, and the consequence section output from the record splitting unit 832 as the output.

[0122] The method for calculating similarity by the similarity calculation unit 840 is the same as in the eighth embodiment and will be described later with reference to Figures 25 to 28.

[0123] According to this embodiment, the question generation unit 870 generates a question sentence from the consequence unit, and the question answer unit 872 extracts multiple answer passages for that question sentence from the web text archive 262. Then, the similarity calculation unit 876 and the related text selection unit 878 select the answer passage that is most similar to the cause unit. As a result, by combining the input text corresponding to the cause unit of the causal relationship, the related text that is most closely related to the input text 110 from the answer passages obtained from the consequence unit of that causal relationship, and the consequence unit of that causal relationship, a hypothesis generation neural network 86 Six training data sets are created.

[0124] 10. Tenth Embodiment Figure 22 shows the functional configuration of the tenth embodiment of this invention. Referring to Figure 22, the learning data generation system 920 according to this tenth embodiment includes a learning data generation unit 930 for generating learning data for a hypothesis-generating neural network 936. The learning data generated by the learning data generation unit 930 is stored in a learning data storage unit 932. The learning unit 934 uses the learning data stored in the learning data storage unit 932 to train the hypothesis-generating neural network 936, so that the hypothesis-generating neural network 936 can be used in place of the hypothesis-generating neural network 122 shown in Figure 1.

[0125] The learning data generation unit 930 includes a portion that combines the components of the learning data generation unit 820 shown in Figure 20 and the learning data generation unit 860 shown in Figure 21. More specifically, the learning data generation unit 930 includes a record reading unit 830, a record splitting unit 832, a question generation unit 834, a question answering unit 836, a web text archive 262, a response storage unit 838, and a similarity calculation unit 840, similar to those shown in Figure 20. The learning data generation unit 930 further includes a question generation unit 870, a question answering unit 872, a response storage unit 874, and a similarity calculation unit 876, similar to those shown in Figure 21.

[0126] The learning data generation unit 930 further includes an associated text selection unit 940 that reads the answer passage with the highest similarity calculated by the similarity calculation unit 840 from the response storage unit 838 and the answer passage with the highest similarity calculated by the similarity calculation unit 876 from the response storage unit 874, selects the answer passage with the higher similarity as associated text and outputs it, and a learning data assembly unit 942 that takes the cause part output by the record splitting unit 832 as input, the answer passage output by the associated text selection unit 940 as associated text, and the consequence part output by the record splitting unit 832 as output, combines these, and saves them as learning data in the learning data storage unit 932.

[0127] The process up to the similarity calculation by the similarity calculation units 840 and 876 is the same as in the eighth and ninth embodiments, respectively. In this learning data generation unit 930, the related text selection unit 940 compares the similarity calculated by the similarity calculation unit 840 with the similarity calculated by the similarity calculation unit 876, reads the answer passage corresponding to the higher of the two similarities from the response storage unit 838 or response storage unit 874, and outputs it to the learning data assembly unit 942. As a result, by combining the eighth and ninth embodiments, learning data can be generated in which an appropriate answer passage is used as the related text as the learning result.

[0128] 11. Eleventh Embodiment Figure 23 shows the configuration of a training data generation system 970 that generates training data for a hypothesis-generating neural network 986 according to the 11th embodiment of the present invention. In this example, the training data for the hypothesis-generating neural network 986 is a combination of input, related text, and output, similar to that generated by previous training data generation systems. However, this embodiment differs from other embodiments, particularly the 10th embodiment, in terms of the configuration of the related text.

[0129] Referring to Figure 23, the learning data generation system 970 according to this 11th embodiment includes a learning data generation unit 980 that generates learning data for the hypothesis-generating neural network 986. The learning data generated by the learning data generation unit 980 is stored in the learning data storage unit 982. The learning unit 984 is stored in the learning data storage unit 9 8 The hypothesis-generating neural network 986 is trained using the training data stored in 2. As a result, the hypothesis-generating neural network 986 can be used in place of the hypothesis-generating neural network 122 shown in Figure 1.

[0130] The learning data generation unit 980 includes a portion that combines the components of the learning data generation unit 820 shown in Figure 20 and the learning data generation unit 860 shown in Figure 21. More specifically, the learning data generation unit 98 Unit 0 includes a record reading unit 830, a record splitting unit 832, a question generation unit 834, a question answering unit 836, a web text archive 262, a response storage unit 838, a similarity calculation unit 840, and a related text selection unit 842, similar to those shown in Figure 20. Learning data generation unit 9 8 The 0 further includes a question generation unit 870, a question answering unit 872, a response storage unit 874, a similarity calculation unit 876, and a related text selection unit 878, similar to those shown in Figure 21.

[0131] The learning data generation unit 980 further takes the cause part output by the record splitting unit 832 as input, the related text formed by combining the answer passage output by the related text selection unit 842 and the answer passage output by the related text selection unit 878, and the consequence part output by the record splitting unit 832 as output, combines these, and stores them as learning data in the learning data storage unit 982. 0 include.

[0132] In this eleventh embodiment, the process up to the selection of answer passages by the related text selection unit 842 and the related text selection unit 878 is the same as in the eighth and ninth embodiments, respectively. In this learning data generation unit 980, the learning data assembly unit 990 connects the answer passages selected by the related text selection unit 842 and the answer passages selected by the related text selection unit 878 and incorporates them into the learning data as related data. In this case, [SEP] is inserted between the answer passages.

[0133] Figure 24 shows an example of the structure of training data output by the training data assembly unit 990. Referring to Figure 24, this training data 1010 has the format "[CLS] + "Develop a dialogue system" (input 1020) + [SEP] + First related text 1022 + [SEP] + Second related text 1024 + [SEP] + "Support the mobility of the elderly" (output 1026). The first related text 1022 and the second related text 1024 are the answer passages obtained from the cause and consequence parts, respectively, that have the highest similarity to the cause and consequence parts.

[0134] In actual hypothesis generation, for example, in a configuration like the one shown in Figure 1, two related texts stored in the related text storage unit 116 can be selected, concatenated via [SEP], added to the input text 110, and input to the hypothesis generation neural network 986.

[0135] As a result, according to this 11th embodiment, the 8th and 9th embodiments are combined in a different manner than the 10th embodiment, and learning data is generated in which appropriate answer passages are combined as related texts as learning results.

[0136] 12.Similarity calculation In the eighth, ninth, tenth, and eleventh embodiments of this invention, for example, the similarity between the cause section and the answer passage, and between the conclusion section and the answer passage, is calculated. The method for calculating this similarity will be described below with reference to Figures 25 to 28.

[0137] Referring to Figure 25, in this embodiment, a similarity calculation model 1070 that performs training using deep learning is used to calculate similarity. Figure 25 shows the functional configuration of the learning system 1050 for training the similarity calculation model 1070.

[0138] Referring to Figure 25, the learning system 1050 includes a web text archive 262, a topic word extraction unit 1060 that extracts topic words from the web text archive 262, and a topic word storage unit 1062 for storing the topic words extracted by the topic word extraction unit 1060.

[0139] Here, a topic word is a particularly specific one. re However, for example, in web text archive 262, it refers to the N most frequently occurring nouns. However, predetermined stop words are not used as topic words.

[0140] The learning system 1050 further generates training data for the similarity calculation model 1070 based on the web text stored in the web text archive 262 and the topic words stored in the topic word storage unit 1062, using a similarity calculation model training data generation unit 106 4 This includes the following. The function of the similarity calculation model 1070 in this embodiment is to output a feature vector for each sentence or passage that is used to calculate the similarity between that sentence or passage and other sentences or passages. The configuration of the similarity calculation model training data generation unit 1064 and the configuration of the feature vectors necessary for this will be described later.

[0141] The learning system 1050 further includes a learning data storage unit 1066 for storing the learning data generated by the similarity calculation model learning data generation unit 1064, and a learning unit 1068 for training the similarity calculation model 1070 using the learning data stored in the learning data storage unit 1066.

[0142] Figure 26 schematically shows the learning configuration of the similarity calculation model 1070 used in this embodiment. Referring to Figure 26, the input text and the feature vectors calculated for that input text are used as training data for learning the similarity calculation model 1070.

[0143] The similarity calculation model 1070 includes a language model 1100 known as BERT (Bidirectional Encoder Representations from Transformers), and a vector output unit 1102 consisting of a linear layer + softmax layer that takes the output corresponding to the first "CLS" from the output of the language model 1100 as input. The BERT used as the language model 1100 is pre-trained and fine-tuned by a method described later.

[0144] The training data 1104 includes combinations of input text with [CLS] and [SEP] appended to the beginning and end, respectively, and feature vectors pre-calculated for that input text.

[0145] The feature vectors for the training data 1104 are generated as follows. From the web text archive 262 shown in Figure 25, one sentence to be used for feature vector generation is extracted as the input sentence. Topic words appearing in the three sentences, which consist of this sentence and the sentences before and after it, are identified. The number of these topic words is then denoted as m. In this case, if one topic word appears multiple times in the three sentences... only Even if it appears, the number of that topic word is considered to be 1.

[0146] As the feature vector, a vector with the same number of elements as the number of topic words stored in the topic word memory unit 1062 is used. For topic words that appear in the above three sentences, the value of the corresponding element is set to 1 / m, and for topic words that do not appear in the above three sentences, the value of the element is set to 0. The vector thus obtained is the feature vector of the above input sentence (1 sentence).

[0147] For each sentence included in web text archive 262, the feature vector can be automatically calculated in advance using the method described above.

[0148] Referring to Figure 26, in the training (fine-tuning) of the similarity calculation model 1070, the input text of the training data 1104 is input to the language model 1100. The output vector 1106 is obtained as the output of the vector output unit 1102 of the similarity calculation model 1070. During training, the parameters of the similarity calculation model 1070 are trained to minimize the sum of the losses L calculated by the following formula between the feature vectors of the training data 1104 and the output vector 1106.

[0149]

number

[0150] By performing this fine-tuning, the similarity calculation model 1070 learns to predict topic words that appear in the three sentences surrounding a given sentence when that sentence is input.

[0151] Figure 27 shows the control structure of the program that implements the similarity calculation model learning data generation unit 1064 shown in Figure 25 in flowchart form. Referring to Figure 27, this program includes a step 1130 that performs initial processing, a step 1132 that performs step 1134 for each text in the web text archive 262 shown in Figure 25, and a step 1136 that performs termination processing in response to the completion of step 1132 and terminates the execution of this program.

[0152] Step 1134 includes step 1140, which performs step 1142 for each sentence in the text to be processed.

[0153] Step 1142 includes step 1150, which generates a feature vector for the sentence to be processed, referring to the topic word storage unit 1062 shown in Figure 25, together with the sentences before and after it, using the method described above; step 1152, which follows step 1150, which combines the target sentence with the feature vector calculated in step 1150 to generate a record of training data; and step 1154, which saves the record of training data generated in step 1152 to the training data storage unit 1066 shown in Figure 25, thereby ending the execution of step 1142.

[0154] By running this program, training data for the similarity calculation model 1070 can be generated.

[0155] This section describes how to calculate the similarity between a given sentence or passage and a given answer passage using the similarity calculation model 1070 that has been trained in this manner. In the following explanation, as an example, we will describe the case in which the similarity between one of the hypothetical passages (hypothetical output word sequences) stored in the response memory unit 838 and the conclusion part of a text pair read from the inter-sentence semantic relationship DB 260 is calculated, as shown in Figure 20.

[0156] Referring to Figure 28, the word sequence 1180 of the output (consequence) is input to the similarity calculation model 1070, and the output vector 1184 is obtained as the output of the similarity calculation model 1070. On the other hand, the word sequence 1182 of the target passage is input to the similarity calculation model 1070, and similarly the output vector 1186 is obtained. Output vectors 1184 and 1186 are vectors of the same dimension, and the cosine similarity between them can be calculated. The calculated cosine similarity is taken as the similarity 1190 between the conclusion and the target passage. For each of the target passages, this cosine similarity calculation unit 1188 calculates the similarity value 1190, thereby identifying the passage with the maximum similarity.

[0157] In all of the eighth, ninth, tenth, and eleventh embodiments, the similarity can be calculated using this similarity calculation model 1070.

[0158] 13. Implementation by computer Figure 29 is an external view of an example of a computer system that realizes the above embodiment. Figure 30 is a block diagram showing an example of the hardware configuration of the computer system shown in Figure 29.

[0159] Referring to Figure 29, this computer system 1250 includes a computer 1270 having a DVD (Digital Versatile Disc) drive 1302, and a keyboard 1274, a mouse 1276, and a monitor 1272, all connected to the computer 1270, for user interaction. Of course, these are just examples of configurations for when user interaction is required, and any general hardware and software available for user interaction (e.g., touch panels, voice input, pointing devices in general) can be used.

[0160] Referring to Figure 30, the computer 1270 includes, in addition to the DVD drive 1302, a CPU (Central Processing Unit) 1290, a GPU (Graphics Processing Unit) 1292, a bus 1310 connected to the CPU 1290, GPU 1292, and DVD drive 1302, a ROM (Read-Only Memory) 1296 connected to the bus 1310 for storing the computer 1270's boot-up program, etc., a RAM (Random Access Memory) 1298 connected to the bus 1310 for storing program instructions, system programs, and work data, etc., and a non-volatile memory SSD (Solid State Drive) 1300 connected to the bus 1310. The SSD 1300 is for storing programs executed by the CPU 1290 and GPU 1292, as well as data used by programs executed by the CPU 1290 and GPU 1292. The computer 1270 further includes a network interface 1308 that provides a connection to a network 1286 enabling communication with other terminals, and a USB port 1306 that allows for the insertion and removal of a USB (Universal Serial Bus) memory 1284 and provides communication between the USB memory 1284 and various parts within the computer 1270.

[0161] Computer 1270 further includes an input / output interface 1304 connected to a bus 1310 with a microphone 1282, a speaker 1280, and a motion capture device (not shown), for reading audio signals, video signals, and text data generated by the CPU 1290 and stored in RAM 1298 or SSD 1300 according to instructions from the CPU 1290, performing analog conversion and amplification processing to drive the speaker 1280, digitizing the analog audio signal from the microphone 1282 and storing it in RAM 1298 or SSD 1300 at an address specified by the CPU 1290, and receiving motion capture signals from the motion capture device and storing them at an address specified by the CPU 1290.

[0162] In the above embodiment, the hypothesis generation systems 100, 510, 590, 730, and 770, the learning data generation systems 200, 250, 620, 680, 810, 850, 920, 970, and 1050, as well as the programs for realizing the functions of each of their components, the neural network parameters, and the neural network program are all stored in a storage medium of an external device (not shown) connected via, for example, the SSD 1300, RAM 1298, DVD 1278, or USB memory 1284 shown in Figure 30, or via network I / F 1308 and network 1286. Typically, this data and parameters are written to the SSD 1300 from an external source, for example, and loaded into the RAM 1298 when executed by the computer 1270.

[0163] The computer programs for operating this computer system to realize the functions of the hypothesis generation systems 100, 510, 590, 730, and 770, the learning data generation systems 200, 250, 620, 680, 810, 850, 920, 970, and 1050, and each of their components, are stored on a DVD 1278 inserted into the DVD drive 1302 and transferred from the DVD drive 1302 to the SSD 1300. Alternatively, these programs are stored on a USB memory 1284, the USB memory 1284 is inserted into the USB port 1306, and the programs are transferred to the SSD 1300. Alternatively, these programs may be transmitted to a computer 1270 via the network 1286 and stored on the SSD 1300.

[0164] The program is loaded into RAM1298 at runtime. Since training and testing neural networks involves a large amount of computation, it is preferable to implement each part of the embodiment of the present invention as an object program consisting of the computer's native code rather than a scripting language, especially for the program portion that performs numerical calculations.

[0165] The CPU 1290 reads the program from RAM 1298 according to the address indicated by an internal register called the program counter (not shown), interprets the instructions, reads the data necessary for executing the instructions from RAM 1298, SSD 1300, or other devices according to the address specified by the instructions, and executes the processing specified by the instructions. The CPU 1290 stores the execution result data at an address specified by the program, such as RAM 1298, SSD 1300, or a register within the CPU 1290. At this time, the value of the program counter is also updated by the program. The computer program may be loaded directly into RAM 1298 from DVD 1278, USB memory 1284, or via a network. In addition, some tasks (mainly numerical calculations) within the program executed by the CPU 1290 are dispatched to GPU 1292 according to the instructions included in the program or according to the analysis results when the CPU 1290 executes the instructions.

[0166] A program that implements the functions of each part according to the above embodiment in cooperation with the computer 1270 includes a plurality of instructions written and arranged to operate the computer 1270 to implement those functions. Some of the basic functions necessary to execute these instructions are provided by an operating system (OS) running on the computer 1270, a third-party program, or modules of various toolkits installed on the computer 1270. Therefore, this program does not necessarily have to include all the functions necessary to implement the system and method of this embodiment. This program only needs to include instructions that perform the operation of each of the above-described devices and their components by statically linking appropriate functions or functions of the "programming toolkit" in a controlled manner to obtain the desired result, or by dynamically linking to those functions during program execution. The method of operating the computer 1270 for this purpose is well known and will not be repeated here.

[0167] Furthermore, the GPU1292 is capable of parallel processing, allowing for the simultaneous, parallel, or pipelining execution of large amounts of computation associated with machine learning. For example, parallel computation elements discovered in the program during compilation, or during program execution, are dispatched from the CPU1290 to the GPU1292 as needed, executed, and the results are returned to the CPU1290 either directly or via a predetermined address in RAM1298, and assigned to a predetermined variable in the program.

[0168] The embodiments disclosed herein are illustrative and not limited to those embodiments. The scope of the present invention is defined by the claims, with reference to the detailed description of the invention, and includes all modifications within the meaning and scope equivalent to the wording contained herein. [Explanation of symbols]

[0169] 100, 510, 590, 730, 770 Hypothesis Generation System 110 Input Text 112 Keywords 114, 470, 490, 520, 570, 692 Related Text Creation Department 116, 572 Related text memory section 122, 452, 524, 604, 782, 826, 866, 936, 986 Hypothesis Generating Neural Networks 126, 526, 606, 784 Hypothesis Continuous Generation Control Unit 140, 220, 560, 834, 870 Question generation part 144, 210 large corpora 146, 836, 872 Question and Answer Section Related texts: 148, 342, 474, 492, 540, 1022, 1024 200, 250, 440, 620, 680, 810, 850, 920, 970 Training Data Generation System 212 Question text + previous context extraction part 214, 264, 450, 630, 696, 820, 860, 930, 980 Learning data generation unit 260 Inter-sentence semantic relationship database 268, 824, 864, 934, 984, 1068 Learning Department 304 Knowledge of inter-sentence semantic relationships 370 Types of inter-sentence semantic relationships 1070 Similarity Calculation Models 1100 language models 1188 Cosine Similarity Calculation Unit

Claims

1. A related text creation unit that receives text input and creates related text related to that text, A hypothesis generation device, comprising a hypothesis generation model that receives the aforementioned text and the aforementioned related text as input and is pre-trained to generate hypotheses from the aforementioned text.

2. The aforementioned related text creation unit, A question generation unit that generates one or more questions based on the aforementioned text, The hypothesis generation apparatus according to claim 1, further comprising: a question response unit for searching an existing text archive for a passage containing answers to the one or more questions generated by the question generation unit, and outputting it as the related text.

3. The hypothesis generation device further, The hypothesis generation apparatus according to claim 1, further comprising a hypothesis continuous generation unit that inputs the hypothesis generated by the hypothesis generation model into the related text creation unit in place of the input text, and generates a new hypothesis using the hypothesis generation model.

4. The hypothesis generation device further, A first selector that selects either the input text or the hypothesis generated by the hypothesis generation model and inputs it into the related text creation unit, The hypothesis generation apparatus according to claim 1, further comprising a second selector, which, as input to the hypothesis generation model, selects either the input text or the hypothesis generated by the hypothesis generation model, and inputs it together with the related text to the hypothesis generation model, thereby causing the hypothesis generation model to generate a new hypothesis.

5. The hypothesis generation device according to claim 1, further comprising an input selector that selects either the input text or the related text generated by the related text creation unit and inputs it to the related text creation unit.

6. The computer receives text input and creates related text associated with that text, A hypothesis generation method comprising: a hypothesis generation step in which a computer receives the text and the related text as input and inputs them into a hypothesis generation model that has been pre-trained to generate hypotheses from the text to generate a new hypothesis.

Images