Question generation system and question generation method

Abstract

To improve external knowledges given to a generation AI so as to improve the accuracy of an answer generated by the generation AI to a question.SOLUTION: The question generation system extracts a text from a predetermined range in a division unit obtained by dividing a target document, inputs the text to a generation AI, and acquires a new question sentence generated by the generation AI based on the text. Then, the question generation system calculates the similarity between the new question sentence and the question sentence stored in the storage unit. When it is determined that the similarity is equal to or less than the threshold, the question generation system outputs the generated new question sentence.SELECTED DRAWING: Figure 9

Description

[Technical Field]

[0001] The present invention relates to a question generation system and a question generation method. [Background technology]

[0002] There are question-and-answer systems such as chatbots that automatically output answers to user questions expressed in text or voice. The question-and-answer system stores in advance combinations (hereinafter sometimes referred to as QA data) of anticipated questions (hereinafter sometimes referred to as "Q sentences") and corresponding answers (hereinafter sometimes referred to as "A sentences"). The question-and-answer system searches the QA data based on the input user question, and retrieves and outputs the A sentence corresponding to the Q sentence that is most similar to the user question.

[0003] Here, the QA data is created in advance based on text (such as terms and conditions, a collection of regulations, or a manual) that describes information that forms the basis of questions and answers. For example, Patent Document 1 discloses a technology for generating QA data by processing text in a manual according to predetermined rules. There is also a method for generating QA data using artificial intelligence (AI). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Publication No. 2022-80691 Summary of the Invention [Problem to be solved by the invention]

[0005] In recent years, question-and-answer systems have emerged that use generative AI to automatically generate and output answers to user questions.

[0006] When generating AI answers based on previously learned knowledge, it may experience hallucination, where it outputs incorrect answers to user questions. To avoid hallucination, generating AI is controlled using a technique called RAG (Retrieval-Augmented Generation), which generates answers based on previously given external knowledge.

[0007] In the RAG method, a block of text (hereafter referred to as a "chunk") containing information similar to the user's question is given to the generative AI as external knowledge.

[0008] Here, for example, external knowledge is a chunk similar to the question among chunks divided into certain units based on a document base such as the number of characters or paragraph units, or a semantic base such as sentence units with similar meaning. External knowledge may also be a combination of a Q sentence similar to the user's question and the text from which the Q sentence was generated.

[0009] The accuracy of the answers generated by the generative AI depends heavily on the quality of the external knowledge it is given. If the quality of the external knowledge given to the generative AI is low, the accuracy of the answers generated by the generative AI to questions will decrease.

[0010] The present invention has been made in consideration of the above circumstances, and aims to improve the quality of external knowledge provided to a generation AI so as to improve the accuracy of the answers that the generation AI generates to questions. [Means for solving the problem]

[0011] As one aspect for solving the above problem, there is provided a question generation system for generating a question from a target document, the question generation system having a processor, a memory, and a storage unit for storing the question generated by a generation AI (Artificial Intelligence), wherein the processor extracts text from a predetermined range in a division unit obtained by dividing the target document, inputs the text to the generation AI, obtains a new question generated by the generation AI based on the text, calculates a similarity between the new question and the question stored in the storage unit, determines whether the similarity is below a threshold, and outputs the new question if it is determined that the similarity is below the threshold. [Effects of the Invention]

[0012] According to the present invention, the quality of external knowledge provided to a generation AI can be improved so that the accuracy of answers generated by the generation AI to questions can be improved. [Brief explanation of the drawings]

[0013] [Figure 1] FIG. 1 is a diagram showing an outline of a RAG that is a premise of an embodiment. [Figure 2] FIG. 1 is a diagram showing the configuration of a question generation system according to a first embodiment. [Figure 3] FIG. 3 is a diagram showing QA data according to the first embodiment. [Figure 4] FIG. 2 is a diagram showing context data according to the first embodiment. [Figure 5] FIG. 2 is a diagram showing an outline of question generation according to the first embodiment. [Figure 6] 10 is a flowchart showing a question generation process according to the first embodiment. [Figure 7] 10 is a flowchart showing a Q sentence generation and output process according to the first embodiment. [Figure 8] FIG. 10 is a diagram showing an outline of question generation according to the second embodiment. [Figure 9] 10 is a flowchart showing a Q sentence generation and output process according to the second embodiment. [Figure 10]FIG. 11 is a diagram showing an outline of question generation according to the third embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0014] The following description of the embodiments will be given with reference to the drawings. Note that the following embodiments do not limit the scope of the invention as claimed, and not all of the elements and combinations thereof described in the embodiments are necessarily essential to the solution of the invention.

[0015] In the following description, a program realizing each processing function unit is executed by a processor to perform a predetermined process using storage resources (e.g., memory) and / or communication interface devices (e.g., ports) as appropriate. The processor is, for example, a CPU (Central Processing Unit). Therefore, the process executed by each processing function unit may be a process performed by the processor or a computer having the processor.

[0016] In the following explanation, various information is expressed in table format, but the information is not limited to table format and may be in CSV (Comma Separated Values) or other formats. Since various information does not depend on the data format, for example, "XXX table" can be called "XXX information."

[0017] Furthermore, among the various information items, names such as "# (number)," "identification information," "ID," and "name" for items that include information to distinguish the corresponding record from other records may be interchangeable. For example, "user ID" may be interchangeable with "user identification information."

[0018] (Overview of RAG, which is the premise of the embodiment) Prior to describing the first to third embodiments, RAG (Retrieval-Augmented Generation), which is the premise of the embodiments, will be described. FIG. 1 is a diagram showing an overview of RAG, which is the premise of the embodiments. RAG methods include Method 1 and Method 2, which search for sentences containing answers to a user's question from a target document dc11 and generate an answer to the question based on the search results. Note that the target document dc1 may be either a document with a clause structure having hierarchical items such as chapters, sections, articles, paragraphs, and items, or a free-form document with no particular format. Examples of documents with a clause structure include, but are not limited to, insurance policies, product manuals, and collections of regulations.

[0019] In Method 1, an answer sentence is generated based on multiple chunks ck, which are created by dividing a target document dc1 based on formatting criteria such as the number of characters and line breaks. Each of the multiple chunks ck is converted into sentence vector information vc11. Meanwhile, a question sentence dc2 entered by the user is converted into sentence vector information vc2.

[0020] Then, by searching for vector information vc11 similar to vector information vc2, a chunk similar to the question is found. This similar chunk is called chunk ck1. Question d2 and chunk ck1 similar to question d2 are input to a generation AI (Artificial Intelligence) a1 along with a prompt such as, "Please generate an answer to {question} from {chunk similar to question}." The answer dc3 generated and output by generation AI a1 is then displayed on the display screen d1 to the user who inputted question d2.

[0021] In method 2, answer sentences are generated based on multiple QA pairs obtained by dividing the target document dc1 using a rule base or generation AI. The multiple QA pairs are combinations of question sentences (Q sentences) created from the target document dc1 and answer sentences (A sentences) corresponding to the question sentences. The multiple QA pairs are each converted into sentence vector information vc21.

[0022] Then, by searching for vector information vc21 similar to vector information vc2, a Q sentence similar to the question sentence is found. The sentence in the target document dc1 that is the source of the QA pair and contains this similar Q sentence is set as the source sentence se1. The question sentence d2 and the source sentence se1 are input to the generation AIa1 along with a prompt such as "Please generate an answer sentence from {source sentence of the QA pair} to {question sentence}." The answer sentence dc3 generated and output by the generation AIa1 is then displayed on the display screen d1 to the user who input the question sentence d2.

[0023] Alternatively, in method 2, the question d2 and the A sentence se2 corresponding to the searched similar Q sentence may be displayed as an answer sentence dc3 on the display screen d1 to the user who input the question d2.

[0024] In this way, the key to obtaining a high-quality answer from the generation AI a1 is how accurately to search for the Q statement necessary to identify the "chunk similar to the question statement," the "source statement for generating the QA pair," or the "source statement for generating the QA pair." In the embodiment described below, by extracting the question statement from the target document dc1 with high accuracy, the quality of the information input to the generation AI together with the user's question statement in the RAG mechanism is improved, enabling the generation AI to generate a more accurate answer.

[0025] [Embodiment 1] FIG. 2 is a diagram showing the configuration of the question generation system 100 according to the first embodiment.

[0026] The question generation system 100 includes a processor 111, a memory 112M, a network interface 113IF, and a storage 114.

[0027] The question generation system 100 is connected to an external storage 101 and a user terminal 102 via a network interface 113IF and a predetermined communication path.

[0028] The processor 111 loads a program from the storage 114 into the memory 112M and executes it, thereby realizing a Q statement generation unit 112a, a statement generation unit 112b, and a Q statement generation possibility determination unit 112c. These processing functions will be described later with reference to flowcharts.

[0029] The external storage 101 stores QA data 11 and context data 12 .

[0030] FIG. 3 is a diagram showing QA data 11 according to the first embodiment. The QA data 11 stores, for each piece of context data 12, a QA pair, which is a combination of a question sentence (Q sentence) extracted from the context data 12 and an answer sentence (A sentence) corresponding to the question sentence. The QA data 11 has columns for OAID 111, question sentence 112, answer sentence 113, and context ID 113. The OAID 111 is identification information for the QA pair. The question sentence 112 is the question sentence (Q sentence) that constitutes the QA pair. The answer sentence 113 is the answer sentence (A sentence) that constitutes the QA pair. The context ID 113 is identification information for the document (corresponding to the target document dc1 in FIG. 1) from which the corresponding QA pair was extracted, and includes information such as the chapter or section number, article number, etc.

[0031] Fig. 4 is a diagram showing the context data 12 according to the first embodiment. The context data 12 is information corresponding to the target document dc1 in Fig. 1. In the first embodiment, the context data 12 is insurance policy terms and conditions having a clause structure.

[0032] (Outline of Question Generation According to Embodiment 1) FIG. 5 is a diagram showing an outline of question generation according to the first embodiment.

[0033] First, in step S1A, chunks ck are extracted from the context data 12. In the first embodiment, chunks ck are multiple division units obtained by dividing the context data 12 based on a document base, a semantic base, or any other arbitrary criterion. A document base is a division method based on a sentence structure such as the number of characters or a paragraph unit. A semantic base is a division method based on the similarity between adjacent sentences, for example.

[0034] Next, in step S2A, it is determined whether or not a Q-sentence can be generated. Methods for determining whether or not a Q-sentence can be generated include a rule-based method (based on the sentence structure of the context data 12) and a method based on an index of hallucination suppression in sentence generation.

[0035] The rule base determines that a Q statement can be generated if the context data 12 has a clause structure, and determines that a Q statement cannot be generated if the context data 12 does not have a clause structure. Having a clause structure corresponds to the case where the context data 12 has a list structure within a paragraph, a table structure, or a multi-level list structure. A list structure is a structure in which items are listed in list form. A table structure is a structure in which items are listed in table form.

[0036] Furthermore, since a clause structure appears before a clause structure and has a preamble including a demonstrative pronoun for the clause structure, whether or not a sentence has a clause structure can also be determined by the presence or absence of a preamble.

[0037] Indicator-based hallucination suppression in sentence generation is a method in which a generated sentence reconstructed from an original sentence in context data 12 using a generation AI is determined to have a clause structure using a known indicator that indicates whether the generated sentence is equivalent to the original sentence.

[0038] If it is determined in step S2A that the context data 12 does not have a clause structure (step S2A: determined as not possible), then in step S3A, additional information for generating Q from the clause structure is obtained and added to the chunk ck. This additional information is selected based on the clause structure. For example, the additional information is the sentence at the next higher level in the clause structure relative to the smallest unit (the deepest part of a nested clause structure). However, if the text to be added contains the same word (noun) as the source, it will not be added. Furthermore, if the text exceeds a specific item unit (for example, a clause unit (a unit separated by "article" items)), the addition is terminated.

[0039] In the example of Figure 5, step S3A has chunk ck as follows: "1) The fee is calculated based on the insurance period. 2) For the insurance holder, if a payment request is made... 3) If a special clause has been signed..." Additional information for this chunk ck is "A. The insurance amount for life insurance will be paid using the calculation method specified in each of the following items."

[0040] Next, in step S4A, the chunk ck with the additional information is input to the generation AI, and the generation AI obtains a sentence based on the chunk ck. For example, let's assume that the generation AI obtains the sentence, "When a policyholder takes out insurance, the premium for the life insurance policy will be paid using a calculation method based on the insurance period."

[0041] Next, in step S5A, the sentence generated by the generation AI in step S4A is input to the generation AI along with a prompt, and a Q statement generated by the generation AI is obtained. Here, let us assume that the generation AI has obtained, for example, "How do you calculate the amount of life insurance coverage?"

[0042] On the other hand, if it is determined in step S2A that the context data 12 has a clause structure (step S2A: determined as possible), in step S5A, the chunk ck is input to the generation AI together with a prompt, and the Q sentence generated by the generation AI is obtained. The Q sentence obtained in step S5A is the object of the embodiment.

[0043] (Q sentence generation process according to the first embodiment) 6 is a flowchart showing the Q statement generation process according to embodiment 1. The Q statement generation process is executed in response to an instruction from the user.

[0044] First, in step S11, the Q statement generator 112a reads the target context data 12. Next, in step S12, the Q statement generator 112a sets a target range for the context data 12 read in step S11. Setting the target range is the same as the division of the context data 12 into chunks ck described above.

[0045] Next, in step S13, the Q statement generation unit 112a determines whether the target range set in step S12 meets the summarization conditions. The summarization conditions refer to a certain volume or greater, such as the number of characters in the target range exceeding a threshold. If the target range meets the summarization conditions (step S13: YES), the Q statement generation unit 112a proceeds to step S14. On the other hand, if the target range does not meet the summarization conditions (step S13: NO), the Q statement generation unit 112a proceeds to step S15. In step S14, the statement generation unit 112b summarizes the target range using a generation AI or the like. Note that steps S13 and S14 can be omitted. That is, the summary sentence of the division unit (target range) of the summary target generated by the generation AI is acquired, and the division unit corresponding to the division unit of the summary target is replaced with the acquired summary sentence. This adjusts the division unit to a volume usable by the generation AI and reduces noise contained in the division unit, allowing the generation AI to generate an appropriate answer.

[0046] Next, in step S15, the Q statement generator 112a executes a Q statement generation and output process. The details of the Q statement generation and output process will be described later with reference to FIG.

[0047] Next, in step S16, the Q statement generator 112a determines whether the number of times the Q statement generation and output process in step S15 has been executed (the number of times questions have been generated) is equal to or greater than a threshold. If the number of times the Q statement generation and output process has been executed is equal to or greater than the threshold (step S16 YES), the Q statement generator 112a ends the question statement generation process. On the other hand, if the number of times the Q statement generation and output process has been executed is less than the threshold (step S16 NO), the Q statement generator 112a proceeds to step S17.

[0048] In step S17, the Q statement generation unit 112a changes the target range to one different from the target range set in step S12. Here, for example, the target range is changed to another chunk ck. When step S17 ends, the Q statement generation unit 112a returns the process to step S13.

[0049] (Q sentence generation and output process according to the first embodiment) 7 is a flowchart showing the Q sentence generation and output process according to the first embodiment. First, in step S12a, the Q sentence generation unit 112a reads text in units of a predetermined number of lines from the target range set in step S12 or S17. Next, in step S12b, the Q sentence generation possibility determination unit 112c determines whether or not a clause structure exists in the read range read in step S12a. The determination of whether or not a clause structure exists is a determination of whether or not a Q sentence can be generated (difficulty level), and is the same as step S2A (FIG. 5).

[0050] The Q sentence generation possibility determination unit 112c moves the process to step S12c if there is a clause structure in the read range (step S12b YES), and moves the process to step S12h if there is no clause structure (step S12b NO).

[0051] In step S12c, the Q statement generation unit 112a sets the last line of the article structure determined to exist in step S12b as the target line. Next, in step S12d, the Q statement generation unit 112a determines whether there is an item of an article structure higher than the target line set in step S12c in the article structure. If an item of a higher article structure exists, the Q statement generation unit 112a proceeds to step S12e, and if there is no item of a higher article structure, the Q statement generation unit 112a proceeds to step S12h.

[0052] In step S12e, the Q statement generation unit 112a adds the line immediately above the target range, which is included in the item of the higher-level article structure determined to exist in step S12e, to the target range as additional information.The Q statement generation unit 112a then converts the target range with the additional information added into a sentence.

[0053] Next, in step S12f, the Q sentence generation unit 112a determines whether the sentence obtained by converting the target range to which one line was added in step S12e satisfies the summarization condition. If the summarization condition is met (step S12f YES), the Q sentence generation unit 112a proceeds to step S12g. On the other hand, if the summarization condition is not met (step S12f NO), the Q sentence generation unit 112a proceeds to step S12h. In step S12g, the sentence generation unit 112b summarizes the sentence in the target range using a generation AI or the like. Note that steps S12f and S12g can be omitted.

[0054] Next, in step S12h, the Q statement generation unit 112a inputs the sentences in the target range to the generation AI and obtains the Q statement generated by the generation AI. Next, in step S12i, the Q statement generation unit 112a outputs the Q statement obtained in step S12h.

[0055] In step S12b, the addition of the target range is determined based on whether or not there is a clause structure, but this is not limited to this and the decision may also be made based on an index of hallucination suppression in sentence generation, such as the well-known RAGAS (Retrieval-Augmented Generation ASsessment).

[0056] In step S12c, sentence conversion is started from the deepest part (last line) of the sentence structure, but it is not limited to this and may start from the top of the sentence structure or from a specified indent of the sentence structure.

[0057] In step S12e described above, the line up to the next higher in the article structure is added as additional information to the target range. However, this is not a limitation, and the line up to the top of the article structure may be added as additional information in order to the target range, and then the sentence may be converted. Alternatively, a restriction rule may be set in advance to restrict the addition of article structures along the way. For example, it may be specified that the line up to the Nth level in the article structure is to be added as additional information, or that it is to be added until an article number or chapter number appears, or it may be limited by the number of characters, etc.

[0058] In addition, in the Q statement generation output process in Figure 7, the scope of the article structure is expanded and Q statements are generated at the end, but Q statement generation may be performed at each expansion stage and QA data may be generated at every final stage.

[0059] In the first embodiment, a clause structure is used as an example, but the present invention is not limited to a clause structure, and any context data that has a sentence structure related to the sentences before and after the sentences, such as a paragraph with a chapter number, can be used.

[0060] (Effects of the First Embodiment) In the first embodiment, when a target document is determined to have a clause structure, additional information to be added to the text is determined based on the clause structure. Therefore, according to the first embodiment, when it is difficult to generate a Q sentence, such as when there is a sentence structure such as a list or a table, the question sentence is converted into a format suitable for generation, and a Q sentence can be generated with high accuracy. Furthermore, since high-quality Q&A sentences can be generated, the quality of external knowledge provided to the generation AI is improved, and the accuracy of the answers generated by the generation AI to user questions is improved.

[0061] [Embodiment 2] In embodiment 1, the difficulty of generating a Q sentence based on the target chunk is determined, and if the difficulty is high, additional information is added to the chunk based on the clause structure, and then the chunk is input to the generation AI, converted into a sentence, and a Q sentence is generated.

[0062] However, even if additional information is added to chunks based on the article structure, the generated Q sentences will have the same content as other similarly generated Q sentences, which may result in duplicate generation of similar Q sentences and loss of diversity in Q sentences.

[0063] Therefore, in embodiment 2, when adding additional information to a chunk for generating a Q sentence, if the similarity with other Q sentences is below a threshold, the range for searching for additional information is gradually expanded until the similarity exceeds the threshold, and the additional information is cumulatively added to the target range to generate the Q sentence.

[0064] In the description of the second embodiment, differences from the first embodiment will be mainly described, and overlapping descriptions will be omitted.

[0065] (Outline of Question Generation According to Embodiment 2) 8 is a diagram showing an outline of question generation according to embodiment 2. First, in step S1B, it is determined whether a Q sentence can be generated for chunk ck extracted from the context data 12.

[0066] Next, in step S2B, additional information for generating a Q sentence is obtained from the article structure and added to chunk ck that was determined to be unable to generate a Q sentence. Next, in step S3B, chunk ck with the additional information added in step S2B is input to the generation AI, and a sentence written by the generation AI based on this is obtained. Next, in step S4B, the sentence written by the generation AI obtained in step S3B is input to the generation AI along with a prompt, and the Q sentence generated by the generation AI is obtained.

[0067] Note that steps S1B to S4B are similar to steps S2A to S5A (FIG. 5), respectively.

[0068] Next, in step S5B, the ambiguity of the Q sentence generated in step S4B is determined. In step S4B, for example, the similarity with other Q sentences generated from the same target article is calculated, and if the similarity is below a threshold, it is determined that there is no ambiguity with other Q sentences. On the other hand, if the similarity exceeds the threshold, it is determined that there are many similarities with other Q sentences, that it is ambiguous, and that it cannot be distinguished from other Q sentences. Similarity can be determined using sentence vectors, frequency of appearance of phrases, etc.

[0069] If the Q sentence is determined to be unambiguous in step S5B, it is adopted as the Q sentence generation result and output. If the Q sentence is determined to be ambiguous, the process proceeds to step S6B.

[0070] In step S6B, additional information to be added to chunk ck is selected based on the article structure. When step S6B is completed, the process returns to step S2B.

[0071] The additional information selected in step S6B may be information at a higher level, the same level, or a lower level in the text structure of the chunk ck determined to be ambiguous.

[0072] Furthermore, the search range for additional information, such as a higher hierarchy, a lower hierarchy, or the same hierarchy, may be broadened as the value of the index representing ambiguity (for example, similarity) increases, so that more additional information is added. In other words, the method for expanding the search range for additional information may be changed depending on the similarity, and the additional information added to the chunk ck (text) may be changed.

[0073] For example, a first threshold and a second threshold smaller than the first threshold may be set, and if the similarity exceeds the first threshold, additional information may be searched for in a higher hierarchy, if the similarity is equal to or smaller than the first threshold and exceeds the second threshold, additional information may be searched for in the same hierarchy, and if the similarity is equal to or smaller than the second threshold, a Q statement may be output.

[0074] In step S2B, to which the process is transferred from step S6B, the additional information selected in step S6B is further added to chunk ck. The more the loop process of steps S2B to S6B is repeated, the more additional information is added to chunk ck.

[0075] In the comparative example, for example, the Q sentence generated from the two chunks "I" and "II" extracted from the context data 12 shown in FIG. 12, "1) The fee is calculated based on the insurance period," is "How do you calculate the fee?". Therefore, it is determined that the degree of similarity is high and the ambiguity is high.

[0076] On the other hand, in the second embodiment, for "1) The fee is calculated based on the insurance period" in "A," a sentence in the next higher level, "A The insurance amount for life insurance is paid using the calculation method specified in each of the following items," is added to the chunk ck as additional information. Similarly, for "1) The fee is calculated based on the insurance period" in "B," a sentence in the next higher level, "B The insurance amount for disability insurance is paid using the calculation method specified in each of the following items," is added to the chunk ck as additional information. When a Q sentence is generated based on the chunk ck with the additional information added, "How is the insurance amount for life insurance calculated?" is generated for "A," and "How is the insurance amount for disability insurance calculated?" is generated for "B." As can be seen, in the second embodiment, unambiguous Q sentences can be generated, thereby improving the accuracy of Q sentence generation.

[0077] (Q sentence generation and output process according to the second embodiment) 9 is a flowchart showing a Q statement generation and output process according to the second embodiment. In the second embodiment, the Q statement generation and output process of FIG. 9 is executed instead of the Q statement generation and output process (FIG. 7) according to the first embodiment. The Q statement generation and output process of the second embodiment differs from the Q statement generation and output process according to the first embodiment in that steps S12j to S12l are executed between steps S12h and S12i. Otherwise, the Q statement generation and output process of the second embodiment is similar to the Q statement generation and output process according to the first embodiment.

[0078] The Q statement generated in step S12h is assumed to be stored in a predetermined storage area.

[0079] In step S12j, the Q statement generation unit 112a determines whether the Q statement generated in step S23h is the first Q statement generated from the target context data 12. If it is the first Q statement generated (step S12jYES), the Q statement generation unit 112a returns the process to step S12h, and if it is the second or subsequent Q statement generated (step S12jNO), the process proceeds to step S12k.

[0080] In step S12k, the Q statement generation unit 112a calculates the similarity between the Q statement newly generated in step S12h and a Q statement generated in the past and stored in a predetermined storage area. Next, the Q statement generation unit 112a determines whether the similarity calculated in step S12k is equal to or less than a threshold. If the similarity is equal to or less than the threshold (step S12lYES), the Q statement generation unit 112a proceeds to step S12i, and if the similarity exceeds the threshold (step S12lNO), the Q statement generation unit 112a returns to step S12d.

[0081] In step S12i, the Q-statement generation unit 112a adopts the Q-statement whose similarity calculated in step S12k is equal to or less than the threshold value as the formal generation result and outputs it.

[0082] (Effects of the second embodiment) In the second embodiment, if a Q sentence generated based on a chunk generated from context data having a clause structure is ambiguous (the similarity with other Q sentences exceeds a threshold), the range of the chunk is expanded and additional information is added, and Q sentence generation is repeated until the ambiguity is eliminated. Therefore, according to the second embodiment, it is possible to generate a Q sentence with high accuracy and no ambiguity.

[0083] [Embodiment 3] In the second embodiment, as shown in FIG. 8, when it is determined that a Q statement cannot be generated (the chunk ck is complex (has a clause structure)), additional information is added to the chunk ck for generating the Q statement.

[0084] Here, even if the target context data 12 is an arbitrary document that does not have a clause structure, it is possible to generate an unambiguous Q sentence by adding additional information extracted from a sentence in a predetermined vicinity of the chunk ck to the chunk ck. In the following, a third embodiment relating to this point will be described.

[0085] In the explanation of the third embodiment, the differences from the second embodiment will be mainly explained, and overlapping explanations will be omitted.

[0086] (Outline of Question Generation According to the Third Embodiment) 10 is a diagram showing an overview of question generation according to embodiment 3. In question generation according to embodiment 3, first, in step S1C, chunks ck extracted from the context data 12 are input to the generation AI, and a sentence is obtained by the generation AI converting the chunks ck into sentences based on the chunks ck. The chunks ck are division units obtained by mechanically dividing the context data 12 according to a predetermined rule (number of characters, etc.), for example.

[0087] Next, in step S2C, the sentence generated by the generation AI in step S2C is input to the generation AI along with a prompt, and the Q sentence generated by the generation AI is obtained. Next, in step S3C, the ambiguity of the Q sentence generated in step S2C is determined. A Q sentence determined to be unambiguous in step S3C is adopted as the Q sentence generation result and output. On the other hand, if the Q sentence is determined to be ambiguous, processing proceeds to step S4C.

[0088] In step S4C, additional information to be added to chunk ck is selected. When step S4C is completed, the process returns to step S5C.

[0089] The additional information selected in step S4C may include a predetermined number of lines of text before and after the chunk ck determined to be ambiguous in the context data 12 from which the chunk ck is divided. Alternatively, the additional information may be selected using the same method as in the second embodiment.

[0090] In step S5C, the additional information selected in step S4C is added to the chunk ck. Step S1C is then executed again for the chunk ck to which the additional information has been added in step S5C.

[0091] (Effects of the third embodiment) In the third embodiment, even if the context data is a general document that does not have a clause structure, it is possible to generate an accurate Q sentence that does not have ambiguity, as in the second embodiment.

[0092] Although the embodiments of the present disclosure have been described above in detail, the present disclosure is not limited to the above-described embodiments and can be modified in various ways without departing from the spirit of the present disclosure. For example, the above-described embodiments have been described in detail to clearly explain the present invention, and the present disclosure is not necessarily limited to those having all of the described configurations. Furthermore, some of the configurations of the above-described embodiments can be added to, deleted from, or replaced with other configurations.

[0093] Furthermore, the above-described configurations, functional units, processing units, etc. may be partially or entirely implemented in hardware, for example, by designing them as integrated circuits. The above-described configurations, functions, etc. may also be implemented in software by a processor interpreting and executing a program that implements each function. Information such as the programs, tables, and files that implement each function can be stored in memory, storage devices such as HDDs and SSDs, or recording media such as IC cards, SD cards, and DVDs.

[0094] In addition, in the above-mentioned drawings, the control lines and information lines shown are those that are considered necessary for explanation, and do not necessarily show all the control lines and information lines that are actually implemented. For example, it may be considered that almost all components are actually connected to each other. [Explanation of symbols]

[0095] 100: Question generation system, 111: Processor, 112M: Memory, 112a: Q sentence generation unit, 112b: sentence generation unit, 112c: Q sentence generation possibility determination unit, a1: Generation AI.

Claims

1. A question generation system that generates a question sentence from a target sentence, the question generation system includes a processor, a memory, and a storage unit that stores the question generated by a generation AI (Artificial Intelligence), The processor: extracting text from a predetermined range in each division unit obtained by dividing the target document; inputting the text into the generating AI; Obtain the new question sentence generated by the generation AI based on the text; Calculating a similarity between the new question sentence and the question sentence stored in the storage unit; determining whether the similarity is equal to or less than a threshold; If it is determined that the similarity is equal to or less than a threshold, the new question sentence is output. A question generation system characterized by:

2. The question generation system of claim 1, The processor: If it is determined that the similarity exceeds a threshold, Add additional information included in a range obtained by expanding the predetermined range in the division unit to the text and input the added information to the generation AI; Obtain the new question sentence generated by the generation AI based on the text to which the additional information has been added; It is determined whether the similarity between the new question sentence and the question sentence stored in the storage unit is equal to or less than a threshold value. Each process is repeated until it is determined that the similarity is equal to or less than the threshold value. A question generation system characterized by:

3. The question generation system of claim 1, The processor: The method of expanding the predetermined range is changed depending on the degree of similarity, thereby changing the additional information included in the range obtained by expanding the predetermined range in the division unit. A question generation system characterized by:

4. The question generation system of claim 1, The division unit is a division unit obtained by dividing the target document based on the clause structure. A question generation system characterized by:

5. 5. The question generation system of claim 4, The processor: determining whether the target document has the clause structure; When it is determined that the target document has the clause structure, additional information included in a range expanded from the predetermined range to be added to the text is determined based on the clause structure. A question generation system characterized by:

6. 3. The question generation system according to claim 2, The division units are division units obtained by mechanically dividing the target sentence according to a predetermined rule. A question generation system characterized by:

7. The question generation system of claim 1, The processor: determining whether each of the division units satisfies a summary condition for generating a summary sentence; inputting, into the generating AI, a division unit to be summarized that satisfies the summarization condition among the division units; Obtaining a summary sentence of the summary target division unit generated by the generation AI; The division unit corresponding to the division unit to be summarized is replaced with the acquired summary sentence. A question generation system characterized by:

8. A question generation method executed by a question generation system that generates a question sentence from a target sentence, comprising: the question generation system includes a processor, a memory, and a storage unit that stores the question generated by a generation AI (Artificial Intelligence), the processor: extracting text from a predetermined range in each division unit obtained by dividing the target document; inputting the text into the generating AI; Obtain the new question sentence generated by the generation AI based on the text; Calculating a similarity between the new question sentence and the question sentence stored in the storage unit; determining whether the similarity is equal to or less than a threshold; If it is determined that the similarity is equal to or less than a threshold, the new question sentence is output. A question generation method characterized by having each process.

Images