Question generation method, knowledge extraction method, method for constructing question and response system, question generation program, knowledge extraction program, question generation system, and knowledge extraction system

Abstract

A question generation method according to the present invention comprises: a theme information reception step for receiving input of theme information for generating a question; an extraction step for extracting local information from the received theme information; a supplementary information generation step for generating supplementary information for the extracted local information; a sentence acquisition step for acquiring a sentence generated on the basis of a prompt containing the theme information, the local information, and the supplementary information by using a large-scale language model generated by learning local information as features; and a determination step for determining whether or not to set the sentence as a question related to the theme on the basis of a degree of content overlap and / or a thematic relevance of the sentence.

Description

Question generation method, knowledge extraction method, question answering system construction method, question generation program, knowledge extraction program, question generation system, and knowledge extraction system

[0001] The present invention relates to a question generation method, a knowledge extraction method, a question answering system construction method, a question generation program, a knowledge extraction program, a question generation system, and a knowledge extraction system.

[0002] In recent years, the transfer of skilled techniques has become an issue in the manufacturing industry. There is an urgent need to preserve the knowledge of skilled workers, especially undocumented tacit knowledge, in some form of formalized form. To formalize human tacit knowledge, questionnaires and interviews are typically used. Known techniques for formalizing tacit knowledge include, for example, a method of conducting interviews suitable for extracting tacit knowledge, as described in Non-Patent Document 1.

[0003] Yabutani Rie, "Proof of the Effectiveness of the Functional Approach Tacit Knowledge Extraction Method," a method for extracting tacit knowledge, Japanese Society for Artificial Intelligence Workshop Paper, SIG-KST-043-03 (2023-03-23)

[0004] Meanwhile, questionnaires, one of the conventional methods, do not impose time constraints on the questioner or respondent, but only obtain answers to pre-determined questions, making it difficult to obtain tacit knowledge. Furthermore, interviews make it easy to extract tacit knowledge because questions can be set based on the answers, but as shown in Non-Patent Document 1, they require skills and techniques on the part of the questioner. Therefore, there has been a demand for technology that does not require skills and techniques on the part of the questioner and can reliably extract tacit knowledge.

[0005] The present invention has been made in light of the above circumstances, and its purpose is to provide a question generation method, a knowledge extraction method, a question answering system construction method, a question generation program, a knowledge extraction program, a question generation system, and a knowledge extraction system that can extract tacit knowledge efficiently and reliably.

[0006] In order to solve the above-mentioned problems and achieve the object, a question generation method according to one embodiment of the present invention includes a theme information receiving step for receiving input of theme information for generating a question; an extraction step for extracting local information from the received theme information; a supplementary information generating step for generating supplementary information for the extracted local information; a sentence acquisition step for acquiring a sentence generated using a large-scale language model generated by learning the local information as a feature based on a prompt including the theme information, the local information, and the supplementary information; and a question review step for determining whether to set the sentence as a question related to the theme information based on the degree of content redundancy and / or thematic relevance of the sentence.

[0007] In one aspect of the present invention, in the question generation method of the above invention, if it is determined in the question examination step that the question should not be set as a question related to the theme information, the sentence acquired in the sentence acquisition step is set as a supplemental prompt that includes the sentence as an inappropriate example and includes instructions to encourage improvement of output, and the sentence acquisition step is executed again based on the prompt and the supplemental prompt.

[0008] A question generation method according to one aspect of the present invention, in the above invention, further includes a topic setting step of setting a topic of a question related to the theme information as a guideline for questions regarding the theme information, and in the sentence acquisition step, information related to the set topic of the question is added to the prompt.

[0009] In one aspect of the present invention, in the question generation method of the above invention, the topic setting step includes theme information, and sets the topic of the question using a large-scale language model generated by learning local information as features based on a prompt that encourages the generation of a topic of the question.

[0010] In one aspect of the question generation method of the present invention, in the above invention, the question examination step compares the similarity between information obtained by vectorizing the sentence and information obtained by vectorizing other sentences generated before the one to be evaluated, and determines whether the content overlaps.

[0011] In one aspect of the present invention, in the question generation method of the above invention, the question examination step determines whether the sentence is related to the set theme based on the relevance between information obtained by vectorizing the sentence and information obtained by vectorizing the theme information.

[0012] In one aspect of the question generation method of the present invention, in the above invention, the extraction step extracts the local information from the theme information by referring to a database that pre-stores local terms and sentences containing the local terms.

[0013] In one aspect of the question generation method of the present invention, in the above invention, the extraction step refers to a database that stores local terms and information obtained by pre-vectorizing sentences containing the local terms, and extracts the local information by comparing the similarity between the information and information obtained by vectorizing the theme information.

[0014] a first supplemental information generation step for generating first supplemental information for the extracted first local information; a sentence acquisition step for acquiring a sentence generated using a large-scale language model generated by learning local information as features based on the theme information, the first local information, and a prompt including the first supplemental information; a question examination step for determining whether or not to set the sentence as a question related to the theme information based on content overlap and / or theme relevance; a question output step for outputting a question determined to be set as the question by the question examination step; an answer acceptance step for receiving answer information from a respondent; a second extraction step for extracting second local information from the answer information; and a second supplemental information generation step for generating second supplemental information for the extracted second local information. and a knowledge information generating step of converting information of a series of processes including the question, the answer information, the second local information, and the second supplementary information into a set format to generate knowledge information.

[0015] A knowledge extraction method according to one aspect of the present invention, in the above invention, further includes an update step of updating the prompt by adding the answer information, the second local information, and the second supplementary information to the prompt, and repeatedly executes the sentence acquisition step, the question examination step, the quality output step, the answer acceptance step, the second extraction step, and the second supplementary information generation step, which are executed based on the updated prompt.

[0016] In the knowledge extraction method according to one aspect of the present invention, in the above invention, the knowledge information generating step converts the knowledge information into a knowledge graph format and a text format to generate the knowledge information.

[0017] A method for constructing a question-answering system according to one aspect of the present invention constructs a question-answering system that utilizes knowledge information generated using the knowledge extraction method according to the present invention.

[0018] A question generation program according to one aspect of the present invention causes a computer to execute the following steps: a theme information receiving step for receiving input of theme information for generating a question; an extraction step for extracting local information from the received theme information by referring to a memory unit; a supplemental information generating step for generating supplemental information for the extracted local information; a sentence acquisition step for acquiring a sentence generated using a large-scale language model that is generated by learning local information as a feature based on the theme information, the local information, and a prompt including the supplemental information; and a question review step for determining whether to set the sentence as a question related to the theme information based on the degree of content overlap and / or thematic relevance of the sentence.

[0019] A knowledge extraction program according to one aspect of the present invention includes a theme information receiving step of receiving an input of theme information for generating a question; a first extraction step of extracting first local information from the received theme information by referring to a storage unit; a first supplemental information generating step of generating first supplemental information for the extracted first local information; a sentence acquisition step of acquiring a sentence generated using a large-scale language model generated by learning local information as a feature, based on the theme information, the first local information, and a prompt including the first supplemental information; and a sentence acquisition step of analyzing the sentence based on content overlap and / or theme relevance. Based on this, the computer is caused to execute a question examination step of determining whether or not to set the sentence as a question related to the theme information, a question output step of outputting the question set by the question examination step, an answer acceptance step of accepting answer information from the answerer, a second extraction step of extracting second local information from the answer information, a second supplementary information generation step of generating second supplementary information for the extracted second local information, and a knowledge information generation step of converting information of a series of processes including the question, the answer information, the second local information, and the second supplementary information into a set format to generate knowledge information.

[0020] A question generation system according to one embodiment of the present invention includes a theme information receiving unit that receives input of theme information for generating questions; an extraction unit that extracts local information from the received theme information by referring to a memory unit; a supplemental information generation unit that generates supplemental information for the extracted local information; a sentence acquisition unit that acquires sentences generated using a large-scale language model that is generated by learning local information as features based on the theme information, the local information, and a prompt including the supplemental information; and a question review unit that determines whether to set the sentences as questions related to the theme information based on the degree of content overlap and / or thematic relevance of the sentences.

[0021] A knowledge extraction system according to one aspect of the present invention includes a theme information receiving unit that receives input of theme information for generating a question, a first extraction unit that extracts first local information from the received theme information by referring to a storage unit, a first supplemental information generating unit that generates first supplemental information for the extracted first local information, a sentence acquisition unit that acquires sentences generated using a large-scale language model that is generated by learning local information as features based on the theme information, the first local information, and a prompt including the first supplemental information, and a sentence acquisition unit that calculates content overlap and / or theme relatedness for the sentences. the system includes a question scrutiny unit that determines whether to set the sentence as a question related to the theme information based on the degree of the sentence scrutiny; a question output unit that outputs the question determined by the question scrutiny unit to be set as the question; an answer reception unit that receives answer information from a respondent; a second extraction unit that extracts second local information from the answer information; a second supplemental information generation unit that generates second supplemental information for the extracted second local information; and a knowledge information generation unit that converts information of a series of processes including the question, the answer information, the second local information, and the second supplemental information into a set format to generate knowledge information.

[0022] The question generation method, knowledge extraction method, question answering system construction method, question generation program, knowledge extraction program, question generation system, and knowledge extraction system according to the present invention have the effect of enabling tacit knowledge to be extracted efficiently and reliably.

[0023] FIG. 1 is a block diagram showing a schematic configuration of a knowledge processing system according to an embodiment of the present invention. FIG. 2 is a block diagram showing a configuration of a knowledge extraction device included in the knowledge processing system according to an embodiment of the present invention. FIG. 3 is a block diagram showing a configuration of a sentence generation device included in the knowledge processing system according to an embodiment of the present invention. FIG. 4 is a sequence diagram illustrating the flow of knowledge extraction processing according to an embodiment of the present invention. FIG. 5 is a diagram (part 1) illustrating an example of interaction with a respondent during knowledge extraction according to an embodiment of the present invention. FIG. 6 is a diagram (part 2) illustrating an example of interaction with a respondent during knowledge extraction according to an embodiment of the present invention. FIG. 7 is a sequence diagram illustrating the flow of knowledge extraction processing according to a first modified example of the present invention. FIG. 8 is a sequence diagram illustrating the flow of knowledge extraction processing according to a second modified example of the present invention. FIG. 9 is a diagram showing summary information in text format and knowledge graph format of undocumented tacit knowledge information extracted from a skilled technician in charge of equipment maintenance at a steelworks by the knowledge extraction processing of the present invention. 10A shows an example of a response from a large-scale language model provided in a RAG environment with knowledge information generated by the knowledge extraction process of the present invention, as a result of a question-answering system being asked to provide a response about the cause and countermeasures when damage occurs to the main machine (main drive or spindle) of a hot rolling line in a steelworks. Fig. 10B shows a comparison of a response from a large-scale language model provided in a RAG environment with a response from a large-scale language model not provided in a RAG environment, as a result of a question-answering system being asked to provide a response about the cause and countermeasures when damage occurs to the main machine (main drive or spindle) of a hot rolling line in a steelworks.

[0024] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. In all the drawings of the embodiment below, the same or corresponding parts are designated by the same reference numerals. Furthermore, the present invention is not limited to the embodiment described below.

[0025] (Embodiment) Fig. 1 is a block diagram showing a schematic configuration of a knowledge processing system according to one embodiment of the present invention. As shown in Fig. 1, the knowledge processing system 1 includes a knowledge extraction device 10, a sentence generation device 20, and a server 30. The knowledge processing system 1 is configured so that the knowledge extraction device 10 and the sentence generation device 20 can input and output data to and from each other, and the knowledge extraction device 10 and the sentence generation device 20 can each read information from the server 30. Note that in this embodiment, the knowledge extraction device 10 configures a question generation system and a knowledge extraction system using at least some of its components.

[0026] Here, data input and output between the knowledge extraction device 10 and the sentence generation device 20 can be performed by network communication via a network or cloud, or by contactless communication such as Bluetooth (registered trademark). Data can also be transferred via a disc recording medium such as a Universal Serial Bus (USB) memory, a Compact Disc (CD), a Digital Versatile Disc (DVD), or a Blu-ray (registered trademark) Disc (BD). The network is configured by appropriately combining wired and wireless communications, and is composed of a communication network such as the Internet or a mobile phone network. The network can be, for example, one or a combination of dedicated lines, public communication networks such as the Internet, local area networks (LANs), wide area networks (WANs), telephone communication networks such as mobile phones, public lines, and virtual private networks (VPNs).

[0027] 2 is a block diagram showing the configuration of a knowledge extraction device included in a knowledge processing system according to an embodiment of the present invention. The knowledge extraction device 10 includes a communication unit 11, an input / output unit 12, an extraction unit 13, a prompt generation unit 14, a question scrutiny unit 15, a knowledge information generation unit 16, a control unit 17, and a storage unit 18. The communication unit 11, the input / output unit 12, the extraction unit 13, the prompt generation unit 14, the question scrutiny unit 15, the control unit 17, and the storage unit 18 constitute a question generation device.

[0028] The communication unit 11 is, for example, a LAN interface board, a wired communication circuit for wired communication, or a wireless communication circuit for wireless communication. The LAN interface board, wired communication circuit, or wireless communication circuit is connected to a network. The communication unit 11, which functions as a transmitter and receiver, is connected to the network and communicates with the text generation device 20 and the server 30.

[0029] The input / output unit 12 may be configured, for example, with a touch panel display or a speaker / microphone. The input / output unit 12 as an input means may include an interface that inputs various information transmitted from an external server via a communication unit and outputs the information to the control unit 17. The input / output unit 12 also includes a user interface, such as a keyboard, input buttons, a lever, a touch panel for manual input superimposed on a display such as an LCD, or a microphone for voice recognition. An operator or the like can input predetermined information to the control unit 17 by operating the input / output unit 12. The input / output unit 12 as an output means displays predetermined images on a display monitor, displays characters or figures on the screen of a touch panel display, or outputs sound from a speaker, under the control of the control unit 17. That is, the input / output unit 12 is configured to notify predetermined information to the outside. The input and output units of the input / output unit 12 may be configured separately.

[0030] The extraction unit 13 breaks down the input linguistic information into words and extracts local terms. Local terms are uncommon technical terms, such as those used in manufacturing sites. For example, the extraction unit 13 extracts local terms from the linguistic information input via the input / output unit 12. In this case, the extraction unit 13 references the server 30 to read a pre-set database of local terms, and extracts local terms and sentences using the local terms. The breakdown of the linguistic information into words can be achieved by morphological analysis or a method using large language models (LLM).

[0031] The prompt generation unit 14 generates a prompt for generating a question for acquiring tacit knowledge from a respondent. Specifically, the prompt generation unit 14 generates the prompt by searching for the extracted local terms with reference to the server 30 and supplementing the meaning of the terms. In this embodiment, for example, information is linked from a word list in which terms and explanations are paired. The prompt generated by the prompt generation unit 14 is transmitted to the sentence generation device 20, and the sentence generation device 20 generates a question (sentence) for acquiring tacit knowledge based on the prompt.

[0032] The question examination unit 15 functions as a determination unit that determines whether the content of a question (sentence) acquired from the sentence generation device 20 overlaps with the content of the question and whether the content is related to the set theme. If it is determined that the content of the generated question does not overlap with that of a previously adopted question and is related to the theme, the prompt is confirmed as a question.

[0033] The knowledge information generation unit 16 generates knowledge information as a record of interactions with respondents. The knowledge information generation unit 16 generates the knowledge information in a set format. Examples of the format include a knowledge graph, text data, and a JSON file intended for use in other applications.

[0034] The extraction unit 13, the prompt generation unit 14, the question examination unit 15, and the knowledge information generation unit 16 are configured using processors such as a CPU (Central Processing Unit), a DSP (Digital Signal Processor), and an FPGA (Field-Programmable Gate Array).

[0035] Specifically, the control unit 17 includes a processor having hardware such as a CPU, DSP, or FPGA, and a main storage unit such as a RAM (Random Access Memory) or a ROM (Read Only Memory) (neither of which are shown). In the following description, the knowledge extraction device 10 will be described as having a function of performing speech recognition on speech data input via the input / output unit 12 or the like and converting it into characters.

[0036] The storage unit 18 is configured with a storage medium selected from volatile memory such as RAM, non-volatile memory such as ROM, erasable programmable ROM (EPROM), a hard disk drive (HDD), and removable media. The removable media may be, for example, a USB memory or a disk recording medium such as a CD, DVD, or BD. The storage unit 18 may also be configured with a computer-readable recording medium such as an externally attachable memory card.

[0037] The storage unit 18 can store an operating system (OS), various programs such as knowledge extraction applications, various tables, various databases, and the like, for executing the operations of the knowledge extraction device 10. These various programs can also be recorded on computer-readable recording media such as hard disks, flash memories, CD-ROMs, DVD-ROMs, and flexible disks, and distributed widely.

[0038] 3 is a block diagram showing the configuration of a text generation device provided in a knowledge processing system according to an embodiment of the present invention. The text generation device 20 includes a communication unit 21, an input / output unit 22, a text generation unit 23, a vectorization unit 24, a control unit 25, and a storage unit 26.

[0039] The communication unit 21 is, for example, a LAN interface board, a wired communication circuit for wired communication, or a wireless communication circuit for wireless communication. The LAN interface board, the wired communication circuit, or the wireless communication circuit is connected to a network. The communication unit 21, which functions as a transmitter and a receiver, is connected to the network and communicates with the knowledge extraction device 10 and the server 30.

[0040] The input / output unit 22 may be configured, for example, with a display, a touch panel display, a speaker / microphone, etc. The input / output unit 22 as input means may include an interface that inputs various information transmitted from an external server via a communication unit and outputs the information to the control unit 25. The input / output unit 22 also includes a user interface, such as a keyboard, input buttons, a lever, a touch panel for manual input superimposed on a display such as an LCD, or a microphone for voice recognition. An operator or the like can input predetermined information to the control unit 25 by operating the input / output unit 22. The input / output unit 22 as output means displays predetermined images on a display monitor, displays characters or figures on the screen of a touch panel display, or outputs sound from a speaker, under the control of the control unit 25. That is, the input / output unit 22 is configured to notify predetermined information to the outside. The input and output units of the input / output unit 22 may be configured separately.

[0041] The sentence generation unit 23 generates sentences to be used as questions using prompts acquired from the knowledge extraction device 10. The sentence generation unit 23 generates sentences using a trained model. The trained model in this embodiment is a large-scale language model (LLM) generated by machine learning using a multilayer neural network including an input layer, an intermediate layer, and an output layer, with local terms as features. A known method can be used for the machine learning.

[0042] The vectorization unit 24 vectorizes the sentence generated by the sentence generation unit 23. Through this vectorization, the sentence is quantified into a character string, coordinates in a multidimensional space, etc. For vectorization, known methods such as Bag of Words and distributed representation can be adopted.

[0043] The sentence generating unit 23 and the vectorizing unit 24 are configured using a processor such as a CPU, a DSP, or an FPGA.

[0044] Specifically, the control unit 25 includes a processor having hardware such as a CPU, a DSP, an FPGA, and a main storage unit such as a RAM or a ROM (neither of which is shown).

[0045] The storage unit 26 is configured with a storage medium selected from volatile memory such as RAM, non-volatile memory such as ROM, EPROM, HDD, and removable media. The removable media may be, for example, a USB memory or a disc storage medium such as a CD, DVD, or BD. The storage unit 26 may also be configured with a computer-readable storage medium such as an externally attachable memory card.

[0046] The storage unit 26 can store various programs such as an OS and a sentence generation application, various tables, various databases, etc., for executing the operations of the sentence generation device 20. These various programs can also be recorded on computer-readable recording media such as a hard disk, flash memory, CD-ROM, DVD-ROM, or flexible disk, and distributed widely.

[0047] The server 30 is realized using a storage medium selected from volatile memory such as RAM, non-volatile memory such as ROM, EPROM, HDD, removable media, etc., and a computer equipped with a general-purpose processor such as a CPU. The server 30 includes a storage unit in which local terms and sentences using local terms (local documents) are stored in advance.

[0048] (Knowledge Extraction Process) Figure 4 is a sequence diagram illustrating the flow of knowledge extraction processing according to one embodiment of the present invention. When the knowledge extraction process is started in the knowledge extraction device 10, the control unit 17 determines whether or not a setting input of theme information has been received (step S101: theme information receiving step). At this time, the theme information is input, for example, by voice or character input on a keyboard via the input / output unit 12 functioning as a theme information receiving unit. If no setting input of theme information has been received (step S101: No), the control unit 17 repeats the input confirmation. If the control unit 17 determines that theme information has been received (step S101: Yes), the control unit 17 proceeds to step S102.

[0049] In step S102, the extraction unit 13 (first extraction unit) extracts local terms (first local terms) from the input theme information ((first) extraction step). At this time, the extraction unit 13 extracts local information (first local information) from the theme information, for example, by referring to a database (e.g., the server 30) that pre-stores local terms and sentences containing the local terms. Alternatively, the extraction unit 13 may extract local information (first local information) from the theme information by referring to a database (e.g., the server 30) that pre-stores information obtained by vectorizing local terms and sentences containing the local terms, and comparing the similarity between the information and information obtained by vectorizing the theme information.

[0050] Then, the prompt generation unit 14, which functions as a supplementary information generation unit (first supplementary information generation unit), searches for local information for the extracted local term by referring to the server 30, and generates supplementary information (first supplementary information) that supplements the meaning of the local term based on the search results (step S103).

[0051] Thereafter, the prompt generation unit 14 generates a prompt by supplementing the meaning of the local term based on the search result (step S104). The prompt generation unit 14 transmits the generated prompt to the sentence generation device 20 via the communication unit 11. This prompt includes, for example, theme information, local information (first local information), and supplemental information (first supplemental information) that supplements the meaning of the local term. Here, steps S103 and S104 correspond to a supplemental information generation step.

[0052] When the sentence generation device 20 receives a prompt from the knowledge extraction device 10 via the communication unit 21, the sentence generation device 20 uses the received prompt to generate a sentence as a question for acquiring tacit knowledge (step S105). In this embodiment, the sentence generation unit 23 generates a sentence using an LLM. The knowledge extraction device 10 receives the sentence generated by the sentence generation device 20 via the communication unit 11, which functions as a sentence acquisition unit (step S106: sentence acquisition step).

[0053] Furthermore, in the sentence generation device 20, the vectorization unit 24 vectorizes the sentence generated in step S105 (step S107). The sentence generation device 20 transmits the generated sentence and its vector information to the knowledge extraction device 10 via the communication unit 21. At this time, the sentence generation device 20 stores the generated sentence and its vector information in the storage unit 26 or the server 30. The knowledge extraction device 10 acquires the vector information generated by the sentence generation device 20 via the communication unit 21 (step S108).

[0054] In the knowledge extraction device 10, when the sentence and vector are acquired from the sentence generation device 20, the question examination unit 15 determines whether the acquired question has overlapping content (step S109). The question examination unit 15 calculates the similarity between the question used in the current exchange and the current sentence from the vector information and compares the calculated similarity with a preset threshold. If the similarity is higher than the threshold, the question examination unit 15 determines that the question has overlapping content (step S109: Yes) and proceeds to step S104 to generate a sentence with a different content. If the similarity is equal to or lower than the threshold, the question examination unit 15 determines that the question has not overlapping content (step S109: No) and proceeds to step S110. Note that in step S109, an example of determining whether or not there is overlap is described, targeting the question used in the current exchange. However, similarity with questions used in previous exchanges prior to the current exchange may also be determined.

[0055] In step S110, the question inspection unit 15 determines whether the sentence is related to the current theme. The question inspection unit 15 calculates the distance between the theme and the sentence (local term) from the vector and compares the calculated distance with a preset threshold. If the distance is greater than the threshold, the question inspection unit 15 determines that the sentence is not highly related to the theme (step S110: No) and proceeds to step S104. If the distance is equal to or less than the threshold, the question inspection unit 15 determines that the sentence is highly related to the theme (step S110: Yes) and proceeds to step S111. The question inspection unit 15 may execute step S111 before step S109, or may execute steps S109 and S110 simultaneously. When proceeding to step S104 to generate a sentence again, a supplementary prompt is set, which includes a sentence determined to be redundant or unrelated to the theme as an inappropriate example and includes an instruction to improve the output by creating a sentence different from the sentence. Furthermore, a threshold value may be set for the number of repetitions of resetting a question, and if the number of repetitions exceeds the threshold value, the exchange may be terminated.

[0056] In step S111, if the question examination unit 15 determines in steps S109 and S110 that the content is not overlapping and is related to the theme, it determines to set the sentence as a question. The series of processes in steps S109 to S111 corresponds to the determination step.

[0057] In step S112, the control unit 17 causes the input / output unit 12, which functions as a question output unit, to output the sentence generated in S105 as a question (question output step). As a result, the question is output as a display or voice.

[0058] Thereafter, the control unit 17 determines whether an answer (answer information) to the output question has been input from the input / output unit 12, which functions as an answer receiving unit (step S113: answer receiving step). If no answer has been input (step S113: No), the control unit 17 repeats the input confirmation. Furthermore, if the control unit 17 confirms that an answer has been input via the input / output unit 12 (step S113: Yes), the control unit 17 proceeds to step S114. Note that the control unit 17 may proceed to step S114 if no answer has been received within a predetermined time since the question was output.

[0059] In step S114, the extraction unit 13 (second extraction unit) extracts local terms (second local terms) from the input linguistic information, which is the answer (answer information), as related terms related to the theme information (second extraction step).

[0060] Then, the prompt generator 14, functioning as a supplemental information generator (second supplemental information generator), generates supplemental information (second supplemental information) for the answer (step S115). At this time, the prompt generator 14 refers to the server 30 to search for local information (second local information) for the local term extracted in step S114, and generates supplemental information that supplements the meaning of the local term in the answer based on the search result.

[0061] After generating the supplemental information, the control unit 17 determines whether end information has been input (step S116). The control unit 17 determines, for example, whether a preset input, such as a voice containing a keyword related to end, such as "I'm done," has been input. If the control unit 17 determines that end information has not been input (step S116: No), the control unit 17 proceeds to step S104 and generates a new question. On the other hand, if the control unit 17 determines that end information has been input (step S116: Yes), the control unit 17 proceeds to step S117. When proceeding to step S104 to continue generating questions, the prompt generation unit 14 adds the answer from the respondent (answer information), the local information (second local information) generated in step S115, and the supplemental information (second supplemental information) to the prompt, thereby updating the prompt (update step).

[0062] In step S117, the knowledge information generator 16 generates knowledge information based on information about the series of processes, including the question, the answer (answer information), the local information (second local information), and the supplemental information (second supplemental information) in the series of processes (knowledge information generation step). The knowledge information generator 16 generates the knowledge information in an output format specified by the user through voice input or the like, or in a preset output format.

[0063] Here, the output format should be such that the knowledge information obtained through the dialogue between the knowledge system 1 and the respondent is organized into a format that can be easily used by other users and systems. Specific output formats of the knowledge information include a text format and a knowledge graph format.

[0064] The text format has the advantage of being able to record specific information in detail, and examples of such formats include the FAQ (Frequently Asked Questions) format, which summarizes questions and answers about a topic, and the summary information (document summary) format, which uses natural language processing (NLP) to summarize knowledge gained through question and answer interactions and concisely summarize important points and concepts about the topic. The FAQ format has the advantage that it can be saved as structured data, making it easy to integrate with other systems and applications, and information can be obtained through APIs, and organized information is easy for other users to understand. The summary information format has the advantage that other users can grasp important information in a short amount of time, enabling efficient reference, and summarized information can be quickly referenced by other systems, improving the efficiency of information search.

[0065] The knowledge graph format is a format that visually shows the relevance and structure of information using nodes (entities) and edges (relationships) based on important entities extracted from questions about topic information and their answers, and the relationships between the entities. This has the advantage of clarifying the relationships between data and improving search performance.

[0066] Tacit knowledge information can be stored in a database in both knowledge graph and text formats. Storing tacit knowledge information in both knowledge graph and text formats in a database enables technological advances in the system. The knowledge graph format significantly improves the efficiency of information search in other systems by making the relationships between entities explicit. This enables fast and accurate search results even for complex queries and enables advanced analysis that leverages data interrelationships. Storing information in text format also provides the flexibility to search for information in natural language using natural language processing technology. Combining these two formats maintains data integrity and consistency, improving overall system performance. As a result, the use of tacit knowledge information is promoted, facilitating integration with other systems, and contributing to the evolution of overall data search technology.

[0067] Furthermore, the knowledge graph format visually shows the relationships between entities, helping users intuitively understand complex information, while the text format leverages natural language processing technology, allowing users to input questions in natural language, improving search flexibility. The combination of these two formats allows users to gain deeper insights, significantly improving the overall efficiency of data search and the user experience.

[0068] In this embodiment, the input / output unit 12 is described as having the functions of a theme information receiving unit, a question output unit, and an answer receiving unit, and the extraction unit 13 is described as having the functions of the first and second extraction units, but each function may also be performed as a separate block.

[0069] (Processing Example) Next, a processing example of this embodiment will be described with reference to Figs. 5 and 6. Figs. 5 and 6 are diagrams for explaining an example of an exchange with a respondent during knowledge extraction according to one embodiment of the present invention. Fig. 5 shows an example of an exchange at the start of processing. Fig. 6 shows an example of an exchange at the end of processing. Note that Figs. 5 and 6 show the content of the exchange between the knowledge extraction device 10 and the respondent. In the figures, speech bubbles with speech outlets on the left side show the output content of the knowledge extraction device 10, and speech bubbles with speech outlets on the right side show the answer content from the respondent.

[0070] When the knowledge extraction process begins, the knowledge extraction device 10 executes the process. Below, we will explain an example in which the following theme and prompt are set and the process is executed. Note that when setting the prompt, local terminology is supplemented and a supplementary prompt is generated (see steps S103 and S104). (Theme) About BO of CC (Local terminology) CC: Abbreviation for continuous casting machine BO: Abbreviation for breakout (Here, the problem is a "molten steel leak") (Basic prompt) Knowledge is extracted from humans by conducting interviews based on conversations between humans and large-scale language models. Therefore, the large-scale language model should generate the necessary questions. Continue asking questions that delve deeper into the reasons for the answers given to the questions. Only one question can be asked at a time. (Theme setting prompt: Theme information) The theme is "About BO of CC." (Theme setting content supplementary prompt: Local information + supplementary information) Here, CC stands for continuous casting machine, and BO stands for breakout. Breakout is a typical operational problem that occurs in continuous casting machines, and is a problem that causes molten steel to leak.

[0071] First, the knowledge extraction device 10 outputs the following question (see FIG. 5): "Let me start the question. What causes BO?" In response, the respondent gives the following answer: "It is said that restrictive BO is caused by burn-in."

[0072] In response to the above answer, the knowledge extraction device 10 outputs the following question: "I understand. So, what causes image sticking?" In response to this, the respondent gives the following answer: "It's related to powder and Vc."

[0073] In response to the above answer, the knowledge extraction device 10 outputs the following question: "So the powder and the casting speed are involved. How is the powder involved?" In response to this, the respondent gives the following answer: "I don't know." At this time, the knowledge extraction device 10 adds that "Vc" is the "casting speed" (see step S114).

[0074] In response to the answer, the knowledge extraction device 10 outputs the following question: "I understand. I'll ask another question. What are the main measures to prevent BO?" At this time, the knowledge extraction device 10 internally generates a candidate question, for example, "Why might powder be involved?" to reconfirm the relationship with powder in response to the answer. However, after comparing the vectors, it is determined that the similarity is high and is replaced with another question (What are the main measures to prevent BO?).

[0075] In this way, the knowledge extraction device 10 continues to output questions based on the presence or absence of overlap and thematic relevance.

[0076] When the respondent enters the answer "I'm done," the knowledge extraction device 10 outputs the following question (see FIG. 6): "I understand. What output format would you like?" In response to this, the respondent gives the following answer: "Please output as a knowledge graph."

[0077] Based on the above answers, the knowledge extraction device 10 outputs the current interaction as a knowledge graph and stores it in the storage unit 18. The results of the interaction may be displayed, or may be stored in the storage unit 18 without being displayed. If the output format is determined in advance, the question about the output format may be omitted, or the output format may be clearly stated before proceeding with the process.

[0078] According to the embodiment described above, questions are generated using a large-scale language model (LLM), and questions are vectorized to evaluate question duplication and thematic relevance, and the questions are scrutinized. The output of duplicate questions and questions that deviate from the theme is suppressed. This suppresses duplicate answers related to the extraction of tacit knowledge and answers that deviate from the theme, making it possible to extract tacit knowledge efficiently and reliably.

[0079] In the conventional interview format, the schedules of the questioner and respondent need to be coordinated, which takes time to compile tacit knowledge. Furthermore, the content of the interview must be compiled afterwards, which creates the problem of time-consuming information processing per person. In contrast, according to this embodiment, extraction work can be performed solely at the respondent's convenience, and compilation is also performed on the device side, making it possible to efficiently extract and store tacit knowledge.

[0080] (Modification 1) Next, Modification 1 of this embodiment will be described with reference to Fig. 7. The configuration of the knowledge processing system according to Modification 1 is the same as that of the above-described knowledge processing system 1, and therefore a description thereof will be omitted. Fig. 7 is a sequence diagram for explaining the flow of knowledge extraction processing according to Modification 1 of the present invention.

[0081] When the knowledge extraction process is started in the knowledge extraction device 10, similar to steps S101 to S103, the device accepts input of theme information settings, extracts local terms (first local terms) from the input theme information, and generates supplementary information (first supplementary information) for the extracted local terms that supplements the meaning of the local terms (steps S201 to S203).

[0082] Thereafter, in the present first modification, the prompt generating unit 14 sets topics for questions related to the theme information as a pre-processing process for generating guidelines for questions regarding the theme information (steps S204 to S207).

[0083] A question topic refers to a specific aspect or important element related to the topic information that can be used to deepen information and discussion. For example, if the topic information is a breakout in continuous casting, examples of question topics include the causes, detection methods, prevention measures, impacts, and post-processing methods of the breakout. By identifying one or more topics of questions related to the topic information as guidelines and repeatedly generating questions based on the identified one or more topics, deviation from the topic information can be prevented, unnecessary questions can be reduced, and a wide range of tacit knowledge related to the topic information can be efficiently acquired from various perspectives. Furthermore, systematically organizing the collected tacit knowledge related to the topic information facilitates future reference and use, promoting the accumulation of knowledge. In this way, question guidelines for the topic information are generated before generating prompts for the sentence generation device 20 to generate sentences that serve as questions for acquiring tacit knowledge related to the topic information.

[0084] In step S204, the prompt generation unit 14 generates a prompt to prompt the user to generate one or more topics for questions related to the theme information as a guideline for questions about the theme information. A specific example of a prompt to prompt the user to generate topics for questions is, "Please generate one or more topics for questions related to the theme information. The topics to be generated must reflect important aspects related to the theme information. Specifically, please consider aspects such as causes, detection methods, prevention measures, impacts, and post-processing methods. However, this does not need to be limited to these. Please limit the number of topics to about five." The prompt generation unit 14 transmits the generated prompt to the sentence generation device 20 via the input / output unit 12.

[0085] When the sentence generation device 20 receives a prompt for generating a question topic from the knowledge extraction device 10 via the input / output unit 22, the sentence generation device 20 generates a sentence to be used as a question topic as a guideline for a question regarding the theme information (step S205). In this embodiment, the sentence generation unit 23 generates a sentence to be used as a question topic using the LLM.

[0086] Then, the knowledge extraction device 10 acquires the sentences (one or more topics of the question) generated by the sentence generation device 20 as guidelines for questions regarding the theme information (step S206).

[0087] After acquiring the question guidelines, the prompt generation unit 14 sets the topic of the question (step S207: topic setting step). Then, the prompt generation unit 14 generates a prompt by adding information related to the updated question topic to the theme information, local information, and supplementary information that supplements the meaning of the local term (step S208). The prompt generation unit 14 transmits the generated prompt to the document generation device 20. Thereafter, similar to steps S105 to S116, the prompt generation unit 14 generates sentences, acquires vector information, determines the question, extracts local terms, and generates supplementary information, and determines whether or not termination information has been input (steps S209 to S220).

[0088] If the control unit 17 determines that no end information has been input (step S220: No), it proceeds to step S207 and sets the topic of the next question. Then, it performs steps S208 to S219 again for the set topic. On the other hand, if the control unit 17 determines that end information has been input (step S220: Yes), it proceeds to step S221.

[0089] In step S221, the knowledge information generator 16 generates knowledge information based on information about a series of processes, including questions, answers (answer information), local information (second local information), and supplemental information (second supplemental information) in the series of processes (knowledge information generation step). The knowledge information generator 16 generates the knowledge information in an output format specified by the user through voice input or the like, or in a preset output format.

[0090] According to the first variant example described above, questions are generated using a large-scale language model (LLM), and questions are scrutinized by vectorizing them to evaluate question duplication and thematic relevance, and the output of duplicate questions and questions that deviate from the theme is suppressed. This suppresses duplicate answers related to the extraction of tacit knowledge and answers that deviate from the theme, making it possible to extract tacit knowledge efficiently and reliably.

[0091] Furthermore, according to Modification 1, one or more topics of questions related to the theme information are set in advance as guidelines for generating questions, which prevents deviation from the theme information and reduces unnecessary questions, making it possible to efficiently acquire a wide range of tacit knowledge related to the theme information from various perspectives. Furthermore, according to Modification 1, the tacit knowledge related to the collected theme information is systematically organized, making it easy to reference and use after storing it in a database, and allowing for even more efficient acquisition of tacit knowledge.

[0092] (Variation 2) Next, Variation 2 of this embodiment will be described with reference to Fig. 8. Fig. 8 is a sequence diagram for explaining the flow of knowledge extraction processing according to Variation 2 of the present invention. Fig. 8 is a configuration diagram for explaining Variation 2 in which the sequences shown in Figs. 4 and 7 are executed using a plurality of agents. In the following explanation, the processing flow will be described using, for example, the processing flow shown in Fig. 7 as an example.

[0093] An agent is a program that automatically executes specific tasks, and cooperates with the generation AI via an API to utilize a large-scale language model to collect information and generate questions. The agent shown in FIG. 8 is composed of a question construction agent 41, a local term reference agent 42, a question adjustment agent 43, a question duplication confirmation agent 44, a topic relevance confirmation agent 45, and a knowledge organization agent 46, and realizes the functions of each unit included in the knowledge extraction device 10. Each agent communicates with each other via the API and exchanges information. The agent also cooperates with an input / output unit including a user interface, a server 51, a database 50, etc. via the API.

[0094] The question construction agent 41 interacts with the user interface (input / output unit) via the API and determines whether or not the theme information setting input has been received (see step S201). The question construction agent 41 identifies the topic of a question related to the received theme information (see steps S204 and S206). The question construction agent 41 shares the theme information and information on the identified question topic with other agents via the API. The question construction agent 41 performs part of the functions of the input / output unit 12 and the prompt generation unit 14, which function as a theme information receiving unit.

[0095] The local term reference agent 42 references a server or database via an API, and cooperates with the generation AI to utilize a large-scale language model to extract local terms (first local terms or second local terms) from the theme information shared by the question construction agent 41 (step S202 or step S218), and generates supplementary information ((first) supplementary information or second supplementary information) for the extracted local information. The local term reference agent 42 performs part of the functions of the extraction unit 13 and the prompt generation unit 14.

[0096] The question adjustment agent 43 generates prompts including thematic information, local information, and supplemental information, and acquires candidate sentences for the generated questions using a large-scale language model generated by learning the local information as features via an API (see steps S208 and S210). The question adjustment agent 43 executes some of the functions of the prompt generation unit 14 and is responsible for some of the functions of the prompt generation unit 14.

[0097] The question adjustment agent 43 also supervises the other agents via the API and assigns tasks to them. The question adjustment agent 43 instructs the question overlap confirmation agent 44 and the theme relevance confirmation agent 45 via the API to perform the evaluations in steps S213 and S214, and determines whether the question content overlaps and whether the sentence is related to the theme based on the evaluations from each agent, and determines whether to reacquire a sentence that is a candidate for a question (see steps S208 and S210) or to confirm the sentence that is a candidate for a question as a question (see step S215). The question adjustment agent 43 performs part of the functions of the question scrutiny unit 15.

[0098] The question adjustment agent 43 determines whether to repeatedly generate questions that dig deeper into the answer information of the respondent about the set question topic, or to move on to the next question topic (see step S220), based on the question topic generated by the question construction agent 41. The question adjustment agent 43 performs some of the functions of the control unit 17.

[0099] Based on instructions from the question adjustment agent 43, the question duplication confirmation agent 44 evaluates whether the content of the sentence that is a candidate question acquired by the question adjustment agent 43 overlaps with previous questions (step S213), and returns an evaluation to the question adjustment agent 43. The question duplication confirmation agent 44 performs part of the function of the question scrutiny unit 15. Based on instructions from the question adjustment agent 43, the theme relevance confirmation agent 45 evaluates the relevance of the sentence that is a candidate question acquired by the question adjustment agent 43 with theme-related information (step S214), and returns an evaluation to the question adjustment agent 43. The theme relevance confirmation agent 45 performs part of the function of the question scrutiny unit 15.

[0100] The knowledge organization agent 46 generates knowledge information based on the questions, answers (answer information), local information, and supplementary information generated by a series of agent processes, and stores the resulting knowledge information in a database in a knowledge graph format or a text format via an API (see step S221). The knowledge organization agent 46 executes the functions of the knowledge information generator 16.

[0101] (Question Answering System Construction Method) A question answering system may be constructed that utilizes the knowledge information generated by the above-described knowledge extraction process. Specifically, a RAG (Retrieval Augmented Generation) environment in which the generated knowledge information is provided to a large-scale language model can be constructed as the question answering system. In the RAG environment, the large-scale language model acquires knowledge information from an external tacit knowledge base to generate responses, thereby enabling responses to questions based on the respondent's experience, insight, and specialized knowledge.

[0102] An example of the processing of this embodiment will be described with reference to FIGS. 9 and 10, in which the system is applied to a question answering system for troubleshooting manufacturing equipment in a steelworks.

[0103] 9A and 9B are diagrams showing (a) summary information in text format and (b) knowledge graph format of undocumented tacit knowledge information extracted from skilled technicians in charge of equipment maintenance at a steelworks by the knowledge extraction process of the present invention. As shown in Fig. 9, undocumented tacit knowledge information was extracted from skilled technicians in charge of equipment maintenance at a steelworks by the knowledge extraction process of the present invention, and stored in a database in the form of summary information in text format and knowledge graph format.

[0104] 10A and 10B are diagrams showing the results of responses from a question-answering system regarding causes and countermeasures when damage occurs to the main machine (main drive or spindle) of a hot rolling line at a steelworks. FIG. 10A is a diagram showing an example of a response from a large-scale language model to which knowledge information generated by the knowledge extraction process of the present invention is applied in a RAG environment. FIG. 10B is a diagram showing an example of a response from a large-scale language model to which knowledge information is not applied in a RAG environment. Comparing FIGS. 10A and 10B , the response shown in FIG. 10B illustrates generally assumed causes and countermeasures, while the response shown in FIG. 10A presents specific causes and countermeasures, as well as the implementation status of the countermeasures, based on the deeper insight and specialized knowledge of a skilled equipment maintenance technician.

[0105] In this way, by building a question-answering system that utilizes knowledge information generated by knowledge extraction processing, users can quickly acquire specialized knowledge by obtaining information based on the experience and knowledge of skilled technicians. Because tacit knowledge is based on practical experience, past mistakes and errors can be avoided, enabling safer and more efficient work. Users can then respond in a way that is appropriate to the actual situation, improving the speed of troubleshooting. Furthermore, by incorporating the tacit knowledge of skilled technicians into a question-answering system, knowledge can be passed on, allowing the next generation of users to inherit and utilize this valuable knowledge.

[0106] (Recording Medium) In the above-described embodiment, a program for executing the processing method performed by the learning device 2 or the state determination device 3 can be recorded on a recording medium readable by a computer or other machine, a wearable device, or other device (hereinafter referred to as a computer, etc.). By having a computer, etc., read and execute the program from the recording medium, the computer, etc. functions as a mobile object control device. Here, a computer-readable recording medium refers to a non-transitory recording medium that stores information such as data and programs through electrical, magnetic, optical, mechanical, or chemical action and can be read by a computer, etc. Examples of such recording media that are removable from a computer, etc. include flexible disks, magneto-optical disks, CD-ROMs, CD-R / Ws, DVDs, BDs, DATs, magnetic tapes, and memory cards such as flash memory. Furthermore, examples of recording media that are fixed to a computer, etc. include hard disks and ROMs. Furthermore, SSDs can be used as both recording media that are removable from a computer, etc. and recording media that are fixed to a computer, etc.

[0107] Although one embodiment of the present invention has been specifically described above, the present invention is not limited to the above embodiment, and various modifications based on the technical concept of the present invention are possible. For example, the model given in the above embodiment is merely an example, and a different model may be used as needed. The present invention is not limited by the descriptions and drawings that form part of the disclosure of the present invention according to this embodiment. For example, in the above embodiment, an example of generating sentences using a trained model in which features are trained in a large-scale language model has been described, but a configuration using a pre-created (untrained) large-scale language model or a model (framework) based on a large-scale language model such as a model using RAG (Retrieval-Augmented Generation) may also be used.

[0108] In the above-described embodiment, deep learning using a neural network is used as an example of machine learning, but machine learning based on other methods may also be performed. For example, other supervised learning methods such as support vector machines, decision trees, naive Bayes, and k-nearest neighbor methods may also be used. Furthermore, semi-supervised learning may also be used instead of supervised learning.

[0109] In addition, in the embodiment, an example in which the extraction unit 13 separates and extracts each word has been described. However, other methods may be used, such as separating and extracting each word using morphological analysis, or extracting only nouns. In the present embodiment, information is linked from a word list in which terms and descriptions are paired. However, information may also be supplemented by performing an RAG search of local documents using a large-scale language model. Finally, a prompt to be provided to the large-scale language model in advance may be created by combining the basic prompt, the theme setting prompt, and the supplementary prompt for the theme setting content.

[0110] In addition, in one embodiment, an example has been described in which the knowledge extraction device 10 and the sentence generation device 20 read information from a common server 30 that serves as a memory unit, but they may each read information from a different server, or they may each read information from a memory unit provided in each device.

[0111] In addition, in one embodiment, the above-mentioned "unit" can be read as "circuit" etc. For example, a control unit can be read as a control circuit.

[0112] In one embodiment, at least some of the "units" included in each device may be implemented in different devices. In this case, for example, the question generation system and the knowledge extraction system are configured by combining different devices.

[0113] Further advantages and modifications will readily occur to those skilled in the art. The disclosure in its broader aspects is not limited to the specific details and representative embodiments shown and described above. Accordingly, various modifications may be made without departing from the spirit or scope of the general inventive concept as defined by the appended claims and their equivalents.

[0114] As described above, the question generation method, knowledge extraction method, question answering system construction method, question generation program, knowledge extraction program, question generation system, and knowledge extraction system according to the present invention are suitable for extracting tacit knowledge efficiently and reliably.

[0115] REFERENCE SIGNS LIST 1 knowledge processing system 10 knowledge extraction device 11, 21 communication unit 12, 22 input / output unit 13 extraction unit 14 prompt generation unit 15 question scrutiny unit 16 knowledge information generation unit 17, 25 control unit 18, 26 storage unit 20 text generation device 23 text generation unit 24 vectorization unit

Claims

1. A question generation method comprising: a theme information receiving step for receiving input of theme information for generating a question; an extraction step for extracting local information from the received theme information; a supplemental information generating step for generating supplemental information for the extracted local information; a sentence acquisition step for acquiring a sentence generated using a large-scale language model generated by learning local information as a feature based on the theme information, the local information, and a prompt including the supplemental information; and a question review step for determining whether or not to set the sentence as a question related to the theme information based on the degree of content redundancy and / or thematic relevance of the sentence.

2. The question generation method described in claim 1, wherein, if it is determined in the question examination step that the sentence acquired in the sentence acquisition step should not be set as a question related to the theme information, the sentence acquired in the sentence acquisition step is set as a supplementary prompt that includes the sentence as an inappropriate example and includes instructions to encourage improvement of output, and the sentence acquisition step is executed again based on the prompt and the supplementary prompt.

3. The question generation method according to claim 1 or 2, further comprising a topic setting step of setting a topic of a question related to the theme information as a guideline for a question regarding the theme information, and in the sentence acquisition step, adding information related to the set topic of the question to the prompt.

4. The question generation method according to claim 3, wherein the topic setting step includes theme information, and sets the topic of the question using a large-scale language model generated by learning local information as features based on a prompt that prompts the user to generate a topic of the question.

5. The question generation method according to claim 1, wherein the question examination step compares the similarity between information obtained by vectorizing the sentence and information obtained by vectorizing other sentences generated before the one being evaluated, to determine whether the contents overlap.

6. The question generation method of claim 1, wherein the question examination step determines whether the sentence is related to the set theme based on the relevance between information obtained by vectorizing the sentence and information obtained by vectorizing the theme information.

7. The question generation method according to claim 1, wherein the extraction step extracts the local information from the theme information by referring to a database that stores local terms and sentences containing the local terms in advance.

8. The question generation method according to claim 1, wherein the extraction step refers to a database that stores information obtained by vectorizing local terms and sentences containing the local terms in advance, and extracts the local information by comparing the similarity between the information and information obtained by vectorizing the theme information.

9. A theme information receiving step for receiving input of theme information for generating a question; a first extraction step for extracting first local information from the received theme information; a first supplemental information generating step for generating first supplemental information for the extracted first local information; a sentence acquisition step for acquiring a sentence generated using a large-scale language model generated by learning local information as features based on the theme information, the first local information, and a prompt including the first supplemental information; a question review step for determining whether or not to set the sentence as a question related to the theme information based on content overlap and / or theme relevance; a question output step for outputting a question determined to be set as the question by the question review step; an answer receiving step for receiving answer information from a respondent; a second extraction step for extracting second local information from the answer information; and a second supplemental information generating step for generating second supplemental information for the extracted second local information. a knowledge information generating step of converting information of a series of processes including the question, the answer information, the second local information, and the second supplementary information into a set format to generate knowledge information.

10. The knowledge extraction method according to claim 9, further comprising an update step of updating the prompt by adding the answer information, the second local information, and the second supplementary information to the prompt, and repeatedly executing the sentence acquisition step, the question examination step, the quality output step, the answer acceptance step, the second extraction step, and the second supplementary information generation step based on the updated prompt.

11. The knowledge extraction method according to claim 9 or 10, wherein in the knowledge information generation step, the knowledge information is generated by converting the knowledge information into a knowledge graph format and a text format.

12. A method for constructing a question-answering system, comprising constructing a question-answering system that utilizes knowledge information generated using the knowledge extraction method according to claim 9.

13. A question generation program that causes a computer to execute the following steps: a theme information receiving step that receives input of theme information for generating questions; an extraction step that extracts local information from the received theme information by referring to a memory unit; a supplemental information generating step that generates supplemental information for the extracted local information; a sentence acquisition step that acquires sentences generated using a large-scale language model that is generated by learning local information as features based on the theme information, the local information, and a prompt including the supplemental information; and a question review step that determines whether or not to set the sentences as questions related to the theme information based on the degree of content redundancy and / or thematic relevance of the sentences.

14. A theme information receiving step for receiving input of theme information for generating a question; a first extraction step for extracting first local information from the received theme information by referring to a storage unit; a first supplemental information generating step for generating first supplemental information for the extracted first local information; a sentence acquisition step for acquiring a sentence generated using a large-scale language model generated by learning local information as features based on the theme information, the first local information, and a prompt including the first supplemental information; a question review step for determining whether or not to set the sentence as a question related to the theme information based on content overlap and / or theme relevance; a question output step for outputting the question set by the question review step; an answer receiving step for receiving answer information from a respondent; a second extraction step for extracting second local information from the answer information; and a second supplemental information generating step for generating second supplemental information for the extracted second local information. a knowledge information generation step of converting information of a series of processes including the question, the answer information, the second local information, and the second supplementary information into a set format to generate knowledge information.

15. A question generation system comprising: a theme information receiving unit that receives input of theme information for generating questions; an extraction unit that extracts local information from the received theme information by referring to a memory unit; a supplemental information generation unit that generates supplemental information for the extracted local information; a sentence acquisition unit that acquires sentences generated using a large-scale language model that is generated by learning local information as features based on the theme information, the local information, and a prompt including the supplemental information; and a question review unit that determines whether to set the sentences as questions related to the theme information based on the degree of content overlap and / or thematic relevance of the sentences.

16. A theme information receiving unit that receives input of theme information for generating a question; a first extraction unit that extracts first local information from the received theme information by referring to a storage unit; a first supplemental information generation unit that generates first supplemental information for the extracted first local information; a sentence acquisition unit that acquires a sentence generated using a large-scale language model that is generated by learning local information as features based on the theme information, the first local information, and a prompt including the first supplemental information; a question review unit that determines whether or not to set the sentence as a question related to the theme information based on content overlap and / or theme relevance; a question output unit that outputs a question determined by the question review unit to be set as the question; an answer receiving unit that receives answer information from a respondent; a second extraction unit that extracts second local information from the answer information; and a second supplemental information generation unit that generates second supplemental information for the extracted second local information. a knowledge information generation unit that converts information of a series of processes including the question, the answer information, the second local information, and the second supplementary information into a set format to generate knowledge information.

Images