Question answer generation system and question answer generation method

Abstract

This question-answer generation system generates divided documents with metadata by assigning the metadata representing document features extracted from a document to each of the divided documents obtained by dividing the document, and searches for a divided document with metadata having similarity to a question. The question-answer generation system adds, to the question, the divided document with metadata acquired as the search result, inputs same to a language model, and acquires an answer.

Description

Question and answer generation system and question and answer generation method

[0001] The present invention relates to a system and a method for generating answers to questions.

[0002] In recent years, question-answering technology using artificial intelligence (AI) that generates answers to questions entered by users in natural language has been attracting attention. A method for this question-answering technology, known as Retrieval-Augmented Generation (RAG), uses external information related to the questions entered into the AI ​​to improve the accuracy of the answers generated by the AI ​​(see, for example, Non-Patent Document 1).

[0003] RAG technology combines document segmentation and numerical vectorization to efficiently access large amounts of document data and retrieve appropriate context and information from it. Specifically, documents related to an input question are given, and the documents are segmented to search for appropriate information. Each segmented document is then converted into a numerical vector and converted into a format that is easy for the language model of the generation AI to understand. Similarly, a question input by a user is also subjected to numerical vectorization, and the similarity of each vector is compared to obtain segmented documents that are highly similar to the question. These segmented documents are then input to the generation AI along with the question content. This RAG technology allows the generation AI to generate more accurate answers and responses to questions and input contexts by referring to the segmented documents.

[0004] In this way, by using generative AI and RAG technology together, it is possible to obtain appropriate answers based on information that the generative AI has not learned, such as a company's own information.

[0005] Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yi Dai, Jiawei Sun, Qianyu Guo, Meng Wang and Haofen Wang.,“Retrieval-Augmented Generation for Large Language Models : A Survey”, [online], [searched on May 6, 2020], Internet <URL: https: / / arxiv.org / pdf / 2312.10997>

[0006] To achieve high accuracy in question-answering technology using generative AI and RAG technology, it is necessary to obtain documents that are appropriately segmented for the question entered by the user. However, if the unit of document segmentation in the RAG system is too small, there is a possibility that the context will be lost, while if the unit of segmentation is too large, there is a possibility that the efficiency of search and processing will decrease.

[0007] To solve this problem, there is a method of dividing a document into meaningful segments (chapters, sections, paragraphs, etc.) and storing document data in segments.

[0008] However, there are some documents, such as PDF documents, for which it is not easy to divide and save a paper document in segments. Furthermore, documents handled in corporate activities usually follow a specific format to ensure efficiency and consistency in internal communication and external information provision. Therefore, not all documents can necessarily be saved in segments.

[0009] In consideration of the above points, the present invention aims to provide a question-answering technology that uses generative AI and RAG technology, which enables more accurate answers to questions to be obtained regardless of the type of document that is the target of RAG.

[0010] In order to achieve the above-mentioned object, the present invention provides a question and answer generation system that generates answers to questions, the question and answer generation system having a processor and memory, wherein the processor divides a document into a plurality of divided documents of predetermined units, generates metadata-attached divided documents by assigning metadata extracted from the divided documents to each of the divided documents to represent the characteristics of the document, searches for the metadata-attached divided documents that are similar to the question, adds the metadata-attached divided documents obtained as a result of the search to the question and inputs them into a language model, and obtains the answer generated by the language model using the question with the metadata-attached divided documents attached as input.

[0011] According to the present invention, in a question answering technique using generative AI and RAG technology, it is possible to obtain a more accurate answer to a question regardless of the type of document that is the target of RAG.

[0012] 1 is a diagram showing the configuration of a question and answer generation system according to an embodiment. A diagram showing a document format according to an embodiment. A diagram showing a document according to an embodiment. A diagram showing document segmentation according to an embodiment. A diagram showing document segmentation and its problems according to an embodiment. A diagram showing document configuration information according to an embodiment. A schematic diagram showing metadata generation processing according to an embodiment. A diagram showing metadata-attached divided documents according to an embodiment. A diagram showing a document data numeric vector record table according to an embodiment. A diagram showing document retrieval based on cosine similarity between numeric vectors according to an embodiment. A diagram showing document retrieval based on cosine similarity between numeric vectors according to an embodiment. A diagram showing query content combining an acquired document and a question according to an embodiment. A diagram showing an answer output in response to a query to a language model according to an embodiment. A flowchart showing question and answer generation processing according to an embodiment. A flowchart showing metadata-attached divided document generation processing according to an embodiment. A diagram showing the hardware configuration of a computer.

[0013] In the following description, a "processor" may refer to one or more processor devices. The at least one processor device may typically be a microprocessor device such as a CPU (Central Processing Unit), but may also be another type of processor device such as a GPU (Graphics Processing Unit). The at least one processor device may be a single-core or multi-core. The at least one processor device may also be a processor core. The at least one processor device may also be a processor device in a broader sense, such as a hardware circuit that performs part or all of the processing (e.g., an FPGA (Field-Programmable Gate Array), a CPLD (Complex Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit)).

[0014] In the following description, information that provides an output in response to an input may be described using expressions such as "xxx table." This information may be data of any structure (for example, structured data or unstructured data). Therefore, "xxx table" may be referred to as "xxx information." In the following description, the structure of each table is an example, and one table may be divided into two or more tables, or all or part of two or more tables may be one table.

[0015] In the following description, processing may be described using a "program" as the subject. A program is executed by a processor to perform a predetermined process using a storage device and / or an interface device, etc., as appropriate. Therefore, the subject of processing may be a processor (or a device such as a controller having the processor). A program may be installed in a device such as a computer from a program source. The program source may be, for example, a program distribution server or a computer-readable (e.g., non-transitory) recording medium. In the following description, two or more programs may be realized as one program, or one program may be realized as two or more programs.

[0016] Hereinafter, each embodiment of the present invention will be described in detail with reference to the drawings. Note that each embodiment described below is merely an example, and the forms to which the present invention can be applied are not limited to the following embodiments. The technology related to each embodiment of the present invention can be used, for example, in a service that provides answers to questions entered by a user in natural language, but the target of use is not limited to this and can be used for various targets.

[0017] (Functional Configuration of Question and Answer Generation System 1000) First, the functional configuration of the question and answer generation system 1000 according to this embodiment will be described with reference to Fig. 1. Fig. 1 is a diagram showing the configuration of the question and answer generation system 1000 according to the embodiment.

[0018] The question and answer generation system 1000 aims to improve answer accuracy in question answering technology that uses a generative AI system and RAG technology. The question and answer generation system 1000 generates an answer 2400 based on an input question 2300, a document 2100 related to the input question 2300, and input document configuration information 2200.

[0019] Specifically, in this embodiment, the metadata setting unit 1130 assigns metadata (general information not present in the divided documents) to documents divided by the document dividing unit 1120. As a result, even if the document dividing unit 1120 has lost context, for example, document data with supplemented information is generated.

[0020] Furthermore, the document data with the information supplemented by the document data numeric vector conversion unit 1140 is converted into a numeric vector, which is then stored in a database by the document data numeric vector recording unit 1250. This allows the numeric vector similarity evaluation and search unit 1240 to generate the answer 2400 by referencing document data with a higher degree of similarity.

[0021] The generated metadata and document composition information 2200 are stored in the metadata recording unit 1134 and can be referenced the next time the document composition information 2200 is input.

[0022] In this embodiment, the document 2100 is a sentence consisting of, for example, several hundred to several thousand words, and is a sentence having a specific format or style. Examples of the document 2100 include specifications, design documents, instruction manuals, and internal company documents.

[0023] The document composition information 2200 is information related to the structure and characteristics of the document 2100. For example, it may include rules such as formatting and style such as chapters, sections, and paragraphs in the document 2100, and related information related to the document 2100, such as background. The document composition information 2200 may be a sentence written in a natural language (natural sentence), a sentence obtained as a result of speech recognition, or a collection of data resulting from input by a user via a certain user interface.

[0024] The question 2300 is a natural sentence. The sentence serving as the question 2300 may be, for example, a sentence simply listing keywords, or may be a sentence obtained as a result of speech recognition.

[0025] Each component of the question and answer generation system 1000 will be described below.

[0026] The question and answer generation system 1000 includes a document data generation unit 1100 and an answer generation unit 1200 .

[0027] The document data generation unit 1100 includes a document acquisition unit 1110 , a document division unit 1120 , a metadata setting unit 1130 , and a document data numeric vector conversion unit 1140 .

[0028] The document data generation unit 1100 generates document data that has been converted into numerical vectors and that has been provided with metadata, based on the document 2100 and document configuration information 2200 .

[0029] The document acquisition unit 1110 acquires an input document 2120. Hereinafter, for convenience of explanation, it is assumed that the input document 2120 is a document in the document format 2110 of FIG. 2A. The format of the document format 2110 may be any other format as long as it conforms to certain rules. Specifically, the document format 2110 is as shown in FIG. 2B.

[0030] The document divider 1120 divides the document 2100 acquired by the document acquisition unit 1110 into predetermined units. FIG. 3 shows an example of dividing the input document 2120 into two documents, document 1121 and document 1122. The predetermined unit of division (division point) may be specified by the position of a period or the number of characters, etc. Index information is attached to the divided documents so that the order of the divided documents can be determined. In other words, a set S of i divided documents s can be expressed as S = {s(0), s(1), ..., s(i-1)}. Note that the index information may not be attached to the documents, but may be managed externally and linked to the documents.

[0031] 3, the information "Refrigerator Instruction Manual" and "<Maintenance>" are missing from the document information of the divided document 1122. If document 1122 is used in this state, it is possible that the information "Refrigerator Door Maintenance" will not be useful.

[0032] A specific example of when information cannot be utilized will be described with reference to Figures 3 and 4. Assume that an input document 2100 is divided, and divided documents 1121, 1122, 1123, and 1124 are obtained.

[0033] If one wishes to obtain information on "refrigerator door maintenance," it is impossible to determine whether divided document 1122 or divided document 1124 is more appropriate based solely on the information in the divided documents. For example, if one wishes to obtain information on "washing machine door maintenance," referring to divided document 1122 could result in an error in the "maintenance frequency," which could result in damage.

[0034] The metadata setting unit 1130 includes a document composition information user interaction unit 1131 , a metadata generation unit 1132 , a metadata assignment unit 1133 , and a metadata recording unit 1134 .

[0035] The metadata setting unit 1130 generates document data with metadata added based on the input document configuration information 2200, the document 2100 obtained by the document acquisition unit 1110, and the document generated and divided by the document division unit 1120.

[0036] Here, the document composition information user interactive unit 1131 acquires document composition information 2200 input by transmitting and receiving interactive text data between the user and the user terminal device operated by the user. Fig. 5 shows an example of document composition information 2210 input by the user. Note that, in addition to the document composition information 2210, background information related to the document, such as date information and customer name, may also be input.

[0037] Furthermore, the metadata generation unit 1132 generates metadata based on the acquired document 2100 and the acquired document composition information 2200. FIG. 6 shows an overview of a process for generating metadata 2220 by performing natural language processing or the like based on the input document 2120 and the input document composition information 2210. For ease of explanation, FIG. 6 shows an overview of generating one piece of metadata using a portion of the document information, but for a document consisting of multiple chapters, metadata corresponding to the number of chapters is generated based on the document composition information 2210. When generating metadata, index information is assigned so that the order in which the metadata was generated can be determined. That is, a set M of metadata can be expressed as M={m(0), m(1), ..., m(j-1)} using j pieces of metadata m.

[0038] When generating metadata, if background information about the document, such as date information or customer name, is entered in the document configuration information, that information may be included in all metadata. Also, metadata may be generated by referring to or editing the metadata recording unit 1134.

[0039] Furthermore, the metadata assigning unit 1133 generates metadata-attached divided documents M1121, M1122, etc., based on the generated metadata 2220, the divided documents 1121, 1122, etc. Fig. 7 is a diagram showing metadata-attached divided documents M1121 and M1122 according to this embodiment.

[0040] The metadata assigning unit 1133 may generate the metadata 2220 so as to include one or more keywords contained in the input document 2120 and representing characteristics of the input document 2120. The keywords are provided to the question and answer generation system 1000 in the form of a definition file.

[0041] The metadata 2220 may also be generated manually rather than through processing by the metadata adding unit 1133 .

[0042] The document data numeric vector conversion unit 1140 embeds the metadata-attached segmented document M1121, metadata-attached segmented document M1122, etc. into numeric vectors. That is, it vectorizes each word token of the documents that make up the set MS of metadata-attached segmented documents to generate a numeric vector of the metadata-attached segmented documents. That is, the numeric vector VMS of the metadata-attached segmented documents can be expressed as VMS = {BERT(ms(0)), BERT(ms(1)), ..., BERT(ms(i-1))} using the BERT of i metadata-attached segmented documents ms.

[0043] In this embodiment, the multilingual-e5 model, which is a trained model based on the multilingual BERT (Bidirectional Encoder Representations from Transformers), is used for the numerical vectorization. This trained model is disclosed in the well-known document "Devlin, J, et al., BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding., 2018, arXiv preprint arXiv:1810.04805." The source code is also published in "Hugging Face (OSS) intfloat / multilingual-e5-base." Note that any method other than the above may be used as the method for numerical vectorization.

[0044] The answer generation unit 1200 has a question / answer user dialogue unit 1210, a question data numerical vector conversion unit 1220, a generation AI system 1230, a numerical vector similarity evaluation / search unit 1240, and a document data numerical vector recording unit 1250.

[0045] The answer generator 1200 generates an answer based on the question 2300 .

[0046] Here, the question / answer user dialogue unit 1210 acquires a question 2300 input by sending and receiving dialogue text data with a user terminal device operated by a user, and outputs an answer 2400 generated by the generation AI system 1230. The generation AI system 1230 is not limited to one that operates within the answer generation unit 1200, but may also operate on an external server connected via a network. The question 2300 can be expressed as Q = {q0}.

[0047] Furthermore, the question data numeric vector conversion unit 1220 converts the question 2300 into a numeric vector. That is, the question data numeric vector conversion unit 1220 vectorizes each word token of the documents that respectively constitute the question 2300 to generate a question data numeric vector. The question data numeric vector can be expressed as VQ = {BERT(q0)}.

[0048] Furthermore, document data numeric vector recording unit 1250 has a database for recording the document data numeric vectors generated by document data numeric vector conversion unit 1140. As shown in Fig. 8, this database includes, as items, index information unit 1251 for data indexing, numeric vector information unit 1252 representing the document data numeric vectors, and original data document unit 1253 representing information before conversion into the document data numeric vectors. Note that this database structure is merely an example, and other arbitrary elements may be added.

[0049] The numeric vector similarity evaluation and search unit 1240 also calculates and searches the similarity between the query data numeric vector and each numeric vector in the numeric vector information unit 1252. The similarity is calculated using the cosine similarity between the numeric vectors.

[0050] As shown in Fig. 9A, the cosine similarity between vectors is calculated based on each numeric vector in the numeric vector information section 1252 and the query data numeric vector. As a result of the calculation, the index in the index information section 1251 is associated with the similarity and stored, as shown in Fig. 9B. The database in the document data numeric vector recording section 1250 is then searched based on the index in the index information section 1251 associated with the similarity, and data in the original data document section 1253 corresponding to the index is obtained.

[0051] Note that the calculation of similarity is an example, and calculation of similarity between vectors other than the above may be performed. Furthermore, the method of determining which data in the original data document unit 1253 to acquire after similarity calculation is an example, and other evaluations may be performed to acquire the data. Furthermore, the data in the original data document unit 1253 corresponding to the index is not limited to the single data with the highest similarity, and multiple data with a predetermined ranking from the top of the similarity may be acquired.

[0052] Furthermore, an inquiry is made to the generation AI system 1230 based on the content of the original data document part 1253 acquired by the numerical vector similarity evaluation / search part 1240 and the question 2300. As a result, an answer 2400 based on the content of the original data document part 1253 acquired by the numerical vector similarity evaluation / search part 1240 and the question 2300 is output.

[0053] 10 is a diagram showing an example of a query combining an acquired document and a question according to an embodiment. Fig. 10 shows an example of a query based on the content of the original data document section 1253 acquired by the numerical vector similarity evaluation and search unit 1240 and a question 2300.

[0054] 10, the content of the original data document part 1253 is reflected in the "# information" part, and the content of the question 2300 is reflected in the "# question" part. Note that depending on the requirements of the application of this embodiment, the content of the part "Please respond to the next utterance from the user based on the information below" may be changed or instructions may be added.

[0055] Here, the generating AI system 1230 is connected to, for example, the Internet, and accesses the application programming interface (hereinafter referred to as "API") function of the generating AI system 1230 via the Internet. As the API of the generating AI system 1230, for example, the API service of ChatGPT-3.5 or ChatGPT-4 can be used. Note that the generating AI system 1230 may be constructed locally within an organization and accessed via an in-organization network such as a local area network.

[0056] Fig. 11 is a diagram showing the response output from ChatGPT-3.5 in response to an inquiry to ChatGPT-3.5 about the content shown in Fig. 10. As shown in Fig. 11, a response is generated using the information of the acquired document.

[0057] (Question and Answer Generation Process According to the Embodiment) FIG. 12 is a flowchart showing question and answer generation process according to the embodiment.

[0058] First, in step S101, the document divider 1120 divides the document 2100 acquired by the document acquirer 1110 to generate divided documents. The divided documents are exemplified in Figures 3 and 4. Next, in step S102, the metadata generator 1132 generates metadata from the document 2100 acquired by the document acquirer 1110.

[0059] Next, in step S103, the metadata adding unit 1133 executes the metadata-attached divisional document generation process, the details of which are as described with reference to FIG.

[0060] Next, in step S104, the document data numeric vector conversion unit 1140 converts the metadata-attached divided document generated in step S103 into a numeric vector, and stores the vector in the document data numeric vector recording unit 1250. The storage of the metadata-attached divided document converted into a numeric vector is as outlined above with reference to FIG.

[0061] Next, in step S105, the question data numeric vector conversion unit 1220 converts the data of the question 2300 received from the user by the question and answer user dialogue unit 1210 into a numeric vector. Next, in step S106, the numeric vector similarity evaluation and search unit 1240 executes a search process for metadata-attached divided documents that are similar to the question data, based on the data of the question 2300 that has been numeric vector converted in step S105. The metadata-attached divided document search process is as outlined above with reference to Figures 9A and 9B.

[0062] Next, in step S107, the numerical vector similarity evaluation / search unit 1240 inputs the metadata-attached segmented documents obtained by the search in step S106 to the generation AI system 1230 together with the question 2300 received from the user.

[0063] Next, in step S108, the question / answer user dialogue unit 1210 acquires the answer 2400 to the question 2300 output from the generation AI system 1230 to which the segmented document with metadata was input together with the question 2300 in step S107. The question / answer user dialogue unit 1210 outputs the acquired answer 2400 to a user terminal (not shown) or the like.

[0064] 13 is a flowchart showing the process of generating a divided document with metadata according to the embodiment. First, in step S103a, the metadata adding unit 1133 initializes a variable x. Next, in step S103b, the metadata adding unit 1133 initializes a variable y.

[0065] Next, in step S103c, the metadata adding unit 1133 determines whether a divided document s(x) exists. If a divided document s(x) exists (step S103c YES), the metadata adding unit 1133 proceeds to step S103d, and if a divided document s(x) does not exist (step S103c NO), the metadata-added divided document generation process ends.

[0066] In step S103d, the metadata adding unit 1133 adds metadata m(y) to the divided divided documents s(x) to generate metadata-attached divided documents ms(x). The set MS of i metadata-attached divided documents ms(x) can be expressed as MS = {ms(0), ms(1), ..., ms(i-1)}.

[0067] Next, in step S103e, the metadata addition unit 1133 increments the variable x by 1. Next, in step S103f, the metadata addition unit 1133 determines whether s(x) includes the document composition information that makes up m(y). If s(x) includes the document composition information that makes up m(y) (YES in step S103f), the metadata addition unit 1133 returns the process to step S103c, and if not (NO in step S103f), the metadata addition unit 1133 returns the process to step S103g.

[0068] In step S103g, the metadata adding unit 1133 increments the variable y by 1, and returns the process to step S103c.

[0069] By repeating steps S103c to S103g, each element of the metadata set M is assigned to each element of the divided document set S, and metadata-attached divided documents M1121, M1122, etc. are generated. For example, the same element of the metadata set M is assigned to each element of the divided document set S.

[0070] Effect of the embodiment The question and answer generation system 1000 in the above-described embodiment has a newly added mechanism for adding metadata (general information not present in the divided documents) to the divided documents. This makes it possible to generate highly accurate answers to questions even when any document with a specific format or style is used as additional information for the RAG.

[0071] In addition, in the above-described embodiment, by generating metadata from the original document and assigning the metadata to the split documents obtained by dividing the original document, it is possible to assign metadata to the split documents even if the document has not been converted to DX and does not have metadata.

[0072] In the above-described embodiment, the metadata includes one or more keywords that represent the characteristics of the document, which makes it possible to generate the metadata intuitively and easily.

[0073] In the above-described embodiment, the parts corresponding to the structure indicated in the document structure information indicating the structure of the document are extracted as metadata. This makes it possible to segment structured documents in which descriptions of different contents are repeated with the same sentence structure, such as an internal manual, and assign metadata to them.

[0074] In the above-described embodiment, the document structure information includes a format or style of the text, including chapters, sections, and paragraphs of the document, or related information related to the document, which allows the document to be divided based on a format aspect of the text, such as a format or style of the text, including chapters, sections, and paragraphs of the document, or a content aspect of the document, such as related information.

[0075] In the above-described embodiment, the same metadata is assigned to multiple divided documents obtained by dividing the same document. This allows divided documents with the same original document to be understood based on the same metadata and the same criteria, so even divided documents that were not properly divided from the original document can be used as additional information for the RAG.

[0076] (Hardware Configuration of Computer 2000) Fig. 14 is a diagram showing an example of the hardware configuration of the computer 2000. The computer 2000 realizes each unit of the question and answer generation system 1000 by executing a predetermined program.

[0077] The computer 2000 comprises a processor 2001 including a CPU, a main memory device 2002, an auxiliary memory device 2003, a network interface 2004, an input device 2005, and an output device 2006, all of which are interconnected via an internal communication line 2007 such as a bus.

[0078] The processor 2001 controls the overall operation of the computer 2000. The main storage device 2002 is composed of, for example, a volatile semiconductor memory, and is used as a work memory for the processor 2001. The auxiliary storage device 2003 is composed of a large-capacity nonvolatile storage device such as a hard disk device, an SSD (Solid State Drive), or a flash memory, and is used to store various programs and data for a long period of time.

[0079] An executable program 2003 a stored in the auxiliary storage device 2003 is loaded into the main storage device 2002 when the computer 2000 is started up or when necessary, and is executed by the processor 2001 .

[0080] The executable program 2003a may be recorded on a non-transitory recording medium, read from the non-transitory recording medium by a medium reading device, and loaded into the main memory device 2002. Alternatively, the executable program 2003a may be obtained from an external computer via a network and loaded into the main memory device 2002.

[0081] The auxiliary storage device 2003 stores various executable programs 2003a.

[0082] The network interface 2004 is an interface device for connecting the computer 2000 to each network within the system or for communicating with other computers. The network interface 2004 is configured, for example, by a NIC (Network Interface Card) for a wired LAN (Local Area Network) or a wireless LAN.

[0083] The input device 2005 is composed of a keyboard, a pointing device such as a mouse, and the like, and is used by the user to input various instructions and information to the computer 2000. The output device 2006 is composed of a display device such as a liquid crystal display or an organic EL (Electro Luminescence) display, and an audio output device such as a speaker, and is used to present necessary information to the user when necessary.

[0084] The present invention is not limited to the above-described embodiments, and includes various modifications. Furthermore, the above-described embodiments have been described in detail to clearly explain the present invention, and are not necessarily limited to those having all of the described configurations. Furthermore, not only are such configurations removed, but replacement or addition of configurations is also possible. Furthermore, forms in which some or all of the above-described embodiments are appropriately combined to be consistent are also included in the embodiments of the present invention.

[0085] Furthermore, the above-described configurations, functions, processing units, processing means, etc. may be partially or entirely implemented in hardware, for example, by designing them as integrated circuits. The present invention can also be implemented by software program code that implements the functions of the embodiments. In this case, a recording medium on which the program code is recorded is provided to a computer, and a processor within the computer reads the program code stored in the recording medium.

[0086] In this case, the program code itself read from the recording medium will realize the functions of the above-described embodiment, and the program code itself and the recording medium on which it is stored constitute the present invention. Examples of recording media for supplying such program code include flexible disks, CD-ROMs, DVD-ROMs, hard disks, SSDs (Solid State Drives), optical disks, magneto-optical disks, CD-Rs, magnetic tapes, non-volatile memory cards, ROMs, etc.

[0087] Furthermore, the program code that realizes the functions described in this embodiment can be implemented in a wide range of program or script languages, such as assembler, C / C++, perl, Shell, PHP, and Java (registered trademark).

[0088] In the above-described embodiment, the control lines and information lines are those that are considered necessary for the explanation, and not all control lines and information lines in the product are necessarily shown. All components may be interconnected.

[0089] 1000: Question and answer generating device, 2000: Computer, 2001: Processor, 2002: Main memory device.

Claims

1. A question and answer generation system that generates answers to questions, the system having a processor and memory, wherein the processor: divides a document into a plurality of divided documents of predetermined units; generates metadata-attached divided documents by assigning metadata extracted from the document to each of the divided documents, the metadata-attached divided documents being similar to the question; adds the metadata-attached divided documents obtained as a result of the search to the question and inputs them into a language model; and obtains the answer generated by the language model using the question to which the metadata-attached divided documents have been attached as input.

2. A question and answer generation system according to claim 1, wherein the metadata includes one or more keywords that represent characteristics of the document.

3. A question and answer generation system according to claim 1, characterized in that the processor extracts as the metadata a portion corresponding to the structure indicated in document structure information indicating the structure of the document.

4. A question and answer generation system as claimed in claim 3, characterized in that the document structure information includes a format or style of the text including chapters, sections and paragraphs of the document, or related information related to the document.

5. A question and answer generation system according to claim 1, wherein the processor assigns the same metadata to a plurality of divided documents obtained by dividing the same document.

6. A question and answer generation method executed by a question and answer generation system that generates answers to questions, the question and answer generation system having a processor and memory, the question and answer generation method comprising the steps of: dividing a document into a plurality of divided documents of predetermined units; generating metadata-attached divided documents by assigning metadata extracted from the document to each of the divided documents; searching for the metadata-attached divided documents that are similar to the question; adding the metadata-attached divided documents obtained as a result of the search to the question and inputting them into a language model; and obtaining the answer generated by the language model using the question to which the metadata-attached divided documents have been attached as input.

Images