Question answering device and its program

Abstract

In question answering systems, the accuracy of answers obtained using large-scale language models can be improved. [Solution] The question answering device creates a virtual question sentence from a text fragment and stores the text fragment and the virtual question sentence vector, which represents the virtual question sentence created from the text fragment in distributed representation, as one record in the database. For each record stored in the database, the question answering device calculates the similarity between the virtual question sentence vector contained in that record and the question sentence vector, which represents the input question sentence in distributed representation. Based on the similarity calculated for each record, the question answering device extracts a predetermined number of text fragments from the database, which are contained in a predetermined number of records. The question answering device generates an answer to the question sentence based on the question sentence and the predetermined number of extracted text fragments.

Description

Technical Field

[0001] Embodiments of the present invention relate to a question-and-answer device and its program.

Background Art

[0002] In recent years, a question-and-answer system has been developed that uses large language models (LLMs) to create answers to questions in user documents and outputs the answers to the user in documents. In this type of question-and-answer system, a technique called Retrieval Augmenter Generation (RAG) is used. Retrieval Augmenter Generation is a technique in which, in advance, the sentences of a document targeted by a question are divided into a plurality of sentence fragments and stored in a database, and one or more sentence fragments similar to the question document are extracted from the database. The extracted sentence fragments are input into a large language model together with the question document. The large language model utilizes the sentence fragments extracted by Retrieval Augmenter Generation, generates an answer to the question document using the knowledge obtained by learning a vast amount of language data, and outputs the answer in a document.

[0003] Thus, by using the technique of Retrieval Augmenter Generation, the large language model obtains an answer by referring not only to the question document but also to sentence fragments similar to the question. However, the sentence fragments referred to by the large language model are not always appropriate. There is a concern that the accuracy of the answers obtained by the large language model may deteriorate due to the mixing of inappropriate sentence fragments. [[ID=,18]]

Prior Art Documents

Patent Documents

[0004] <,

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] The problem that the embodiments of the present invention aim to solve is to provide a question answering device that can improve the accuracy of answers obtained from large-scale language models. [Means for solving the problem]

[0006] In one embodiment, the question answering device comprises a virtual question creation unit, a storage unit, a similarity calculation unit, a text fragment extraction unit, and an answer generation unit. The virtual question creation unit creates a virtual question from text fragments. The storage unit stores the text fragments and a virtual question vector, which represents the virtual question created from the text fragments in distributed representation, as one record in the database. The similarity calculation unit calculates the similarity between the virtual question vector contained in each record stored in the database and the question vector, which represents the input question in distributed representation. The text fragment extraction unit extracts a predetermined number of text fragments from the database, each containing a predetermined number of records, based on the similarity calculated for each record by the similarity calculation unit. The answer generation unit generates an answer to the question based on the question and the predetermined number of text fragments extracted by the text fragment extraction unit. [Brief explanation of the drawing]

[0007] [Figure 1] Figure 1 is a schematic diagram showing the general configuration of the question answering system according to the embodiment. [Figure 2] Figure 2 is a block diagram showing the main circuit configuration of the question answering device. [Figure 3] Figure 3 is an explanatory diagram of the document fragment database. [Figure 4] Figure 4 is a schematic diagram showing the data structure of a document fragment record stored in the document fragment database in the first embodiment. [Figure 5] Figure 5 is a block diagram showing the configuration of the first function of the question answering device in the first embodiment. [Figure 6] Figure 6 is a block diagram showing the configuration of the second function of the question answering device in the first embodiment. [Figure 7]Figure 7 is a flowchart showing the main steps of information processing performed by the question answering device's processor according to the literature registration program in the first embodiment. [Figure 8] Figure 8 is a flowchart showing the main steps of information processing performed by the processor of the question answering device in accordance with the question answering program in the first embodiment. [Figure 9] Figure 9 is a schematic diagram showing the data structure of a document fragment record stored in the document fragment database in the second embodiment. [Figure 10] Figure 10 is a block diagram showing the configuration of the first function of the question answering device in a second embodiment. [Figure 11] Figure 11 is a block diagram showing the configuration of the second function of the question answering device in the second embodiment. [Figure 12] Figure 12 is a flowchart showing the main steps of the information processing performed by the question answering device's processor according to the second literature registration program. [Figure 13] Figure 13 is a flowchart showing the essential steps of the information processing performed by the question answering device's processor according to the second question answering program. [Figure 14] Figure 14 is a block diagram showing the configuration of the second function of the question answering device in the third embodiment. [Figure 15] Figure 15 is a flowchart showing the main steps of information processing performed by the processor of the question answering device in accordance with the third question answering program in the third embodiment. [Figure 16] Figure 16 is a flowchart showing the main steps of information processing performed by the processor of the question answering device in accordance with the third question answering program in the third embodiment. [Modes for carrying out the invention]

[0008] Below, several embodiments of a question answering device that can improve the accuracy of responses obtained from large-scale language models will be described with reference to the drawings.

[0009] <First Embodiment> [Overview of the Question-Answering System] First, a question-answering system including a question-answering device will be described. FIG. 1 is a schematic diagram showing the schematic configuration of a question-answering system 1 according to an embodiment. The question-answering system 1 includes a question-answering device 10, an administrator terminal 20, and a plurality of user terminals 30. The question-answering system 1 connects the question-answering device 10, the administrator terminal 20, and the plurality of user terminals 30 via a communication network 40. The communication network 40 is a wide-area network such as the Internet or an intranet. A mobile communication network, a public communication network, etc. may be used as part of the communication network 40.

[0010] The question-answering system 1 is a system that creates an answer using a large language model (LLM) for a question by a user's document and answers the user with a document. The large language model uses a technique called Retrieval-Augmented Generation (RAG) to generate an answer to a question. Hereinafter, the technique called Retrieval-Augmented Generation is represented as the RAG method.

[0011] The user terminal 30 is a computer terminal used by a user who obtains an answer to a question using the question-answering system 1. When the user operates the user terminal 30 to input a question in a document, a document of the answer to the question is output from the question-answering device 10 to the user terminal 30. The user terminal 30 displays the answer document on a display device. Alternatively, the user terminal 30 outputs the voice of reading the answer document from a speaker. Thus, the user can obtain an answer to the question. In this way, a personal computer, a tablet terminal, a smartphone, etc. equipped with an input device capable of inputting a question in a document and a display device or a speaker capable of outputting an answer in a document can be the user terminal 30.

[0012] The administrator terminal 20 is a computer terminal used by the administrator of the question-and-answer system 1. The administrator uses the administrator terminal 20 to input electronic data of various documents such as documents, materials, records, books, etc. that can serve as reference materials for various questions, so-called documents. The documents are, for example, PDF files with the extension "pdf", word files with the extension "docx" or "doc", PowerPoint files with the extension "ppt", text files with the extension "txt", etc. A personal computer, tablet terminal, smartphone, etc. that can input this type of document can be the administrator terminal 20. Needless to say, the types of documents are not limited to the PDF files, word files, PowerPoint files, and text files with the above-mentioned extensions.

[0013] The question-and-answer device 10 is a server computer that provides a question-and-answer service with the user terminal 30 as a client. That is, when the question-and-answer device 10 receives a question from the user terminal 30 which is a client, it generates an answer to the question and provides a service to output the answer to the user terminal 30. To provide such a service, the question-and-answer device 10 uses a large language model (LLM) using the RAG method. A large language model is a language model constructed by a large amount of text data and deep learning technology. The question-and-answer device 10 uses the large language model to generate an answer to the question input in the document from the user terminal 30 and outputs the answer to the user terminal 30 in the document.

[0014] [Explanation of the Configuration of the Question-and-Answer Device] Figure 2 is a block diagram showing the main circuit configuration of the question answering device 10. The question answering device 10 comprises a processor 11, main memory 12, auxiliary storage device 13, network interface 14, and system transmission path 15. The system transmission path 15 includes an address bus, data bus, control signal lines, etc. The system transmission path 15 connects the processor 11 to the other parts and transmits data signals exchanged between them. The question answering device 10 constitutes a computer by connecting the processor 11, the main memory 12, the auxiliary storage device 13, and the network interface 14 via the system transmission path 15.

[0015] The processor 11 corresponds to the central part of the computer described above. The processor 11 controls each part to realize various functions as a question answering device 10 according to the operating system or application program. The processor 11 is, for example, a CPU (Central Processing Unit). The processor 11 may also be, for example, an MPU (Micro Processing Unit), SoC (System on a Chip), DSP (Digital Signal Processor), GPU (Graphics Processing Unit), ASIC (Application Specific Integrated Circuit), PLD (Programmable Logic Device), or FPGA (Field-Programmable Gate Array). Alternatively, the processor 11 may be a combination of several of these.

[0016] Main memory 12 corresponds to the main memory portion of the computer described above. Main memory 12 includes a non-volatile memory area and a volatile memory area. In the non-volatile memory area, main memory 12 stores the operating system or application programs. Main memory 12 may also store data necessary for the processor 11 to perform processing to control each part in the non-volatile or volatile memory area. Main memory 12 uses the volatile memory area as a work area where data is rewritten as needed by the processor 11. The non-volatile memory area is, for example, ROM (Read Only Memory). The volatile memory area is, for example, RAM (Random Access Memory).

[0017] The auxiliary storage device 13 corresponds to the auxiliary storage portion of the computer described above. For example, EEPROM (Electric Erasable Programmable Read-Only Memory), HDD (Hard Disk Drive), or SSD (Solid State Drive) can be the auxiliary storage device 13. The auxiliary storage device 13 stores data used by the processor 11 in performing various processes, data created by the processing performed by the processor 11, etc. The auxiliary storage device 13 may also store the application program described above.

[0018] The network interface 14 connects to the communication network 40. The question answering device 10 communicates data with the administrator terminal 20 and user terminal 30 connected to the communication network 40 via the network interface 14 according to a pre-configured communication protocol.

[0019] In the question answering device 10 with this configuration, a portion of the storage area in the auxiliary storage device 13 is used as a text fragment database 16. The text fragment database 16 is a storage area prepared for storing data related to various text fragments necessary for the RAG method.

[0020] Figure 3 is an explanatory diagram of the text fragment database 16. As shown in Figure 3, the text fragment database 16 stores multiple N text fragment records 161 obtained from multiple documents 100, 200, 300, ... Incidentally, in Figure 5, [100-1] to [100-N] represent text fragment records 161 obtained from document 100, [200-1] to [200-N] represent text fragment records 161 obtained from document 200, and [300-1] to [300-N] represent text fragment records 161 obtained from document 300. Note that the number N of text fragment records 161 obtained from a single document is an arbitrary value determined by the length and content of the text described in the document, and is not uniform.

[0021] Figure 4 is a schematic diagram showing the data structure of a single text fragment record 161. A text fragment record 161 is a data record that includes a record number (No.), a text fragment, multiple (five in the figure) virtual question sentences, and the same number of virtual question sentence vectors. The record number is a series of numbers that are issued each time a text fragment record 161 is created. The record number is unique identification information that identifies the text fragment. The record number can be rephrased as, for example, a text fragment ID.

[0022] A text fragment is text data extracted from a document as a sentence within a specified character count. For example, a text fragment consists of sentences of 500 characters or less, separated by punctuation or line breaks.

[0023] A virtual question is text data of a question virtually created from a text fragment so that the information contained in the text fragment serves as the answer. Hereinafter, a question virtually created from a text fragment will be referred to as a virtual question. A virtual question is created by inputting a text fragment into a large-scale language model. In this embodiment, up to five virtual questions are created for one text fragment, and the text data for each is written in a common text fragment record 161.

[0024] A virtual question vector is high-dimensional vector information that represents the meaning of a virtual question using a distributed representation. In this embodiment, each virtual question vector for multiple virtual questions created from a single sentence fragment is described in a common sentence fragment record 161.

[0025] Note that the text fragment record 161 does not necessarily have to contain a virtual question. That is, the text fragment record 161 may consist of a text fragment and a virtual question vector. Here, the text fragment database 16 is a database that stores a text fragment and a virtual question vector, which represents the virtual question created from the text fragment using distributed representation, as one record.

[0026] [Description of the function of the question answering device] Figure 5 is a block diagram showing the configuration of the first function of the question answering device 10. The first function is a function related to the administrator terminal 20. The question answering device 10 has the functions of a document input unit 51, a splitting unit 52, a document fragment acquisition unit 53, a virtual question creation unit 54, a virtual question conversion unit 55, and a storage unit 56 as its first function.

[0027] The document input unit 51 is a function that extracts text from documents, i.e., electronic data of various documents, entered via the administrator terminal 20. When an administrator operates the administrator terminal 20 and electronic data of a document, consisting of a PDF file, Word file, PowerPoint file, or text file, is entered, that electronic data is transmitted to the question answering device 10 via the communication network 40. When the document input unit 51 receives the electronic data of the document via the network interface 14, it extracts text data, i.e., the text of the document, from that electronic data.

[0028] The splitting unit 52 has the function of dividing the text of a document imported via the document input unit 51 into sentence fragments of a predetermined number of characters or less. For example, the splitting unit 52 divides the text into sentence fragments of 500 characters or less. The number of characters in each sentence fragment does not necessarily have to be close to the predetermined number of characters. The splitting unit 52 divides the text at convenient points such as punctuation marks and line breaks within the predetermined number of characters. The divided sentence fragments may have some overlap with other sentence fragments. This function of the splitting unit 52 is generally called chunking.

[0029] The text fragment acquisition unit 53 has the function of acquiring text fragments that have been divided from the document text by the division unit 52. The text fragment acquisition unit 53 acquires one text fragment each time the document text is divided by the division unit 52. The text fragment acquisition unit 53 may also temporarily store the text fragments divided by the division unit 52 in memory and acquire the text fragments one by one from memory at an appropriate time.

[0030] The virtual question generation unit 54 is a function that creates a virtual question from a single text fragment acquired by the text fragment acquisition unit 53. The virtual question generation unit 54 virtually creates a question, i.e., a virtual question, in which the information contained in the text fragment serves as the answer. The virtual question generation unit 54 creates multiple virtual questions using a large-scale language model. For example, the virtual question generation unit 54 can create up to five virtual questions. The number of virtual questions to be created for a single text fragment is arbitrary, but approximately five is appropriate from the standpoint of answer accuracy and processing efficiency.

[0031] The virtual question conversion unit 55 is a function that converts virtual question sentences created by the virtual question creation unit 54 into virtual question vectors. The virtual question conversion unit 55 converts multiple virtual question sentences created by the virtual question creation unit 54 into virtual question vectors that represent the meaning of each virtual question sentence as a high-dimensional numerical vector. The virtual question conversion unit 55 converts multiple virtual question sentences created by the virtual question creation unit 54 into virtual question vectors by utilizing a well-known technique called embedding (embedded representation).

[0032] The storage unit 56 has the function of saving the document fragment record 161 to the document fragment database 16. That is, the storage unit 56 creates a document fragment record 161 that includes, as one record, the document fragment acquired by the document fragment acquisition unit 53, the multiple virtual question sentences created by the virtual question sentence creation unit 54 for that document fragment, and the multiple virtual question sentence vectors converted by the virtual question sentence conversion unit 55 for each of those virtual question sentences, and saves it to the document fragment database 16. The storage unit 56 repeats the process of creating a document fragment record 161 containing the document fragment and saving it to the document fragment database 16 each time the document fragment acquisition unit 53 acquires a document fragment that has been divided from the document by the division unit 52. Alternatively, the storage unit 56 may create a document fragment record that includes the document fragment and the virtual question sentence vector for each virtual question sentence and save it to the document fragment database 16.

[0033] Furthermore, the document input unit 51, the division unit 52, the text fragment acquisition unit 53, the virtual question creation unit 54, the virtual question conversion unit 55, and the storage unit 56 can be rephrased as document input means 51, division means 52, text fragment acquisition means 53, virtual question creation means 54, virtual question conversion means 55, and storage means 56, respectively.

[0034] Figure 6 is a block diagram showing the configuration of the second function of the question answering device 10. The second function is a function related to the user terminal 30. The question answering device 10 has the functions of a question input unit 61, a question text conversion unit 62, a similarity calculation unit 63, a text fragment similarity identification unit 64, a text fragment extraction unit 65, an answer generation unit 66, and an answer unit 67 as its second function.

[0035] The question input unit 61 is a function that takes in questions entered as documents via the user terminal 30. When a question is entered as a document through user operation on the user terminal 30, the document data is transmitted to the question answering device 10 via the communication network 40. When the question input unit 61 receives the document data of the question via the network interface 14, it takes the document data as a question.

[0036] The question text conversion unit 62 is a function that converts a question text received via the question input unit 61 into a question text vector. The question text conversion unit 62 converts a question text into a question text vector, which represents the meaning of the question text as a high-dimensional numerical vector. For example, the question text conversion unit 62 converts a question text input via the question input unit 61 into a question text vector by utilizing a well-known technique called embedding. Here, the core engine for embedding used by the question text conversion unit 62 and the core engine for embedding used by the virtual question text conversion unit 55 are the same. Therefore, the question text vector converted by the question text conversion unit 62 and the virtual question text vector converted by the virtual question text conversion unit 55 become multidimensional vector information representing a common distributed representation.

[0037] The similarity calculation unit 63 is a function that calculates the similarity between the question vector of the question text taken in via the question input unit 61 and the virtual question vector of the document fragment record 161 stored in the document fragment database 16. The similarity between the question vector and the virtual question vector is, for example, cosine similarity. The similarity calculation unit 63 sequentially takes in the document fragment records 161 from the document fragment database 16. Then, for each of the multiple virtual question vectors contained in the document fragment record 161, the similarity calculation unit 63 calculates the cosine value of the angle it makes with the question vector as the similarity.

[0038] The text fragment similarity identification unit 64 is a function that identifies the similarity between the text fragment contained in each text fragment record 161 stored in the text fragment database 16 and the question text. The text fragment similarity identification unit 64 identifies the similarity between the text fragment and the question text based on the similarity between the virtual question text vector and the question text vector calculated for each text fragment record 161 by the similarity calculation unit 63. For example, the text fragment similarity identification unit 64 identifies the maximum value of the similarity with the question text vector calculated for each virtual question text vector as the similarity between the text fragment and the question text.

[0039] The text fragment extraction unit 65 has the function of extracting a predetermined number of text fragments in order from the text fragment similarity records 161 identified by the text fragment similarity identification unit 64. The text fragment extraction unit 65 ranks the text fragments in descending order of similarity identified by the text fragment similarity identification unit 64. The text fragment extraction unit 65 then selects text fragment records 161 in a predetermined rank, for example, up to the 3rd rank, starting from the text fragment record 161 with the highest similarity, and extracts text fragments from each of the selected predetermined number of text fragment records 161.

[0040] The answer generation unit 66 is a function that generates answers to the question. The answer generation unit 66 generates answers using a large-scale language model (LLM). That is, the answer generation unit 66 inputs the question and a predetermined number of text fragments extracted by the text fragment extraction unit 65 into the large-scale language model (LLM) to generate answers to the question.

[0041] The response unit 67 has the function of outputting the response generated by the response generation unit 66 as a document to the user terminal 30. That is, the response unit 67 transmits document data indicating the response via the network interface 14. The transmitted document data is transmitted via the communication network 40 and received by the user terminal 30 that entered the question regarding the response. Thus, the response is output as a document on the user terminal 30.

[0042] The question input unit 61, question text conversion unit 62, similarity calculation unit 63, text fragment similarity identification unit 64, text fragment extraction unit 65, answer generation unit 66, and answer unit 67 can be rephrased as question input means 61, question text conversion means 62, similarity calculation means 63, text fragment similarity identification means 64, text fragment extraction means 65, answer generation means 66, and answer means 67, respectively.

[0043] [Description of the literature registration program] The first function, consisting of a document input unit 51, a splitting unit 52, a document fragment acquisition unit 53, a virtual question creation unit 54, a virtual question conversion unit 55, and a storage unit 56, is realized by information processing performed by the processor 11 in accordance with the document registration program.

[0044] The bibliography registration program is a type of application program stored in the main memory 12 or the auxiliary storage device 13. The method of installing the bibliography registration program in the main memory 12 or auxiliary storage device 13 is not particularly limited. The bibliography registration program can be recorded on a removable recording medium, or distributed via communication through the communication network 40, and then installed in the main memory 12 or auxiliary storage device 13. The recording medium can be any form as long as it can store a program and is readable by the device, such as a CD-ROM or memory card.

[0045] Figure 7 is a flowchart showing the main steps of information processing performed by the processor 11 of the question answering device 10 according to the literature registration program. When the literature registration program is started, the processor 11 waits for literature to be input as ACT1. In this waiting state, when the processor 11 receives electronic data of the literature from the administrator terminal 20, it proceeds to ACT2. As ACT2, the processor 11 extracts the text of the literature from the electronic data.

[0046] The processor 11, having received the text, proceeds to ACT3. In ACT3, the processor 11 divides the text of the document into sentence fragments of a predetermined number of characters or less. For example, the processor 11 divides the text at convenient points such as punctuation marks or line breaks, where each fragment is 500 characters or less.

[0047] The processor 11, having divided the text, proceeds to ACT4. In ACT4, the processor 11 sets the number of divisions, i.e., the number of text fragments divided from the text, into register N. Also, in ACT5, the processor 11 resets counter n to "0".

[0048] After setting the number of text fragments in register N and resetting counter n, processor 11 proceeds to ACT6. Processor 11 increments counter n by "1" in ACT6. Then, in ACT7, processor 11 checks whether the value of counter n exceeds the value of register N. If the value of counter n does not exceed the value of register N, processor 11 proceeds to ACT8. In ACT8, processor 11 selects the nth text fragment from the N fragments that were divided from the text. Incidentally, the "n" in nth is the value of counter n.

[0049] When processor 11 selects the nth sentence fragment, it proceeds to ACT9. Processor 11 resets counter p to "0" as ACT9. Next, processor 11 increments counter p by "1" as ACT10. Then, as ACT11, processor 11 checks whether the value of counter p exceeds the set value P. The set value P is the total number of virtual question sentences to be created from the sentence fragments, and can be an integer of 2 or more. In this embodiment, the set value P is set to "5".

[0050] In ACT11, if the value of counter p does not exceed the set value P, processor 11 proceeds to ACT12. Processor 11 creates a virtual question sentence from the nth sentence fragment as ACT12. That is, processor 11 inputs the nth sentence fragment into a large-scale language model for creating virtual question sentences and creates a virtual question sentence.

[0051] The processor 11, which has created the virtual question, proceeds to ACT 13. The processor 11 converts the virtual question into a virtual question vector as ACT 13. That is, the processor 11 converts the virtual question into a virtual question vector by representing the features of the virtual question as numerical vectors using the embedding engine.

[0052] After converting the virtual question into a virtual question vector, the processor 11 proceeds to ACT 14. In ACT 14, the processor 11 stores the virtual question created in ACT 12 and the feedback question vector obtained in ACT 13 in the virtual question memory. The virtual question memory is, for example, a part of the volatile memory area in the main memory 12.

[0053] The processor 11, having stored the virtual question sentence and the feedback question sentence vector, returns to ACT 10. The processor 11 increments counter p by another "1". After confirming that counter p has not exceeded the set value P, the processor 11 executes the processes of ACT 12 to ACT 14 in the same manner as described above. That is, the processor 11 creates a second virtual question sentence from the nth sentence fragment, converts that virtual question sentence into a virtual question sentence vector, and stores the virtual question sentence and virtual question sentence vector in the virtual question sentence memory.

[0054] In this way, processor 11 executes the processing of ACT12 to ACT14 in ACT11 until counter p exceeds the set value P. Thus, P virtual question sentences are created from the nth sentence fragment, these virtual question sentences are converted into virtual question sentence vectors, and the virtual question sentences and virtual question sentence vectors are stored in the virtual question sentence memory.

[0055] In ACT11, when counter p exceeds the set value P, processor 11 proceeds to ACT15. Processor 11 issues record number z as ACT15. The record number z issued in ACT15 is the largest record number of the document fragment records 161 currently stored in the document fragment database 16 plus "1".

[0056] The processor 11, which issued record number z, proceeds to ACT 16. The processor 11 creates a document fragment record 161 as ACT 16. That is, the processor 11 creates a document fragment record 161 that includes the record number z issued in ACT 15, the text data of the document fragment selected in ACT 8, and the P virtual question sentences and virtual question sentence vectors stored in the virtual question sentence memory in ACT 14.

[0057] The processor 11, having created the document fragment record 161, proceeds to ACT 17. The processor 11 saves the document fragment record 161 to the document fragment database 16 as ACT 17. After that, the processor 11 returns to ACT 6. The processor 11 increments the counter n by another "1". Then, after confirming that the counter n does not have the number of divisions N, the processor 11 executes the processing of ACT 8 to ACT 17 in the same manner as described above. Thus, P virtual question sentences are created for each document fragment divided from the document taken in ACT 2, a document fragment record 161 containing the document fragment and a virtual question sentence vector for each virtual question sentence is created, and the document fragment record 161 is saved to the document fragment database 16.

[0058] Processor 11 repeats the processing of ACT8 to ACT17 until counter n exceeds the number of divisions N in ACT7. When counter n exceeds the number of divisions N, processor 11 terminates the information processing of the procedure based on the literature registration program.

[0059] Here, processor 11 realizes the function of a document input unit 51 through the processing of ACT1 and ACT2. Processor 11 realizes the function of a division unit 52 through the processing of ACT3. Processor 11 realizes the function of a document fragment acquisition unit 53 through the processing of ACT5 to ACT8. Processor 11 realizes the function of a virtual question creation unit 54 through the processing of ACT9 to ACT12. Processor 11 realizes the function of a virtual question conversion unit 55 through the processing of ACT13. Processor 11 realizes the function of a storage unit 56 through the processing of ACT15 to ACT17.

[0060] Note that the processing procedure of the literature registration program shown in the flowchart in Figure 7 is just one example. The order and content of the processing can be changed as appropriate if similar effects can be achieved.

[0061] [Explanation of the Question Answering Program] The second function, consisting of a question input unit 61, a question text conversion unit 62, a similarity calculation unit 63, a text fragment similarity identification unit 64, a text fragment extraction unit 65, an answer generation unit 66, and an answer unit 67, is realized by information processing performed by the processor 11 according to the question answer program.

[0062] The question-answering program is a type of application program stored in the main memory 12 or the auxiliary storage device 13. The method of installing the question-answering program in the main memory 12 or auxiliary storage device 13 is not particularly limited. The question-answering program can be recorded on a removable recording medium, or distributed via communication over the communication network 40, and then installed in the main memory 12 or auxiliary storage device 13. The recording medium can be any form as long as it can store a program and is readable by the device, such as a CD-ROM or memory card.

[0063] Figure 8 is a flowchart showing the main steps of information processing performed by the processor 11 of the question answering device 10 according to the question answering program. When the question answering program is started, the processor 11 waits for a question to be input as ACT21. In this waiting state, when a question is input from the user terminal 30, the processor 11 proceeds to ACT22. As ACT22, the processor 11 takes in the document input as a question, i.e., the text data of the question statement.

[0064] Processor 11, having received the text data of the question, proceeds to ACT23. Processor 11 converts the question into a question vector as ACT23. That is, processor 11 converts the question into a question vector by using the same embedding engine used for vectorizing the virtual question to represent the features of the question as numerical vectors.

[0065] The processor 11, having converted the question text into a question text vector, proceeds to ACT24. The processor 11 stores the question text vector as ACT24 in the question text memory. The question text memory is, for example, a part of the volatile memory area in the main memory 12.

[0066] The processor 11, having stored the question vector, proceeds to ACT25. As ACT25, the processor 11 sets the number of records of all text fragment records 161 stored in the text fragment database 16 into register R. The processor 11 also resets counter r to "0" as ACT26.

[0067] After setting the record count of document fragment record 161 in register R and resetting counter r, processor 11 proceeds to ACT27. In ACT27, processor 11 increments counter r by "1". Then, in ACT28, processor 11 checks whether the value of counter r has exceeded the value of register R.

[0068] If the value of counter r does not exceed the value of register R, processor 11 proceeds to ACT29. As ACT29, processor 11 retrieves from the document fragment database 16 a document fragment record 161 whose record number matches the value of counter r, i.e., document fragment record 161 with record number r. Then, as ACT30, processor 11 individually calculates the similarity of all virtual question vectors contained in document fragment record 161 with record number r to the question vectors stored in the question memory. That is, for each virtual question vector, processor 11 compares the multidimensional vector information that constitutes that virtual question vector with the same-dimensional vector information that constitutes the question vector, and individually calculates the similarity of the vector information, for example, the cosine similarity.

[0069] After calculating the similarity between all virtual question vectors contained in document fragment record 161 and the question vector, the processor 11 proceeds to ACT31. In ACT31, the processor 11 compares the similarity between each virtual question vector and the question vector and selects the one with the highest similarity. Then, in ACT32, the processor 11 sets that highest similarity as the document fragment similarity for document fragment record 161 with record number r.

[0070] The processor 11, having set the sentence fragment similarity, proceeds to ACT33. As ACT33, the processor 11 associates the record number r with the sentence fragment similarity of the sentence fragment record 161 for record number r and stores it in table memory. The table memory is, for example, a part of the volatile memory area in main memory 12.

[0071] Processor 11, having stored the record number r and the sentence fragment similarity in table memory, returns to ACT27. Processor 11 increments counter r by another "1". After confirming that the value of counter r does not exceed the value of register R, Processor 11 executes the processes of ACT29 to ACT33 in the same manner as described above. That is, Processor 11 retrieves sentence fragment record 161 with record number r from sentence fragment database 16, and calculates the similarity between each virtual question vector and the question vector contained in sentence fragment record 161 with record number r. Processor 11 then compares the similarity calculated for each virtual question vector and selects the highest similarity. Processor 11 identifies this highest similarity as the sentence fragment similarity for sentence fragment record 161 with record number r, and stores the sentence fragment similarity together with record number r in table memory.

[0072] The processor 11 repeatedly executes the processes of ACT29 to ACT33 until the value of counter r exceeds the value of register R. Thus, the table memory stores for each document fragment record 161 the document fragment similarity value identified for each document fragment record 161, that is, the maximum similarity value between each virtual question vector and question vector contained in that document fragment record 161.

[0073] In ACT28, when the value of counter r exceeds the value of register R, the processor 11 proceeds to ACT34. In ACT34, the processor 11 compares the fragment similarity of each fragment record 161 stored in table memory. Then, in ACT35, the processor 11 ranks the fragments in descending order of similarity and selects fragments from the 1st place to the Lth place. The "L" in the Lth place is the number of fragments estimated to be necessary and sufficient to obtain an appropriate answer to the question. In this embodiment, "L" is defined as "3".

[0074] The processor 11, having selected the sentence fragment similarity scores from the 1st place to the Lth place, proceeds to ACT36. As ACT36, the processor 11 extracts the text data of the sentence fragments from the L sentence fragment records 161, each of which has been identified as having a sentence fragment similarity score from the 1st place to the Lth place.

[0075] Processor 11, having extracted text data for L sentence fragments, proceeds to ACT37. Processor 11 creates an input sentence for the large-scale language model as ACT37. That is, using well-known prompt engineering, Processor 11 combines the text data of the question sentence taken in ACT22 with the text data of the L sentence fragments extracted in ACT36 to create an input sentence that the large-scale language model can interpret.

[0076] Processor 11, having created the input sentence for the large-scale language model, proceeds to ACT38. Processor 11 converts the input sentence into tokens as ACT38. That is, processor 11 uses a well-known tokenizer to break down the input sentence into words, i.e., tokens, that can be processed by the large-scale language model.

[0077] After converting the input sentence into tokens, the processor 11 proceeds to ACT39. As ACT39, the processor 11 inputs the tokens of the input sentence into the large-scale language model and waits for the large-scale language model to generate an answer. Once an answer is generated, the processor 11 proceeds to ACT40. As ACT40, the processor 11 outputs the text data of the document showing the answer to the user terminal 30. That is, the processor 11 sends the text data of the document showing the answer via the network interface 14 to the user terminal 30, which was the source of the question that was received in ACT21. With this, the processor 11 completes the information processing of the procedure based on the question answering program.

[0078] Here, processor 11 realizes the function of a question input unit 61 through processing of ACT21 and ACT22. Processor 11 realizes the function of a question text conversion unit 62 through processing of ACT23. Processor 11 realizes the function of a similarity calculation unit 63 through processing of ACT25 to ACT30. Processor 11 realizes the function of a text fragment similarity identification unit 64 through processing of ACT31 and ACT32. Processor 11 realizes the function of a text fragment extraction unit 65 through processing of ACT34 to ACT36. Processor 11 realizes the function of an answer generation unit 66 through processing of ACT37 to ACT39. Processor 11 realizes the function of an answer unit 67 through processing of ACT40.

[0079] Note that the processing procedure of the question answering program shown in the flowchart of Figure 8 is just one example. The order and content of the processing can be changed as appropriate, as long as similar effects can be achieved.

[0080] [Explanation of the effects and mechanisms of the question-answering system] As detailed above, when the question answering device 10 receives electronic data of a document from the administrator terminal 20, it divides the text of the document into sentence fragments of a predetermined number of characters or less. The question answering device 10 then generates up to five virtual question sentences for each sentence fragment, with the information in that sentence fragment serving as the answer. The question answering device 10 also converts each virtual question sentence into a virtual question sentence vector, which represents the meaning of that virtual question sentence as a high-dimensional numerical vector. Finally, for each sentence fragment, the question answering device 10 generates a sentence fragment record 161 containing the sentence fragment, the multiple virtual question sentences created from that sentence fragment, and the virtual question sentence vectors converted from each virtual question sentence, and stores it in the sentence fragment database 16.

[0081] Furthermore, when the question answering device 10 receives a question document from the user terminal 30, it converts the question into a question vector, which represents the meaning of the question as a high-dimensional numerical vector. The question answering device 10 then identifies the fragment similarity for all fragment records 161 stored in the fragment database 16. That is, the question answering device 10 calculates the similarity between each of the multiple virtual question vectors contained in the fragment record 161 and the question vector, and identifies the one with the highest similarity as the fragment similarity for that fragment record 161. Having identified the fragment similarity for each fragment record 161, the question answering device 10 ranks the fragment similarities of each fragment record in descending order, and extracts L fragments from the top L fragment records 161, from 1st to Lth. The question answering device 10 then inputs the extracted L fragments along with the question into a large-scale language model to generate an answer to the question. The question answering device 10 outputs text data indicating the answer from the network interface 14.

[0082] Text data transmitted from the network interface 14 of the question answering device 10 is transmitted over the communication network 40 and received by the user terminal 30, which is the source of the question. Upon receiving the text data, the user terminal can, for example, display the document of the text data on a display device. Alternatively, the text data can be converted into speech and output from a speaker. Thus, the user of the user terminal 30 can obtain an answer to their question.

[0083] Here, the input sentences fed into the large-scale language model consist of a question sentence and sentence fragments that generate hypothetical question sentences highly similar to the original question sentence. Therefore, there is no risk of inappropriate sentence fragments being mixed in with the input sentences to the large-scale language model, and thus, highly accurate answers can be obtained from the large-scale language model.

[0084] Thus, according to this embodiment, a question answering device 10 can be provided that can improve the accuracy of answers obtained by a large-scale language model. As a result, a question answering system 1 can be provided that generates answers to user questions in written form using a large-scale language model and provides those answers to the user in written form, resulting in a question answering system 1 with high accuracy.

[0085] <Second Embodiment> Next, a second embodiment, which is a modification of the first embodiment, will be described. The difference between the second embodiment and the first embodiment lies in the data structure of the document fragment record 161. Furthermore, due to this change in data structure, some of the first and second functions of the question answering device 10 have been modified. Other aspects are the same as in the first embodiment. Therefore, Figures 1 to 3 are applied directly to the second embodiment, and a detailed explanation is omitted. In the second embodiment, to distinguish it from the document fragment record 161 of the first embodiment, the document fragment record stored in the document fragment database 16 is referred to as document fragment record 162.

[0086] [Explanation of text fragment records] Figure 9 is a schematic diagram showing the data structure of a text fragment record 162. A text fragment record 162 is a data record that includes a record number (No.), a text fragment, a text fragment vector, multiple (five in the figure) virtual question sentences, and the same number of virtual question sentence vectors. In other words, in the second embodiment, a text fragment vector is added as data that constitutes the text fragment record 162. A text fragment vector is high-dimensional vector information that represents the meaning of a text fragment using a distributed representation.

[0087] [Description of the function of the question answering device] Figure 10 is a block diagram showing the configuration of the first function of the question answering device 10 in the second embodiment. In Figure 10, parts common to Figure 5, which describes the first function of the first embodiment, are denoted by the same reference numerals, and their descriptions are omitted. In other words, in the second embodiment, in addition to the document input unit 51, the division unit 52, the text fragment acquisition unit 53, the virtual question text creation unit 54, the virtual question text conversion unit 55, and the storage unit 56, the first function also includes a text fragment conversion unit 71.

[0088] The text fragment conversion unit 71 has the function of converting the text fragments acquired by the text fragment acquisition unit 53 into text fragment vectors. The text fragment conversion unit 71 converts the text fragments into text fragment vectors, in which the meaning of the text fragments is represented by a high-dimensional numerical vector. The text fragment conversion unit 71 converts the text fragments acquired by the text fragment acquisition unit 53 into text fragment vectors by utilizing a well-known technique called embedding.

[0089] Incidentally, in the second embodiment, the storage unit 56 creates a document fragment record 162 that includes, as one record, the document fragment acquired by the document fragment acquisition unit 53, the document fragment vector converted by the document fragment conversion unit 71, a plurality of virtual question sentences created for the document fragment by the virtual question sentence creation unit 54, and a plurality of virtual question sentence vectors converted by the virtual question sentence conversion unit 55 for each virtual question sentence, and stores it in the document fragment database 16. Alternatively, the storage unit 56 may create a document fragment record that includes the document fragment, the document fragment vector, and a virtual question sentence vector for each virtual question sentence, and store it in the document fragment database 16.

[0090] The text fragment conversion unit 71 can be rephrased as the text fragment conversion means 71.

[0091] Figure 11 is a block diagram showing the configuration of the second function of the question answering device 10 in the second embodiment. In Figure 11, parts common to Figure 6, which describes the second function of the first embodiment, are denoted by the same reference numerals, and their descriptions are omitted. In other words, in the second embodiment, in addition to the question input unit 61, question text conversion unit 62, text fragment extraction unit 65, answer generation unit 66, and answer unit 67, the second function also includes the functions of a first similarity calculation unit 81, a second similarity calculation unit 82, and a text fragment similarity identification unit 83.

[0092] The first similarity calculation unit 81 has the function of calculating the similarity between the question vector of the question text taken in via the question input unit 61 and the virtual question vector of the document fragment record 162 stored in the document fragment database 16. The similarity between the question vector and the virtual question vector is, for example, cosine similarity. The first similarity calculation unit 81 sequentially takes in the document fragment records 162 from the document fragment database 16. Then, for each of the multiple virtual question vectors contained in the document fragment record 162, the first similarity calculation unit 81 calculates the cosine value of the angle it makes with the question vector as the similarity. In this way, the first similarity calculation unit 81 has the same function as the similarity calculation unit 63 in the first embodiment. Therefore, the first similarity calculation unit 81 may be referred to as the similarity calculation unit 63.

[0093] The second similarity calculation unit 82 is a function that calculates the similarity between the question vector of the question text taken in via the question input unit 61 and the text fragment vector of the text fragment record 162 stored in the text fragment database 16. The similarity between the question vector and the text fragment vector is, for example, cosine similarity. The second similarity calculation unit 82 sequentially takes in the text fragment records 162 from the text fragment database 16. Then, for the text fragment vector contained in the text fragment record 161, the second similarity calculation unit 82 calculates the similarity as the cosine value of the angle it makes with the question vector.

[0094] The text fragment similarity identification unit 83 is a function that identifies the similarity between the text fragments contained in each text fragment record 162 stored in the text fragment database 16 and the question text. Specifically, the text fragment similarity identification unit 83 identifies the similarity of the text fragments contained in the text fragment record 162 as the sum of the similarity between the text fragment vector and the question text vector calculated by the second similarity calculation unit 82 and the maximum value of the similarity between the question text vector and the question text vector calculated for each virtual question text vector by the first similarity calculation unit 81. For example, if the similarity between the text fragment vector and the question text vector calculated by the second similarity calculation unit 82 is X, and the maximum value of the similarity between the question text vector and the question text vector calculated for each virtual question text vector by the first similarity calculation unit 81 is Y, the text fragment similarity identification unit 83 identifies the value K calculated by the following equation (1) as the similarity of the text fragments. K = X + Y * α …(1) Here, α is a coefficient that represents the ratio of similarity Y to similarity X, and is any value between 0 and α. In this embodiment, the coefficient α is set to 0.5. That is, the similarity of the text fragment is determined by adding the similarity X between the text fragment vector and the question vector to half the similarity Y between the virtual question vector and the question vector. Thus, the summation value is the sum of the similarity calculated by the second similarity calculation unit 82 and the similarity calculated by the first similarity calculation unit 81 at a predetermined ratio.

[0095] Furthermore, if you want to increase the ratio of similarity Y to similarity X, you can replace equation (1) above with equation (2) below.

[0096] K = X * α + Y …(2) Incidentally, the text fragment extraction unit 65 extracts a predetermined number L of text fragments in order from the text fragment record 161 with the highest similarity K identified by the text fragment similarity identification unit 83.

[0097] The first similarity calculation unit 81, the second similarity calculation unit 82, and the text fragment similarity identification unit 83 can be rephrased as the first similarity calculation means 81, the second similarity calculation means 82, and the text fragment similarity identification means 83.

[0098] [Description of the literature registration program] In the second embodiment, the functions of the first function, which consists of a document input unit 51, a division unit 52, a document fragment acquisition unit 53, a document fragment conversion unit 71, a virtual question creation unit 54, a virtual question conversion unit 55, and a storage unit 56, are realized by information processing performed by the processor 11 according to a second document registration program which is a modified version of the document registration program in the first embodiment.

[0099] Figure 12 is a flowchart showing the main steps of the information processing performed by the processor 11 of the question answering device 10 according to the second document registration program. Parts common to the information processing according to the document registration program of the first embodiment shown in Figure 7 are denoted by the same reference numerals, and their detailed explanations are omitted.

[0100] As can be seen by comparing Figure 7 and Figure 12, in the second embodiment, the processor 11 divides the text taken from the document into sentence fragments of a predetermined number of characters or less in ACT1 to ACT4, and then in ACT8, each time it selects one of the nth divided sentence fragments, it performs a vectorization process as ACT51. That is, the processor 11 converts the sentence fragments into sentence fragment vectors by representing the features of the sentence fragments with numerical vectors using an embedding engine.

[0101] After converting the text fragments into text fragment vectors, the processor 11 proceeds to ACT9. Thereafter, the processor 11 executes the processing of ACT9 to ACT17 in the same manner as in the first embodiment. That is, the processor 11 creates a number of virtual question sentences equal to the set value P from the nth text fragment and converts each virtual question sentence into a virtual question sentence vector. After that, if a record number z is issued, the processor 11 creates a text fragment record 162. That is, the processor 11 creates a text fragment record 162 that includes the record number z issued in ACT15, the text data of the text fragment selected in ACT8, the text fragment vector converted in ACT51, and the P virtual question sentences and virtual question sentence vectors stored in the virtual question sentence memory in ACT14. After creating the text fragment record 162, the processor 11 saves the text fragment record 161 to the text fragment database 16. Then the processor 11 returns to ACT6.

[0102] Here, the processor 11 realizes its function as a text fragment conversion unit 71 through the processing of ACT 51.

[0103] [Explanation of the Question Answering Program] In the second embodiment, the functions of the second function, which is the question input unit 61, the question text conversion unit 62, the first similarity calculation unit 81, the second similarity calculation unit 82, the text fragment similarity identification unit 83, the text fragment extraction unit 65, the answer generation unit 66, and the answer unit 67, are realized by information processing executed by the processor 11 according to a second question answer program which is a modified version of the question answer program in the first embodiment.

[0104] Figure 13 is a flowchart showing the main steps of the information processing performed by the processor 11 of the question answering device 10 according to the second question answering program. Parts common to the information processing according to the question answering program of the first embodiment shown in Figure 8 are denoted by the same reference numerals, and their detailed explanations are omitted.

[0105] As can be seen by comparing Figure 8 and Figure 13, in the second embodiment, the processing of ACT30 to ACT33 in the first embodiment is replaced with the processing of ACT61 to ACT65. That is, in ACT29, the processor 11, which has obtained the document fragment record 161 with record number r from the document fragment database 16, proceeds to ACT61. The processor 11 calculates the second similarity X as ACT61. The second similarity X is the similarity between the document fragment vector contained in the document fragment record 161 with record number r and the question vector stored in the question memory. That is, the processor 11 individually compares the multidimensional vector information that constitutes the document fragment vector and the same-dimensional vector information that constitutes the question vector, and calculates the similarity of the vector information, for example, the cosine similarity, as the second similarity X.

[0106] The processor 11, having calculated the second similarity X, proceeds to ACT62. The processor 11 calculates the first similarity y as ACT62. The first similarity y is the similarity between the virtual question vector contained in the text fragment record 161 of record number r and the question vector stored in the question memory. That is, for each virtual question vector, the processor 11 individually compares the multidimensional vector information constituting the virtual question vector with the same-dimensional vector information constituting the question vector, and calculates the similarity of the vector information, for example, the cosine similarity, as the first similarity y for each virtual question vector.

[0107] After calculating the first similarity y for all virtual question vectors contained in the text fragment record 161, the processor 11 proceeds to ACT63. In ACT63, the processor 11 compares the first similarity y for each virtual question vector and selects the highest first similarity Y.

[0108] The processor 11, having selected the highest first similarity Y, proceeds to ACT64. The processor 11 uses the second similarity X, the highest first similarity Y, and the coefficient α as ACT64 to calculate the sentence fragment similarity K, for example, using equation (1) described above.

[0109] The processor 11, having calculated the text fragment similarity K, proceeds to ACT65. As ACT65, the processor 11 associates the record number r with the text fragment similarity K for the text fragment record 161 of record number r and stores it in table memory.

[0110] The processor 11, having stored the record number r and the text fragment similarity K in table memory, returns to ACT27. The processor 11 then increments counter r by another "1". Thereafter, the processor 11 repeats the processing of ACT29 and ACT61 to ACT65 until the value of counter r exceeds the value of register R. When the value of counter r exceeds the value of register R, the processor 11 executes the processing of ACT34 to ACT40, as in the first embodiment.

[0111] Here, processor 11 realizes the function of a first similarity calculation unit 81 through the processing of ACT62. Processor 11 realizes the function of a second similarity calculation unit 82 through the processing of ACT61. Processor 11 realizes the function of a text fragment similarity identification unit 83 through the processing of ACT63 and ACT64.

[0112] In addition, in the processing procedure shown in the flowchart of Figure 13, it is also possible to execute the processes of ACT62 and ACT63 after the process of ACT29, and then execute the process of ACT61.

[0113] [Explanation of the effects and mechanisms of the question-answering system] In the second embodiment as well, the input sentences fed into the large-scale language model consist of a question sentence and sentence fragments that generate hypothetical question sentences highly similar to the question sentence. Therefore, there is no risk of inappropriate sentence fragments being mixed in as input to the large-scale language model, and thus the accuracy of the answers obtained by the large-scale language model is high.

[0114] Furthermore, in the second embodiment, when determining the similarity of text fragments, the similarity between the question and the text fragment is also considered. Therefore, the stability of the answer accuracy can be increased compared to the first embodiment, in which text fragments were determined solely by the similarity between the question and the hypothetical question.

[0115] <Third Embodiment> Next, a third embodiment, which is a modification of the second embodiment, will be described. The third embodiment differs from the second embodiment in part of the second function of the question answering device 10. In all other respects, it is the same as the second embodiment. Therefore, Figures 1 to 3, 9, 10, and 12 are applied directly to the third embodiment, and a detailed explanation is omitted.

[0116] [Explanation of the second function of the question answering device] Figure 14 is a block diagram showing the configuration of the second function of the question answering device 10 in the third embodiment. In Figure 14, parts common to Figure 11, which describes the second function of the second embodiment, are denoted by the same reference numerals, and their descriptions are omitted. In other words, in the third embodiment, in addition to the question input unit 61, question text conversion unit 62, second similarity calculation unit 82, text fragment extraction unit 65, answer generation unit 66, and answer unit 67, the second function also includes the functions of a second similarity comparison unit 91, record extraction unit 92, first similarity calculation unit 93, and text fragment similarity identification unit 94.

[0117] The second similarity comparison unit 91 has the function of comparing the second similarity X calculated by the second similarity calculation unit 82 for each text fragment record 162 stored in the text fragment database 16. As described in the second embodiment, the second similarity calculation unit 82 calculates the similarity between the question text vector and the text fragment vector as the second similarity X for each text fragment record 162. The second similarity comparison unit 91 compares the magnitude of the second similarity X calculated for each text fragment record 162.

[0118] The record extraction unit 92 has the function of extracting a certain number Q of text fragment records 162 from the text fragment database 16 in descending order of the second similarity X compared by the second similarity comparison unit 91. The certain number Q is an arbitrary value that is greater than the number L used when selecting text fragments in descending order of similarity in the ACT35 process described above, and less than the number of records stored in the text fragment database 16. For example, the certain number Q is half the number of records stored in the text fragment database 16. The record extraction unit 92 extracts a certain number Q of text fragment records 162 from the text fragment database 16 in descending order of the second similarity X.

[0119] The first similarity calculation unit 93 has the function of calculating a first similarity y for each of the fixed number Q text fragment records 162 extracted by the record extraction unit 92. The first similarity y is, for example, the cosine similarity between the question vector and the virtual question vector. For each of the fixed number Q text fragment records 162, the first similarity calculation unit 93 calculates the cosine value of the angle that each of the multiple virtual question vectors contained in that text fragment record 162 makes with the question vector as the first similarity y.

[0120] The text fragment similarity identification unit 94 is a function that identifies the similarity between the text fragments contained in each text fragment record 162 extracted by the record extraction unit 92 and the question text. For each of the text fragment records 162 extracted by the record extraction unit 92, the text fragment similarity identification unit 94 identifies the similarity between the text fragments and the question text based on the first similarity calculation unit 93 which calculates the first similarity y for each of the text fragment records 162. Specifically, the text fragment similarity identification unit 94 identifies the maximum value of the first similarity y as the similarity between the text fragments and the question text.

[0121] The second similarity comparison unit 91, the record extraction unit 92, the first similarity calculation unit 93, and the text fragment similarity identification unit 94 can be rephrased as the second similarity comparison means 91, the record extraction means 92, the first similarity calculation means 93, and the text fragment similarity identification means 94.

[0122] [Explanation of the Question Answering Program] In the second embodiment, the functions of the second function, which is the question input unit 61, question text conversion unit 62, second similarity calculation unit 82, second similarity comparison unit 91, record extraction unit 92, first similarity calculation unit 93, text fragment similarity identification unit 94, text fragment extraction unit 65, answer generation unit 66, and answer unit 67, are realized by information processing executed by the processor 11 according to a third question answer program which is a modified version of the question answer program in the second embodiment.

[0123] Figures 15 and 16 are flowcharts showing the main steps of the information processing performed by the processor 11 of the question answering device 10 according to the third question answering program. Parts common to the information processing according to the second question answering program in the second embodiment shown in Figure 13 are denoted by the same reference numerals, and their detailed explanations are omitted.

[0124] As can be seen by comparing Figure 13 and Figure 15, in the third embodiment, the processing of ACT62 to ACT65 in the second embodiment is replaced with the processing of ACT71. That is, in ACT29, the processor 11 retrieves the document fragment record 161 with record number r from the document fragment database 16, and in ACT61, the processor 11 calculates the second similarity X from the document fragment record 161, and then proceeds to ACT71. As ACT71, the processor 11 stores the record number r and the second similarity X for the document fragment record 161 with record number r in table memory, associating them.

[0125] In the third embodiment, when the value of counter r exceeds the value of register R in ACT28, the processor 11 proceeds to ACT72 in Figure 16. As ACT72, the processor 11 sets half the number of records of all document fragment records 161 stored in the document fragment database 16 to register Q. Then, as ACT73, the processor 11 compares the second similarity X for each document fragment record 161 stored in table memory. As ACT74, the processor 11 ranks the second similarity X in descending order and selects the second similarity X from the 1st place to the Qth place.

[0126] Incidentally, the "Q" in the Qth position is the value set in register Q during the processing of ACT72. The value Q is any value that is greater than the number L used in ACT35 to select sentence fragments in descending order of similarity, and less than the number of records stored in the sentence fragment database 16. In this embodiment, the value Q is defined as half the number of records stored in the sentence fragment database 16.

[0127] Processor 11, having selected the second similarity X from the first-ranked second similarity X to the Qth-ranked second similarity X, proceeds to ACT75. Processor 11 resets counter q to "0" as ACT75. Next, processor 11 increments counter q by "1" as ACT76. Then, as ACT77, processor 11 checks whether the value of counter q has exceeded the value of register Q.

[0128] If the value of counter q does not exceed the value of register Q, processor 11 proceeds to ACT78. As ACT78, processor 11 retrieves the record number r from table memory, which is the record number set as the rank in the ranking of the second similarity X, based on the value of counter q. Then, as ACT79, processor 11 retrieves the document fragment record 162 from the document fragment database 16, which is the document fragment record with the record number r set.

[0129] When processor 11 obtains a text fragment record 162 with a rank of q in the second similarity X, it proceeds to ACT 80. As ACT 80, processor 11 individually calculates the similarity, i.e., the first similarity y, for all virtual question vectors contained in the text fragment record 162 with the question vectors stored in the question memory.

[0130] After calculating the first similarity y for all the virtual question vectors contained in the text fragment record 162, the processor 11 proceeds to ACT81. In ACT81, the processor 11 compares the first similarity y calculated for each virtual question vector and selects the largest first similarity Y. Then, in ACT82, the processor 11 sets this largest similarity Y as the text fragment similarity for the text fragment record 162 whose second similarity X rank is q.

[0131] The processor 11, having set the sentence fragment similarity, proceeds to ACT83. As ACT83, the processor 11 stores the sentence fragment similarity Y in table memory, associating it with the record number r of sentence fragment record 162, which has a rank of q in the second similarity X.

[0132] After storing the sentence fragment similarity in table memory, processor 11 returns to ACT76. Processor 11 increments counter q by another "1". After confirming that the value of counter q does not exceed the value of register Q, processor 11 executes the processes of ACT78 to ACT83 in the same manner as described above. That is, processor 11 sequentially retrieves sentence fragment records 162 with a rank of q in the second similarity X from the sentence fragment database 16, and calculates the first similarity y for all virtual question vectors contained in each sentence fragment record 161. Processor 11 then selects the maximum similarity Y of the first similarity y, identifies this maximum similarity Y as the sentence fragment similarity for sentence fragment record 162 with a rank of q in the second similarity X, and stores the sentence fragment similarity Y in table memory along with the record number r.

[0133] In ACT77, once it is confirmed that counter q exceeds the value of register Q, the processor 11 proceeds to ACT34. Thereafter, the processor 11 executes the processing of ACT34 to ACT4, similar to the first and second embodiments.

[0134] Here, processor 11 realizes its function as a second similarity comparison unit through the processing of ACT73. Processor 11 realizes its function as a record extraction unit 92 through the processing of ACT74 to ACT79. Processor 11 realizes its function as a first similarity calculation unit 93 through the processing of ACT80. Processor 11 realizes its function as a text fragment similarity identification unit 94 through the processing of ACT81 and ACT82.

[0135] [Explanation of the effects and mechanisms of the question-answering system] In the third embodiment as well, the input sentences fed into the large-scale language model consist of a question sentence and sentence fragments that generate hypothetical question sentences highly similar to the question sentence. Therefore, there is no risk of inappropriate sentence fragments being mixed in as input to the large-scale language model, and thus the accuracy of the answers obtained by the large-scale language model is high.

[0136] Furthermore, in the second embodiment, the first similarity y was calculated for all text fragment records 162 stored in the text fragment database 16, but in the third embodiment, the first similarity y is calculated for text fragment records 162 up to the top Q rank of the second similarity X. Therefore, if, for example, "Q" is defined as half the number of text fragment records 162 stored in the text fragment database 16, the number of text fragment records 162 for which the first similarity y is calculated is halved compared to the second embodiment. As a result, the load required for information processing performed by the processor 11 according to the question answering program is significantly reduced compared to the second embodiment, and processing time can be shortened.

[0137] <Other Embodiments> The above describes an embodiment of a question answering device that can improve the accuracy of answers obtained from a large-scale language model, but the embodiment is not limited to this.

[0138] For example, in the first embodiment, the sentence fragment similarity identification unit 64 identified the maximum value of the similarity with the question vector calculated for each virtual question vector as the similarity between the sentence fragment and the question. In this regard, for example, the average value of the similarity with the question vector calculated for each virtual question vector may be identified as the similarity between the sentence fragment and the question.

[0139] Furthermore, in each of the above embodiments, the question answering device 10 uses a portion of the storage area of ​​the auxiliary storage device 13 as the document fragment database 16. In another embodiment, a database server for managing the document fragment database 16 is connected to the communication network 40. The question answering device 10 may then be configured to access the database server via the communication network 40 and search the document fragment database 16. In this case, the functions of the document input unit 51, the division unit 52, and the storage unit 56 may be provided by the database server.

[0140] In addition, several embodiments of the present invention have been described, but these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be carried out in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their variations are included within the scope of the invention, as well as within the scope of the invention and its equivalents as described in the claims. [Explanation of Symbols]

[0141] 1...Question answering system, 10...Question answering device, 11...Processor, 12...Main memory, 13...Auxiliary storage device, 14...Network interface, 15...System transmission path, 16...Document fragment database, 20...Administrator terminal, 30...User terminal, 40...Communication network, 51...Document input unit, 52...Splitting unit, 53...Document fragment acquisition unit, 54...Virtual question text creation unit, 55...Virtual question text conversion unit, 56...Storage unit 61...Question input unit, 62...Question text conversion unit, 63...Similarity calculation unit, 64...Text fragment similarity identification unit, 65...Text fragment extraction unit, 66...Answer generation unit, 67...Answer unit, 71...Text fragment conversion unit, 81...First similarity calculation unit, 82...Second similarity calculation unit, 83...Text fragment similarity identification unit, 91...Second similarity comparison unit, 92...Record extraction unit, 93...First similarity calculation unit, 94...Text fragment similarity identification unit, 161,162...Text fragment records.

Claims

1. A virtual question generation unit that creates virtual question sentences from text fragments, A storage unit that stores the aforementioned text fragment and the virtual question vector, which represents the virtual question created from the text fragment using distributed representations, as one record in a database. A similarity calculation unit calculates the similarity between a virtual question vector contained in a record stored in the database and a question vector representing the input question as a distributed representation. A text fragment extraction unit extracts a predetermined number of text fragments from the database based on the similarity calculated for each record by the similarity calculation unit, An answer generation unit that generates an answer to the question based on the question and the predetermined number of text fragments extracted by the text fragment extraction unit, A question answering device equipped with the following:

2. The aforementioned virtual question generation unit generates multiple virtual question sentences from the text fragments, The storage unit stores the text fragment and the virtual question vector for each of the multiple virtual question sentences created from the text fragment as one record in the database. The similarity calculation unit calculates, for each record stored in the database, the similarity between the multiple virtual question vectors contained in the record and the question vector representing the input question in a distributed representation, for each virtual question vector. The question answering device according to claim 1, wherein the text fragment extraction unit identifies the highest similarity among the similarities calculated for each virtual question vector by the similarity calculation unit for each record stored in the database as the similarity of the text fragment contained in that record, and extracts a predetermined number of text fragments in order from the records with the highest similarity of the text fragments.

3. The records stored in the database include the text fragments and the virtual question vectors, as well as a text fragment vector representing the text fragments in a distributed representation. A second similarity calculation unit calculates the similarity between each record stored in the database and the question vector, which represents the input question as a distributed representation. Furthermore, it is equipped with, The question answering device according to claim 1, wherein the text fragment extraction unit extracts a predetermined number of text fragments from the database that are included in a predetermined number of records, based on the similarity calculated by the second similarity calculation unit and the similarity calculated by the first similarity calculation unit.

4. The question answering device according to claim 3, wherein the text fragment extraction unit identifies the sum of the similarity calculated by the second similarity calculation unit and the similarity calculated by the first similarity calculation unit as the similarity of the text fragments contained in each record stored in the database, and extracts a predetermined number of text fragments in order from the records with the highest similarity of the text fragments.

5. The records stored in the database include the text fragments and the virtual question vectors, as well as a text fragment vector representing the text fragments in a distributed representation. A second similarity calculation unit calculates the similarity between each record stored in the aforementioned database and the sentence vector containing the sentence in that record, and the question sentence vector representing the input question sentence as a distributed representation. A record extraction unit extracts a second predetermined number of records, greater than the predetermined number, from the records stored in the database, in descending order of similarity calculated by the second similarity calculation unit. Furthermore, it is equipped with, The similarity calculation unit calculates, for each of the second predetermined number of records extracted by the record extraction unit, the similarity between the virtual question vector contained in the record and the question vector representing the input question in distributed representation. The question answering device according to claim 1, wherein the text fragment extraction unit identifies the highest similarity among the similarities calculated for each virtual question vector by the similarity calculation unit as the similarity of the text fragment contained in the record for each predetermined number of second records, and extracts a predetermined number of text fragments in order from the records with the highest similarity of the text fragments.

6. The computer of the question answering device, A means for creating a hypothetical question from a text fragment, A storage means that stores the aforementioned text fragment and the virtual question vector, which represents the virtual question created from the text fragment using distributed representation, as one record in a database. Similarity calculation means calculates the similarity between a virtual question vector contained in a record stored in the database and a question vector representing the input question as a distributed representation. A text fragment extraction means that extracts a predetermined number of text fragments from the database based on the similarity calculated for each record by the similarity calculation means, and Answer generation means that generates an answer to the question based on the question and the predetermined number of text fragments extracted by the text fragment extraction means. A program designed to function as such.

Images