Question answering apparatus and program thereof

Abstract

To improve the accuracy of an answer obtained by a large-scale language model by preventing a sentence piece extracted from an inappropriate document from being mixed with a question.SOLUTION: The database unit stores, as one record, a sentence piece extracted from each of a plurality of documents, a sentence piece feature vector representing a distributed representation of the sentence piece, and document identification information for identifying the document from which the sentence piece is extracted. The similarity calculation unit calculates a similarity with the question sentence for each sentence piece by a question sentence feature vector representing a distributed representation of the input question sentence and a sentence piece feature vector of each record stored in the database unit. A document specification part specifies a document related to the question sentence from among the plurality of documents on the basis of the similarity calculated for each sentence piece. The sentence piece extraction unit extracts at least one sentence piece from the specified document in descending order of similarity. The answer generation unit generates an answer to the question sentence on the basis of the question sentence and the extracted sentence piece.SELECTED DRAWING: Figure 7

Description

[Technical Field]

[0001] An embodiment of the present invention relates to a question and answering device and a program therefor. [Background technology]

[0002] In recent years, question-answering systems have been developed that use large-scale language models (LLMs) to generate answers to users' written questions and provide written answers to them. This type of question-answering system uses a technology called Retrieval Augmenter Generation (RAG). Retrieval Augmenter Generation is a technology that divides the text of a document that is the subject of a question into multiple fragments and stores them in a database, then extracts one or more fragments that are similar to the question document from the database. The extracted fragments are then input into the large-scale language model along with the question document. The large-scale language model utilizes the fragments extracted by Retrieval Augmenter Generation to generate an answer to the question document using knowledge gained from learning from a huge amount of language data, and outputs the answer in written form.

[0003] In this way, by using the expanded search generation technique, the large-scale language model can obtain an answer by referring not only to the question document but also to sentence fragments similar to the question, which makes it more likely to obtain an appropriate answer.

[0004] However, the databases searched in the expanded search generation technology store data on sentence fragments segmented from a large number of documents. Therefore, there is a possibility that sentence fragments segmented from inappropriate documents other than the document targeted by the query may be extracted as sentence fragments similar to the query document. If sentence fragments extracted from inappropriate documents are mixed into the sentence fragments input to the large-scale language model, there is a concern that the accuracy of the answers obtained by the large-scale language model may be degraded. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Publication No. 2023-076413 Summary of the Invention [Problem to be solved by the invention]

[0006] The problem that the embodiments of the present invention aim to solve is to provide a question answering device that can improve the accuracy of answers obtained using a large-scale language model by preventing the inclusion of sentence fragments extracted from documents that are inappropriate for the question. [Means for solving the problem]

[0007] In one embodiment, the question and answering device includes a database unit, a similarity calculation unit, a document identification unit, a sentence fragment extraction unit, and an answer generation unit. The database unit stores, as one record, sentence fragments extracted from a plurality of documents, sentence fragment feature vectors representing the distributed representations of the sentence fragments, and document identification information identifying the documents from which the sentence fragments were extracted. The similarity calculation unit calculates a similarity between each sentence fragment and the question using the question feature vector representing the distributed representation of the input question and the sentence fragment feature vector of each record stored in the database unit. The document identification unit identifies a document related to the question from the plurality of documents based on the similarity calculated for each sentence fragment by the similarity calculation unit. The sentence fragment extraction unit extracts at least one sentence fragment from the document identified by the document identification unit in descending order of the similarity calculated by the similarity calculation unit. The answer generation unit generates an answer to the question based on the question and the sentence fragments extracted by the sentence fragment extraction unit. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a schematic diagram showing a schematic configuration of a question and answer system according to an embodiment. [Figure 2] FIG. 2 is a block diagram showing the main circuit configuration of a question and answering device included in the question and answering system. [Figure 3]FIG. 3 is a schematic diagram showing the data structure of a first table in the question and answering device. [Figure 4] FIG. 4 is a schematic diagram showing the data structure of a second table in the question and answering device. [Figure 5] FIG. 5 is an explanatory diagram of a document database that the question and answering device has. [Figure 6] FIG. 6 is a schematic diagram showing the data structure of a sentence fragment record stored in a document database. [Figure 7] FIG. 7 is a block diagram showing the main functional configuration of the question and answering device. [Figure 8] FIG. 8 is a flowchart showing the main steps of information processing executed by the processor of the question and answering device in accordance with the document registration program. [Figure 9] FIG. 9 is a flowchart showing the main steps of information processing executed by the processor of the question and answering device in accordance with the question and answering program. [Figure 10] FIG. 10 is a flowchart showing the main steps of information processing executed by the processor of the question and answering device in accordance with the question and answering program. [Figure 11] FIG. 11 is a flowchart showing the main steps of information processing executed by the processor of the question and answering device in accordance with the question and answering program in the second embodiment. [Figure 12] FIG. 12 is a flowchart showing the main steps of information processing executed by the processor of the question and answering device according to the question and answering program in the third embodiment. [Figure 13] FIG. 13 is a flowchart showing the main steps of information processing executed by the processor of the question and answering device in accordance with the question and answering program in the fourth embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] Below, we will explain, with reference to the drawings, several embodiments of a question answering device that can improve the accuracy of answers obtained using a large-scale language model by preventing the inclusion of sentence fragments extracted from documents that are inappropriate for the question.

[0010] First Embodiment [Outline of the Q&A system] First, a question and answering system including a question and answering device will be described. 1 is a schematic diagram showing a general configuration of a question and answering system 1 according to an embodiment. The question and answering system 1 includes a question and answering device 10, an administrator terminal 20, and a plurality of user terminals 30. The question and answering system 1 connects the question and answering device 10, the administrator terminal 20, and the plurality of user terminals 30 via a communication network 40. The communication network 40 is a wide area network such as the Internet or an intranet. A mobile communication network, a public communication network, or the like may also be used as part of the communication network 40.

[0011] The question answering system 1 is a system that uses a large-scale language model (LLM) to create answers to written questions from users and provides written answers to the users. The large-scale language model generates answers to questions using a technique called Augmented Query Generation (RAG). Hereinafter, this technique called Augmented Query Generation will be referred to as the RAG technique.

[0012] The user terminal 30 is a computer terminal used by a user who obtains an answer to a question using the question and answering system 1. When a user inputs a question in text using the user terminal 30, a text answer to the question is output from the question and answering device 10 to the user terminal 30. The user terminal 30 displays the text answer on a display device. Alternatively, the user terminal 30 outputs a voice reading the text answer from a speaker. In this way, the user can obtain the answer to the question. In this way, the user terminal 30 can be a personal computer, tablet terminal, smartphone, or the like equipped with an input device that can input a question in text and a display device or speaker that can output a text answer.

[0013] Administrator terminal 20 is a computer terminal used by an administrator of question and answer system 1. Using administrator terminal 20, the administrator inputs so-called documents, i.e., electronic data of various documents, such as documents, materials, records, and books, that can serve as reference materials for various questions. Examples of documents include PDF files with the extension "pdf," Word files with the extension "docx" or "doc," PowerPoint files with the extension "ppt," and text files with the extension "txt." A personal computer, tablet device, smartphone, or the like capable of inputting such documents can serve as administrator terminal 20. Needless to say, the types of documents are not limited to PDF files, Word files, PowerPoint files, and text files with the above-mentioned extensions.

[0014] The question answering device 10 is a server computer that provides a question and answer service with a user terminal 30 as a client. That is, when the question and answering device 10 receives a question from the user terminal 30, which is the client, it generates an answer to the question and outputs the answer to the user terminal 30. To provide this service, the question and answering device 10 uses a large-scale language model (LLM) that uses the RAG method. A large-scale language model is a language model constructed using large amounts of text data and deep learning technology. The question and answering device 10 uses the large-scale language model to generate an answer to a question input in text form from the user terminal 30 and outputs the answer in text form to the user terminal 30.

[0015] [Explanation of the configuration of the question answering device] 2 is a block diagram showing the main circuit configuration of the question and answering device 10. The question and answering device 10 includes a processor 11, a main memory 12, an auxiliary storage device 13, a network interface 14, and a system transmission path 15. The system transmission path 15 includes an address bus, a data bus, control signal lines, etc. The system transmission path 15 connects the processor 11 to each of the other components and transmits data signals exchanged between them. The question and answering device 10 constitutes a computer by connecting the processor 11 with the main memory 12, the auxiliary storage device 13, and the network interface 14 via the system transmission path 15.

[0016] The processor 11 corresponds to the central part of the computer. The processor 11 controls each part to realize various functions of the question and answer device 10 in accordance with an operating system or an application program. The processor 11 is, for example, a CPU (Central Processing Unit). The processor 11 may be, for example, an MPU (Micro Processing Unit), an SoC (System on a Chip), a DSP (Digital Signal Processor), a GPU (Graphics Processing Unit), an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), or an FPGA (Field-Programmable Gate Array). Alternatively, the processor 11 may be a combination of two or more of these.

[0017] The main memory 12 corresponds to the main storage portion of the computer. The main memory 12 includes a nonvolatile memory area and a volatile memory area. The main memory 12 stores an operating system or application programs in the nonvolatile memory area. The main memory 12 may store data required for the processor 11 to execute processes for controlling each part in either the nonvolatile or volatile memory area. The main memory 12 uses the volatile memory area as a work area where data is rewritten by the processor 11 as appropriate. The nonvolatile memory area is, for example, ROM (Read Only Memory). The volatile memory area is, for example, RAM (Random Access Memory).

[0018] The auxiliary storage device 13 corresponds to the auxiliary storage portion of the computer. For example, the auxiliary storage device 13 may be an EEPROM (registered trademark) (Electric Erasable Programmable Read-Only Memory), an HDD (Hard Disc Drive), or an SSD (Solid State Drive). The auxiliary storage device 13 stores data used by the processor 11 when performing various processes, data created by the processes in the processor 11, etc. The auxiliary storage device 13 may also store the application programs described above.

[0019] The network interface 14 connects to the communication network 40. The question and answering device 10 performs data communication with the administrator terminal 20 and the user terminal 30 connected to the communication network 40 via the network interface 14 in accordance with a preset communication protocol.

[0020] In the question and answering device 10 configured as above, part of the storage area of ​​the main memory 12 is used as an area for a first table 121 and an area for a second table 122. As shown in Fig. 3, the first table 121 is a data table having a column Ca for table numbers, a column Cb for document IDs, and a column Cc for similarities. As shown in Fig. 4, the second table 122 is a data table having a column Cd for table numbers, a column Ce for document IDs, and a column Cf for average values. The document IDs, similarities, and average values, as well as the uses of the first table 121 and the second table 122, will be explained in detail later.

[0021] Furthermore, question and answering device 10 uses part of the storage area of ​​auxiliary storage device 13 as a literature database 16. Literature database 16 is a storage area prepared for storing data relating to various literature required in the RAG method.

[0022] Fig. 5 is an explanatory diagram of the document database 16. As shown in Fig. 5, the document database 16 stores a plurality of n sentence fragment records 161 obtained from a plurality of documents 100, 200, 300, .... In Fig. 5, [100-1] to [100-n] indicate sentence fragment records 161 obtained from document 100, [200-1] to [200-n] indicate sentence fragment records 161 obtained from document 200, and [300-1] to [300-n] indicate sentence fragment records 161 obtained from document 300. Note that the number n of sentence fragment records 161 obtained from one document is an arbitrary value determined depending on the length and content of the sentences described in the document, and is not uniform.

[0023] FIG. 6 is a schematic diagram showing the data structure of one sentence fragment record 161. The sentence fragment record 161 is a data record that describes a document ID, a sentence fragment, and a sentence fragment feature vector in association with a series of record numbers. The document ID is unique identification information assigned to each document to individually identify each document 100, 200, 300, .... The document ID is an example of document identification information. A sentence fragment is text data extracted as a sentence of a predetermined number of characters or less from a document identified by a document ID. A sentence fragment is text data of a sentence of, for example, 500 characters or less, separated by punctuation or line breaks. The sentence fragment feature vector is multidimensional vector information that represents the individual features of the words that make up the sentence fragment using distributed representations. Here, the document database 16 constitutes a database unit that stores, as one record, sentence fragments extracted from multiple documents, sentence fragment feature vectors that represent the distributed representations of the sentence fragments, and document identification information (document IDs) that identify the documents from which the sentence fragments were extracted.

[0024] [Explanation of the question answering device's functions] 7 is a block diagram showing the main functional configuration of question and answering device 10. Question and answering device 10 has functions as a document input unit 51, an issuing unit 52, a dividing unit 53, a sentence fragment conversion unit 54, and a saving unit 55, as well as functions as a question input unit 61, a question sentence conversion unit 62, a similarity calculation unit 63, a document identification unit 64, a sentence fragment extraction unit 65, an answer generation unit 66, and an answering unit 67.

[0025] The functions of the document input unit 51, the issuing unit 52, the dividing unit 53, the sentence fragment conversion unit 54, and the saving unit 55 are functions associated with the administrator terminal 20. That is, the document input unit 51 has a function of importing text from documents input via the administrator terminal 20, that is, from electronic data of various documents. When electronic data of a document consisting of a PDF file, a Word file, a PowerPoint file, a text file, or the like is input by an administrator operating the administrator terminal 20, the electronic data is transmitted to the question and answering device 10 via the communication network 40. When the document input unit 51 receives the electronic data of a document via the network interface 14, it imports text data from the electronic data, that is, the text of the document.

[0026] The issuing unit 52 has a function of issuing a document ID as document identification information that identifies a document each time a document sentence is acquired in the document input unit 51. The issuing unit 52 issues, for example, a serial number that is counted up each time a document sentence is acquired as the document ID. Note that the document ID is not limited to the serial number described above, and may be any information that can uniquely identify a document.

[0027] The dividing unit 53 has a function of dividing the text of a document imported via the document input unit 51 into text fragments of a predetermined number of characters or less. For example, the dividing unit 53 divides the text into text fragments of 500 characters or less. The number of characters in a text fragment does not necessarily have to be close to the predetermined number of characters. The dividing unit 53 divides the text at convenient points such as punctuation marks and line breaks within the predetermined number of characters. The divided text fragments may partially overlap with other text fragments. This function of the dividing unit 53 is generally called chunk division.

[0028] The sentence fragment conversion unit 54 has a function of converting each sentence fragment divided by the division unit 53 into a sentence fragment feature vector. The sentence fragment conversion unit 54 converts a sentence fragment into a sentence fragment feature vector by using a well-known embedding engine to represent the individual features of the words that make up the sentence fragment as a numerical vector.

[0029] The storage unit 55 has a function of storing sentence fragment records 161 in the document database 16. That is, for each sentence fragment divided by the division unit 53, the storage unit 55 creates a sentence fragment record 161 using the text data of the sentence fragment, the document ID issued by the publication unit 52, and a sentence fragment feature vector obtained by converting the sentence fragment by the sentence fragment conversion unit 54. The storage unit 55 then stores the sentence fragment record 161 in the document database 16.

[0030] The functions of the question input unit 61, question sentence conversion unit 62, similarity calculation unit 63, document identification unit 64, sentence fragment extraction unit 65, answer generation unit 66, and answer unit 67 are functions associated with the user terminal 30. That is, the question input unit 61 has a function of importing a question input in text form via the user terminal 30. When a question is input in text form by a user's operation on the user terminal 30, the text data is transmitted to the question and answering device 10 via the communication network 40. When the question input unit 61 receives the text data of the question via the network interface 14, it imports the text data as a question sentence.

[0031] The question sentence conversion unit 62 has a function of converting a question sentence input via the question input unit 61 into a question sentence feature vector. The question sentence conversion unit 62 converts a question sentence into a question sentence feature vector by using a well-known embedding engine to represent the individual features of the words that make up the question sentence as a numerical vector. Here, the embedding engine used by the question sentence conversion unit 62 and the embedding engine used by the sentence fragment conversion unit 54 are the same. Therefore, the question sentence conversion vector converted by the question sentence conversion unit 62 and the sentence fragment conversion vector converted by the sentence fragment conversion unit 54 are multidimensional vector information that represents a common distributed representation.

[0032] The similarity calculation unit 63 has a function of calculating the similarity between the question sentence and each sentence fragment using the question sentence feature vector of the question sentence imported via the question input unit 61 and the sentence fragment feature vectors of all sentence fragment records 161 stored in the literature database 16. That is, the similarity calculation unit 63 calculates the similarity of the vector information by individually comparing the multidimensional vector information constituting the question sentence feature vector with the same-dimensional vector information constituting the sentence fragment feature vector. The smaller the difference between the vector information being compared, the larger the similarity value. In other words, the larger the similarity value, the more similar the question sentence represented by the question sentence feature vector and the sentence fragment represented by the sentence fragment feature vector are.

[0033] The document identification unit 64 is a function that identifies a document related to the question from among multiple documents based on the similarity calculated for each sentence fragment by the similarity calculation unit 63. Specifically, for each of the multiple documents, the document identification unit 64 selects a predetermined number of similarities in descending order from the similarities calculated by the similarity calculation unit 63 for each sentence fragment extracted from the document, calculates their average, and identifies the document with the largest average as the document related to the question. For example, for each of the multiple documents, the document identification unit 64 selects five similarities in descending order from the similarities calculated for each sentence fragment extracted from the document, calculates their average, and identifies the document with the largest average as the document related to the question. In this case, if fewer than five sentence fragments are extracted from the document, the document identification unit 64 calculates the average of the similarities calculated for all extracted sentence fragments and compares it with the average of the other documents. Note that the number of similarities selected is not limited to five; any number greater than or equal to two may be used.

[0034] The sentence fragment extraction unit 65 has a function of extracting a predetermined number of sentence fragments from the document identified by the document identification unit 64 in descending order of similarity calculated by the similarity calculation unit 63. For example, the sentence fragment extraction unit 65 selects three sentence fragment records 161 from the document identified by the document identification unit 64 in descending order of similarity calculated by the similarity calculation unit 63, and extracts sentence fragments from the selected sentence fragment records 161. Note that the number of sentence fragments extracted in descending order of similarity is arbitrary. It may be one or two, or it may be four or more.

[0035] The answer generation unit 66 is a function that generates an answer to a question sentence. That is, the answer generation unit 66 inputs the question sentence and the sentence fragments extracted by the sentence fragment extraction unit 65 into a large-scale language model (LLM). Then, the answer generation unit 66 generates an answer to the question sentence using the large-scale language model. The answer unit 67 is a function that outputs the answer generated by the answer generation unit 66 to the user terminal 30 as a document. That is, the answer unit 67 transmits document data indicating the answer via the network interface 14. The transmitted document data is transmitted via the communication network 40 and received by the user terminal 30 that entered the question regarding the answer. Thus, the answer is output in document form at the user terminal 30.

[0036] The functions of the document input unit 51, the issuing unit 52, the dividing unit 53, the sentence fragment conversion unit 54, and the saving unit 55 are realized by information processing executed by the processor 11 in accordance with a document registration program. The functions of the question input unit 61, the question sentence conversion unit 62, the similarity calculation unit 63, the document identification unit 64, the sentence fragment extraction unit 65, the answer generation unit 66, and the answering unit 67 are realized by information processing executed by the processor 11 in accordance with a question and answer program.

[0037] The document registration program and the question and answer program are types of application programs stored in the main memory 12 or the auxiliary storage device 13. There are no particular limitations on the method for installing the document registration program and the question and answer program into the main memory 12 or the auxiliary storage device 13. The document registration program or the question and answer program can be recorded on a removable recording medium, or can be distributed via communication via the communication network 40 and installed into the main memory 12 or the auxiliary storage device 13. The recording medium can be in any form, such as a CD-ROM or memory card, as long as it can store the program and is readable by the device.

[0038] [Explanation of the literature registration program] 8 is a flowchart showing the main steps of information processing executed by processor 11 of question and answer device 10 in accordance with the document registration program. When the document registration program is started, processor 11 waits for a document to be input in ACT1. In this standby state, when electronic data of the document is received from administrator terminal 20, processor 11 proceeds to ACT2. In ACT2, processor 11 imports the text of the document from the electronic data.

[0039] After importing the text, the processor 11 proceeds to ACT 3. The processor 11 issues a new document ID in ACT 3. For example, the processor 11 counts up the serial number and issues the serial number as the document ID.

[0040] After issuing the document ID, processor 11 proceeds to ACT 4. In ACT 4, processor 11 divides the text of the document into sentence fragments of a predetermined number of characters or less. For example, processor 11 divides the text at convenient points such as punctuation marks and line breaks, into sentences of 500 characters or less.

[0041] After dividing the sentence, processor 11 proceeds to ACT 5. In ACT 5, processor 11 sets the division number, i.e., the number of sentence fragments divided from the sentence, in register N. In ACT 6, processor 11 also resets counter n to "0".

[0042] After setting the number of sentence fragments in register N and resetting counter n, processor 11 proceeds to ACT 7. In ACT 7, processor 11 counts up counter n by "1". Then, in ACT 8, processor 11 checks whether the value of counter n has exceeded the value of register N.

[0043] If the value of counter n does not exceed the value of register N, processor 11 proceeds to ACT 9. Processor 11 selects one sentence fragment divided from the sentence in ACT 9. Processor 11 then converts the sentence fragment into a sentence fragment feature vector in ACT 10. That is, processor 11 converts the sentence fragment into a sentence fragment feature vector by using a well-known embedding engine to represent the individual features of the words that make up the sentence fragment as a numerical vector.

[0044] After converting the sentence fragment into a sentence fragment feature vector, processor 11 proceeds to ACT11. In ACT11, processor 11 creates a sentence fragment record 161 with the record number as the value of counter n. That is, processor 11 creates sentence fragment record 161 using the record number which is the value of counter n, the document ID issued in ACT3, the text data of the sentence fragment selected in the processing of ACT9, and the sentence fragment feature vector obtained in the processing of ACT10. Once the creation of sentence fragment record 161 is complete, processor 11 proceeds to ACT12. Processor 11 saves the sentence fragment record 161 created in ACT12 in the document database 16.

[0045] After storing the sentence fragment record 161 in the document database 16, the processor 11 returns to ACT 7. In ACT 7, the processor 11 counts up the counter n by another "1". Then, in ACT 8, the processor 11 checks whether the value of the counter n has exceeded the value of the register N.

[0046] If the value of counter n does not exceed the value of register N, processor 11 executes the processes of ACT9 to ACT12 again. That is, processor 11 selects an unprocessed sentence fragment that has not been processed in ACT9 to ACT12 from the N sentence fragments divided from the sentence, and converts that sentence fragment into a sentence fragment feature vector. Processor 11 then creates a sentence fragment record 161 whose record number is the value of counter n, and saves that sentence fragment record 161 in the document database 16.

[0047] The processor 11 repeatedly executes the processes of ACT9 to ACT12 until the value of the counter n exceeds the value of the register N. In this way, the document database 16 stores a sentence fragment record 161 created for each sentence fragment divided from the document sentence.

[0048] In ACT8, when the value of counter n exceeds the value of register N, the processor 11 ends the processing of the document registration program.

[0049] Here, the processor 11 realizes the function of a document input unit 51 by processing ACT1 and ACT2. The processor 11 realizes the function of an issuing unit 52 by processing ACT3. The processor 11 realizes the function of a dividing unit 53 by processing ACT4. The processor 11 realizes the function of a sentence fragment conversion unit 54 by processing ACT5 to ACT10. The processor 11 realizes the function of a storage unit 55 by processing ACT11 and ACT12.

[0050] The processing procedure of the document registration program shown in the flowchart of Figure 8 is an example. The order and content of the processing can be changed as appropriate as long as the same operational effect can be achieved. For example, the timing for issuing a document ID is not limited to after the processing of ACT2, that is, after the text has been imported. The document ID may be issued after the processing of ACT4, that is, after the text has been divided, or after the processing of ACT5, that is, after the number of divisions has been set in register N.

[0051] [Question and Answer Program Description] 9 and 10 are flowcharts showing the main steps of information processing executed by processor 11 of question and answering device 10 in accordance with the question and answering program. When the question and answering program is started, processor 11 waits for a question to be input in ACT21 of Fig. 9. In this standby state, when a question is input from user terminal 30, processor 11 proceeds to ACT22. In ACT22, processor 11 retrieves the document input as the question, i.e., the text data of the question sentence.

[0052] After importing the text data of the question, processor 11 proceeds to ACT 23. In ACT 23, processor 11 converts the question into a question feature vector. That is, processor 11 converts the question into a question feature vector by using the same embedding engine that was used to vectorize the sentence fragments to represent the individual features of the words that make up the question fragment as a numerical vector.

[0053] After converting the question sentence into a question fragment feature vector, the processor 11 proceeds to ACT 24. In ACT 24, the processor 11 stores the question sentence feature vector in a question sentence memory. The question sentence memory is, for example, a part of a volatile memory area in the main memory 12.

[0054] After storing the question sentence feature vector, the processor 11 proceeds to ACT 25. In ACT 25, the processor 11 sets the number of records of all sentence fragment records 161 stored in the document database 16 to register R. In ACT 26, the processor 11 also resets counter r to "0."

[0055] After setting the number of records in the sentence fragment record 161 in register R and resetting counter r, processor 11 proceeds to ACT 27. In ACT 27, processor 11 counts up counter r by "1". Then, in ACT 28, processor 11 checks whether the value of counter r has exceeded the value of register R.

[0056] If the value of counter r does not exceed the value of register R, the processor 11 proceeds to ACT 29. In ACT 29, the processor 11 obtains from the document database 16 a sentence fragment record 161 whose record number matches the value of counter r. Then, in ACT 30, the processor 11 obtains a document ID from that sentence fragment record 161. The processor 11 also obtains a sentence fragment feature vector from that sentence fragment record 161 in ACT 31.

[0057] After acquiring the document ID and the sentence fragment characteristic vector from the sentence fragment record 161, the processor 11 proceeds to ACT 32. In ACT 32, the processor 11 calculates the similarity between the question sentence characteristic vector stored in the question sentence memory and the sentence fragment characteristic vector acquired from the sentence fragment record 161 whose record number matches the value of the counter r. That is, the processor 11 individually compares the multidimensional vector information constituting the question sentence characteristic vector with the same-dimensional vector information constituting the sentence fragment characteristic vector to calculate the similarity of the vector information.

[0058] After calculating the similarity between the question sentence feature vector and the sentence fragment feature vector, processor 11 proceeds to ACT 33. In ACT 33, processor 11 associates the document ID obtained from the sentence fragment record 161 whose record number matches the value of counter r with the similarity calculated in the processing of ACT 32 and stores them in the first table 121. That is, processor 11 writes the document ID in column Cb of the row in first table 121 whose table number matches the value of counter r, and writes the similarity in column Cc.

[0059] After correlating the document ID with the similarity and storing it in the first table 121, the processor 11 returns to ACT 27. In ACT 27, the processor 11 further counts up the counter r by "1". Then, in ACT 28, the processor 11 checks whether the value of the counter r has exceeded the value of the register R.

[0060] If the value of counter r does not exceed the value of register R, the processor 11 executes the processes of ACT29 to ACT33 again. That is, the processor 11 obtains from the document database 16 a sentence fragment record 161 whose record number matches the value of counter r, and obtains a document ID and a sentence fragment characteristic vector from the sentence fragment record 161. The processor 11 then calculates the similarity between the sentence fragment characteristic vector and the question sentence characteristic vector stored in the question sentence memory, and stores the document ID and the similarity in the first table 121 in association with each other.

[0061] The processor 11 repeatedly executes the processes of ACT29 to ACT33 until the value of the counter r exceeds the value of the register R. Thus, the first table 121 stores the document ID of the sentence fragment record 161 and the similarity between the sentence fragment feature vector of the sentence fragment record 161 and the question sentence feature vector, in association with the table number equal to the record number of the sentence fragment record 161.

[0062] In ACT28, when the value of counter r exceeds the value of register R, processor 11 proceeds to ACT41 in FIG. 10. Processor 11 creates a document ID list in ACT41. That is, processor 11 retrieves the document IDs written in column Cb of first table 121 in ascending order of table number in column Ca. Processor 11 then checks whether the document ID is written in column Ce of second table 122. If the document ID is not written in second table 122, processor 11 writes the document ID in column Ce in ascending order of table number in column Cd. If the document ID is already written in second table 122, processor 11 discards the document ID. Through the above processing, a document ID list is created in second table 122.

[0063] After creating the document ID list, processor 11 proceeds to ACT42. In ACT42, processor 11 sets the number of documents in register P. The number of documents is the maximum number of table numbers in which document IDs are written in second table 122. In ACT43, processor 11 also resets counter p to "0".

[0064] After setting the number of documents in register P and resetting counter p, processor 11 proceeds to ACT 44. In ACT 44, processor 11 counts up counter p by "1". Then, in ACT 45, processor 11 checks whether the value of counter p has exceeded the value of register P.

[0065] If the value of counter p does not exceed the value of register P, the processor 11 proceeds to ACT 46. In ACT 46, the processor 11 obtains the document ID written in column Ce of the second table 122 in the same row as the table number that matches the value of counter p.

[0066] Once the document ID has been acquired, processor 11 proceeds to ACT47. In ACT47, processor 11 searches first table 121 and compares all similarities described in column Cc in association with the acquired document ID. Then, processor 11 selects the top K similarities in ACT48. "K" is an arbitrary value, and is set to "5" as an example. However, if it is not possible to select the top K similarities, processor 11 selects all similarities.

[0067] If the top K similarities, or fewer than K similarities because K similarities do not exist, have been selected, processor 11 proceeds to ACT 49. Processor 11 calculates the average value of the selected similarities in ACT 49. Then, processor 11 writes the average value of the similarities in column Cf of the second table 122 in the same row as the table number that matches the value of counter p in ACT 50.

[0068] After describing the average similarity values ​​in the second table 122, the processor 11 returns to ACT 44. In ACT 44, the processor 11 counts up the counter p by another "1". Then, in ACT 45, the processor 11 checks whether the value of the counter p has exceeded the value of the register P.

[0069] If the value of counter p does not exceed the value of register P, processor 11 executes the processes of ACT46 to ACT50 again. That is, processor 11 acquires the document ID described in column Ce of the second table 122 in the same row as the table number that matches the value of counter p, compares all similarities described in first table 121 in association with the acquired document ID, and selects the top K similarities. Then, processor 11 calculates the average value of the selected K or fewer similarities, and enters this average value of similarities in column Cf of the second table 122 in the same row as the table number that matches the value of counter p.

[0070] The processor 11 repeatedly executes the processes in ACT46 to ACT50 until the value of the counter p exceeds the value of the register P. Thus, the second table 122 stores document IDs and the average values ​​of a predetermined number of similarities selected in descending order from the similarities calculated for each sentence fragment extracted from the document identified by the document ID, each associated with a table number having a maximum number of documents.

[0071] In ACT45, when the value of counter p exceeds the value of register P, the processor 11 proceeds to ACT51. In ACT51, the processor 11 compares the average similarity values ​​stored in association with each document ID in the second table 122. Then, in ACT52, the processor 11 identifies the document ID with the largest average similarity value. That is, the processor 11 identifies the document ID with the largest average similarity value as the document ID of a document related to the question.

[0072] Once the document ID of the document related to the query has been identified, the processor 11 proceeds to ACT53. In ACT53, the processor 11 searches the first table 121 and compares all similarities described in column Cc in association with the identified document ID. The processor 11 then selects the top L similarities. "L" is an arbitrary value smaller than "K", and is "3" as an example. However, if it is not possible to select the top L similarities, the processor 11 selects all similarities. Note that "L" may be equal to "K".

[0073] If the top L similarities or fewer than L similarities have been selected, processor 11 proceeds to ACT 55. In ACT 55, processor 11 obtains the table number stored in first table 121 in association with the selected similarity, and extracts text data of the sentence fragment from sentence fragment record 161 that includes the table number as the record number.

[0074] After extracting text data of up to L sentence fragments, processor 11 proceeds to ACT 56. Processor 11 creates an input sentence for the large-scale language model in ACT 56. That is, processor 11 uses well-known prompt engineering to combine the text data of the question captured in ACT 22 with the text data of the sentence fragments extracted in ACT 55 to create an input sentence that can be interpreted by the large-scale language model.

[0075] After creating the input sentence for the large-scale language model, processor 11 proceeds to ACT 57. Processor 11 converts the input sentence into tokens in ACT 57. That is, processor 11 uses a well-known tokenizer to break down the input sentence into words, i.e., tokens, that can be processed by the large-scale language model.

[0076] After converting the input sentence into tokens, processor 11 proceeds to ACT 58. In ACT 58, processor 11 inputs the tokens of the input sentence into the large-scale language model. Then, in ACT 59, processor 11 waits for an answer to be generated in the large-scale language model. Once the answer is generated, processor 11 proceeds to ACT 60. In ACT 60, processor 11 outputs text data of a document indicating the answer to user terminal 30. That is, processor 11 transmits the text data of a document indicating the answer via network interface 14 to user terminal 30, which is the sender of the question accepted as input in ACT 21. With this, processor 11 ends processing of the question and answering program.

[0077] Here, the processor 11 realizes the function of a question input unit 61 by processing ACT21 and ACT22. The processor 11 realizes the function of a question sentence conversion unit 62 by processing ACT23 and ACT24. The processor 11 realizes the function of a similarity calculation unit 63 by processing ACT25 to ACT33. The processor 11 realizes the function of a document identification unit 64 by processing ACT41 to ACT52. The processor 11 realizes the function of a sentence fragment extraction unit 65 by processing ACT53 to ACT55. The processor 11 realizes the function of an answer generation unit 66 by processing ACT56 to ACT58. The processor 11 realizes the function of an answer unit 67 by processing ACT67.

[0078] 9 and 10 are merely examples of the processing procedures of the question and answer program. The order and content of the processing can be changed as appropriate as long as similar operational effects can be achieved.

[0079] [Explanation of the effectiveness of the Q&A system] As described above in detail, when a question text is input from the user terminal 30, the question answering device 10 converts the question text into a question text feature vector. Then, the question answering device 10 calculates the similarity between all the sentence fragment feature vectors stored in the literature database 16 and the question text feature vector.

[0080] For each document, the question answering device 10 compares the similarity between the sentence fragment feature vector of the sentence fragment divided from that document and the question sentence feature vector. Then, the question answering device 10 selects the top K similarities and calculates the average value of the selected K or fewer similarities. The question answering device 10 compares the average values ​​of the similarities calculated for each document and identifies the document with the largest average value as the document related to the question.

[0081] The question answering device 10 extracts the top L sentence fragments with the highest similarity to the question sentence feature vector from sentence fragments of documents identified as documents related to the question sentence. The question answering device 10 then inputs the top L extracted sentence fragments together with the question into a large-scale language model to generate an answer to the question. The question answering device 10 outputs text data indicating the answer from the network interface 14.

[0082] The text data sent from the network interface 14 of the question and answering device 10 is transmitted over the communication network 40 and received by the user terminal 30 that sent the question. At the user terminal that receives the text data, for example, the text data is displayed on a display device, or the text data is converted into voice and output from a speaker. In this way, the user of the user terminal 30 can obtain the answer to the question.

[0083] Here, the input sentences input to the large-scale language model are sentence fragments that have a high degree of similarity to the question sentence, among the sentence fragments divided from the document identified by the document identification unit 64. Because these sentence fragments are extracted from the document that is the subject of the question, the accuracy of the answer obtained by the large-scale language model is high. In other words, there is no risk of sentence fragments extracted from inappropriate documents being mixed in as sentence fragments to be input to the large-scale language model, so there is no concern that the accuracy of the answer obtained by the large-scale language model will deteriorate.

[0084] Thus, according to this embodiment, it is possible to provide a question and answering device 10 that can improve the accuracy of answers obtained by a large-scale language model by preventing sentence fragments extracted from inappropriate documents from being mixed into questions. As a result, it is possible to provide a question and answering system 1 that provides highly accurate answers by using a large-scale language model to create answers to users' written questions and provide written answers to users.

[0085] <Second embodiment> Next, a second embodiment will be described. The second embodiment differs from the first embodiment in the processing content of the function as the document identification unit 64. Other points are the same as in the first embodiment, so Figures 1 to 9 also apply to the second embodiment as they are, and detailed explanations will be omitted. Incidentally, in the first embodiment, the processing related to the function as the document identification unit 64 is explained as the processing of ACT41 to ACT52 in Figure 10.

[0086] FIG. 11 is a flowchart showing the main steps of the information processing of the second embodiment that the processor 11 executes after the value of counter r exceeds the value of register R in ACT28 of FIG. 9. In the second embodiment, when the value of counter r exceeds the value of register R, the processor 11 proceeds to ACT71 of FIG. 11. The processor 11 creates a document ID list in ACT71. Then, the processor 11 sets the number of documents in register P in ACT72. The processor 11 also resets counter p to "0" in ACT73. Thereafter, the processor 11 counts up counter p by "1" in ACT74. Then, the processor 11 checks whether the value of counter p has exceeded the value of register P in ACT75. If the value of counter p has not exceeded the value of register P, the processor 11 acquires the document ID written in column Ce of the second table 122 in the same row as the table number that matches the value of counter p in ACT76. The processing in ACT71 through ACT76 above is the same as the processing in ACT41 through ACT46 described using FIG. 10. Therefore, a detailed description will be omitted here.

[0087] Once the document ID has been acquired, the processor 11 proceeds to ACT 77. In ACT 77, the processor 11 searches the first table 121 and detects all similarities described in column Cc in association with the acquired document ID. Then, in ACT 78, the processor 11 compares all detected similarities individually with a threshold value. The threshold value is an arbitrary value that is deemed to indicate a high degree of similarity.

[0088] After comparing all similarities associated with the same document ID with the threshold, the processor 11 proceeds to ACT 79. The processor 11 selects similarities equal to or greater than the threshold in ACT 79. The processor 11 then calculates the average value of the selected similarities in ACT 80.

[0089] Once the average value of the similarities has been calculated, the processor 11 proceeds to ACT 81. In ACT 81, the processor 11 writes the average value of the similarities in column Cf of the second table 122 in the same row as the table number that matches the value of counter p.

[0090] After describing the average similarity values ​​in the second table 122, the processor 11 returns to ACT 74. In ACT 74, the processor 11 further counts up the counter p by "1". Then, in ACT 75, the processor 11 checks whether the value of the counter p has exceeded the value of the register P.

[0091] If the value of counter p does not exceed the value of register P, processor 11 executes the processes of ACT76 to ACT81 again. That is, processor 11 obtains the document ID described in column Ce of the second table 122 in the same row as the table number that matches the value of counter p, searches first table 121 using that document ID, and detects all similarities described in column Cc in association with that document ID. Then, processor 11 selects similarities that are equal to or greater than a threshold from all the detected similarities, calculates the average value of the selected similarities, and writes the average value of similarities in column Cf of the second table 122 in the same row as the table number that matches the value of counter p.

[0092] The processor 11 repeatedly executes the processes in ACT76 to ACT81 until the value of the counter p exceeds the value of the register P. Thus, the second table 122 stores the document ID and the average value of the similarities equal to or greater than the threshold value among the similarities calculated for each sentence fragment extracted from the document identified by the document ID, in association with the table number having the largest number of documents.

[0093] In ACT 75, when the value of counter p exceeds the value of register P, the processor 11 proceeds to ACT 82. In ACT 82, the processor 11 compares the average similarity values ​​stored in association with each document ID in the second table 122. Then, in ACT 83, the processor 11 identifies the document ID with the largest average similarity value. That is, the processor 11 identifies the document ID with the largest average similarity value as the document ID of a document related to the question.

[0094] Once the document ID of the document related to the question has been identified, the processor 11 proceeds to ACT 53. Thereafter, the processor 11 executes the same processes as in the first embodiment in ACT 53 to ACT 60.

[0095] Here, the processor 11 performs the processing of ACT71 to ACT83 to realize the function of the document identification unit 64. That is, for each of a plurality of documents, the processor 11 calculates the average value of similarities equal to or greater than the threshold calculated by the similarity calculation unit 63 for each sentence fragment extracted from the document, and identifies the document with the largest average value as the document related to the question sentence.

[0096] As described above, in the second embodiment, the question answering device 10 compares, for each document, the similarity between the sentence fragment feature vector of the sentence fragment divided from that document and the question sentence feature vector with a threshold value. Then, the question answering device 10 selects similarities greater than the threshold value and calculates the average value of the selected similarities. The question answering device 10 compares the average values ​​of the similarities calculated for each document and identifies the document with the largest average value as the document related to the question sentence.

[0097] The question answering device 10 extracts the top L sentence fragments with the highest similarity to the question sentence feature vector from sentence fragments of documents identified as documents related to the question sentence. The question answering device 10 then inputs the top L extracted sentence fragments together with the question into a large-scale language model to generate an answer to the question. The question answering device 10 outputs text data indicating the answer from the network interface 14.

[0098] As described above in detail, in the second embodiment, the sentence fragments input to the large-scale language model are limited to sentence fragments extracted from the document that is the subject of the question. Therefore, there is no risk of sentence fragments extracted from inappropriate documents being mixed in, and there is no concern that the accuracy of the answer obtained by the large-scale language model will deteriorate. Therefore, the same effects as those of the first embodiment can be achieved.

[0099] Furthermore, in the second embodiment, it is only necessary to select a similarity above a threshold in ACT79, so there is no need to compare the magnitude of the similarity, as compared to the first embodiment in which the top K similarities are selected in ACT48, thereby simplifying the processing.

[0100] <Third embodiment> Next, a third embodiment will be described. The third embodiment differs from the first embodiment in the processing content of the function as the document identification unit 64. Since other points are the same as those in the first embodiment, Figures 1 to 9 are also applied to the third embodiment as they are, and detailed explanations will be omitted.

[0101] FIG. 12 is a flowchart showing the main steps of the information processing of the third embodiment executed by the processor 11 after the value of counter r exceeds the value of register R in ACT28 of FIG. 9. In the third embodiment, when the value of counter r exceeds the value of register R, the processor 11 proceeds to ACT91 of FIG. 12. The processor 11 creates a document ID list in ACT91. Then, the processor 11 sets the number of documents in register P in ACT92. The processor 11 also resets counter p to "0" in ACT93. Thereafter, the processor 11 counts up counter p by "1" in ACT94. The processor 11 then checks whether the value of counter p has exceeded the value of register P in ACT95. If the value of counter p has not exceeded the value of register P, the processor 11 acquires the document ID written in column Ce of the second table 122 in the same row as the table number that matches the value of counter p in ACT96. The processing in ACT91 through ACT96 above is the same as the processing in ACT41 through ACT46 described using FIG. 10. Therefore, a detailed description will be omitted here.

[0102] Once the document ID has been acquired, processor 11 proceeds to ACT 97. In ACT 97, processor 11 searches first table 121 and detects all similarities described in column Cc in association with the acquired document ID. Then, in ACT 98, processor 11 calculates the average value of all the detected similarities.

[0103] Once the average value of the similarities has been calculated, the processor 11 proceeds to ACT 99. In ACT 99, the processor 11 writes the average value of the similarities in column Cf of the second table 122 in the same row as the table number that matches the value of counter p.

[0104] After describing the average similarity values ​​in the second table 122, the processor 11 returns to ACT 94. In ACT 94, the processor 11 further counts up the counter p by "1". Then, in ACT 95, the processor 11 checks whether the value of the counter p has exceeded the value of the register P.

[0105] If the value of counter p does not exceed the value of register P, processor 11 executes the processes of ACT96 to ACT99 again. That is, processor 11 obtains the document ID described in column Ce of the second table 122 in the same row as the table number that matches the value of counter p, searches first table 121 using that document ID, and detects all similarities described in column Cc in association with that document ID. Processor 11 then calculates the average value of all the detected similarities and writes this average value of similarities in column Cf of the second table 122 in the same row as the table number that matches the value of counter p.

[0106] The processor 11 repeatedly executes the processes in ACT96 to ACT99 until the value of the counter p exceeds the value of the register P. Thus, the second table 122 stores the document ID and the average value of the similarity calculated for each sentence fragment extracted from the document identified by the document ID, in association with the table number having the maximum number of documents.

[0107] In ACT 95, when the value of counter p exceeds the value of register P, the processor 11 proceeds to ACT 100. In ACT 100, the processor 11 compares the average similarity values ​​stored in association with each document ID in the second table 122. Then, in ACT 101, the processor 11 identifies the document ID with the largest average similarity value. That is, the processor 11 identifies the document ID with the largest average similarity value as the document ID of a document related to the question.

[0108] Once the document ID of the document related to the question has been identified, the processor 11 proceeds to ACT 53. Thereafter, the processor 11 executes the same processes as in the first embodiment in ACT 53 to ACT 60.

[0109] Here, the processor 11 performs the processing of ACT91 to ACT101 to realize the function of the document identification unit 64. That is, the processor 11 calculates the average value of the similarities calculated by the similarity calculation unit 63 for each sentence fragment extracted from each of the multiple documents, and identifies the document with the largest average value as the document related to the question sentence.

[0110] As described above, in the third embodiment, the question answering device 10 calculates, for each document, the average value of the similarity between the sentence fragment feature vector of the sentence fragment divided from that document and the question sentence feature vector. The question answering device 10 compares the average values ​​of the similarity calculated for each document, and identifies the document with the largest average value as the document related to the question sentence.

[0111] The question answering device 10 extracts the top L sentence fragments with the highest similarity to the question sentence feature vector from sentence fragments of documents identified as documents related to the question sentence. The question answering device 10 then inputs the top L extracted sentence fragments together with the question into a large-scale language model to generate an answer to the question. The question answering device 10 outputs text data indicating the answer from the network interface 14.

[0112] As described above in detail, in the third embodiment, the sentence fragments input to the large-scale language model are limited to sentence fragments extracted from the document that is the subject of the question. Therefore, there is no risk of sentence fragments extracted from inappropriate documents being mixed in, and there is no concern that the accuracy of the answer obtained by the large-scale language model will deteriorate. Therefore, the same effects as those of the first embodiment can be achieved.

[0113] Furthermore, in the third embodiment, there is no need to select the top K similarities as in the first embodiment, or to select similarities above a threshold as in the second embodiment, thereby further simplifying the processing.

[0114] <Fourth embodiment> Next, a fourth embodiment will be described. The fourth embodiment differs from the first embodiment in the processing content of the function as the document identification unit 64. Since other points are the same as those in the first embodiment, Figures 1 to 9 are also applied to the fourth embodiment as they are, and detailed explanations will be omitted.

[0115] Fig. 15 is a flowchart showing the main steps of the information processing of the fourth embodiment that the processor 11 executes after the value of the counter r exceeds the value of the register R in ACT 28 of Fig. 9. In the fourth embodiment, when the value of the counter r exceeds the value of the register R, the processor 11 proceeds to ACT 111 of Fig. 13. In ACT 111, the processor 11 compares all similarities stored in association with the document IDs in the first table 121. Then, in ACT 112, the processor 11 detects the maximum similarity.

[0116] Having detected the highest similarity, the processor 11 proceeds to ACT 113. In ACT 113, the processor 11 identifies the document ID stored in the first table 121 in association with the highest similarity. That is, the processor 11 identifies the document ID with the highest similarity as the document ID of the document related to the question sentence.

[0117] Once the document ID of the document related to the question has been identified, the processor 11 proceeds to ACT 53. Thereafter, the processor 11 executes the same processes as in the first embodiment in ACT 53 to ACT 60.

[0118] Here, the processor 11 performs the processing of ACT111 to ACT113 to realize the function of the document identification unit 64. That is, the processor 11 identifies the document from which the sentence fragment with the largest similarity calculated by the similarity calculation unit 63 has been extracted as a document related to the question sentence.

[0119] As described above, in the fourth embodiment, the question answering device 10 compares the similarity between the sentence fragment feature vector of each sentence fragment divided from each document and the question sentence feature vector. Then, the question answering device 10 identifies the document for which the largest similarity is calculated as the document related to the question sentence.

[0120] The question answering device 10 extracts the top L sentence fragments with the highest similarity to the question sentence feature vector from sentence fragments of documents identified as documents related to the question sentence. The question answering device 10 then inputs the top L extracted sentence fragments together with the question into a large-scale language model to generate an answer to the question. The question answering device 10 outputs text data indicating the answer from the network interface 14.

[0121] As described above in detail, in the fourth embodiment, the sentence fragments input to the large-scale language model are limited to sentence fragments extracted from the document that is the subject of the question. Therefore, there is no risk of sentence fragments extracted from inappropriate documents being mixed in, and there is no concern that the accuracy of the answer obtained by the large-scale language model will deteriorate. Therefore, the same effects as those of the first embodiment can be achieved.

[0122] Furthermore, in the fourth embodiment, since there is no need to calculate the average value of the similarities as in the first to third embodiments, the processing can be further simplified. Also, since the second table 122 is unnecessary, the configuration can be simplified.

[0123] [Other embodiments] The above describes several embodiments that can improve the accuracy of answers obtained by large-scale language models by preventing the inclusion of sentence fragments extracted from documents that are inappropriate for the question, but such embodiments are not limited to these.

[0124] For example, in the above embodiment, question and answering device 10 uses part of the storage area of ​​auxiliary storage device 13 as literature database 16. In another embodiment, a database server for managing literature database 16 is connected to communication network 40. Question and answering device 10 may then access the database server via communication network 40 and search literature database 16. In this case, the database server may have the functions of literature input unit 51, issuing unit 52, dividing unit 53, sentence fragment conversion unit 54, and saving unit 55.

[0125] Although several embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be embodied in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included within the scope of the invention and the scope of the inventions and their equivalents as defined in the claims. [Explanation of symbols]

[0126] 1...Question answering system, 10...Question answering device, 11...Processor, 12...Main memory, 13...Auxiliary storage device, 14...Network interface, 15...System transmission path, 16...Literature database, 20...Administrator terminal, 30...User terminal, 40...Communication network, 51...Literature input unit, 52...Issuance unit, 53...Dividing unit, 54...Sentence fragment conversion unit, 55...Storage unit, 61...Question input unit, 62...Question sentence conversion unit, 63...Similarity calculation unit, 64...Literature identification unit, 65...Sentence fragment extraction unit, 66...Answer generation unit, 67...Answer unit, 121...First table, 122...Second table, 161...Sentence fragment record.

Claims

1. a database unit that stores, as one record, a sentence fragment extracted from each of a plurality of documents, a sentence fragment feature vector representing a distributed representation of the sentence fragment, and document identification information that identifies the document from which the sentence fragment was extracted; a similarity calculation unit that calculates a similarity between the input question sentence and each of the sentence fragments using a question sentence feature vector representing a distributed representation of the input question sentence and a sentence fragment feature vector of each record stored in the database unit; a document identification unit that identifies a document related to the question sentence from among the plurality of documents based on the similarity calculated for each sentence fragment by the similarity calculation unit; a sentence fragment extraction unit that extracts a predetermined number of sentence fragments from the document identified by the document identification unit in descending order of the similarity calculated by the similarity calculation unit; an answer generation unit that generates an answer to the question sentence based on the question sentence and a predetermined number of sentence fragments extracted by the sentence fragment extraction unit; A question and answer device comprising:

2. 2. The question and answering device of claim 1, wherein the document identification unit selects, for each of the plurality of documents, a predetermined number of similarities in descending order from the similarities calculated by the similarity calculation unit for each sentence fragment extracted from the document, calculates the average value thereof, and identifies the document with the largest average value as a document related to the question sentence.

3. 2. The question and answering device of claim 1, wherein the document identification unit calculates an average value of similarities equal to or greater than a threshold calculated by the similarity calculation unit for each of the plurality of documents for each sentence fragment extracted from the document, and identifies the document with the largest average value as a document related to the question sentence.

4. 2. The question and answering device according to claim 1, wherein the document identification unit calculates, for each of the plurality of documents, an average value of the similarities calculated by the similarity calculation unit for each sentence fragment extracted from the document, and identifies the document with the largest average value as a document related to the question sentence.

5. 2. The question and answering device according to claim 1, wherein the document specifying unit specifies, as a document related to the question, a document from which a sentence fragment for which the similarity calculation unit has calculated the largest similarity has been extracted.

6. a literature input unit for inputting literature; a division unit that divides the document input via the document input unit into sentence fragments; a sentence fragment conversion unit that converts the sentence fragment into the sentence fragment feature vector; an issuing unit that issues the document identification information for the document inputted via the document input unit; a storage unit that stores, for each sentence fragment divided by the dividing unit, the sentence fragment, the sentence fragment feature vector converted by the sentence fragment conversion unit, and the document identification information issued by the issuing unit as one record in the database unit; 6. The question and answering device according to claim 1, further comprising:

7. a question input section for inputting a question; a question sentence conversion unit that converts the question sentence input via the question sentence input unit into the question sentence feature vector; Further comprising: The question and answering device according to claim 6 , wherein the sentence fragment conversion unit and the question conversion unit convert the data into vector information representing a common distributed representation.

8. a computer for a question and answering apparatus including a database unit that stores, as one record, sentence fragments extracted from a plurality of documents, sentence fragment feature vectors that represent distributed representations of the sentence fragments, and document identification information that identifies the documents from which the sentence fragments were extracted; a similarity calculation unit that calculates a similarity between the input question sentence and each of the sentence fragments using a question sentence feature vector representing a distributed representation of the input question sentence and a sentence fragment feature vector of each record stored in the database unit; a document identification unit that identifies a document related to the question sentence from among the plurality of documents based on the similarity calculated for each sentence fragment by the similarity calculation unit; a sentence fragment extraction unit that extracts a predetermined number of sentence fragments from the document identified by the document identification unit in descending order of the similarity calculated by the similarity calculation unit; and an answer generation unit that generates an answer to the question sentence based on the question sentence and a predetermined number of sentence fragments extracted by the sentence fragment extraction unit; A program to function as a

Images