Information Search System

Abstract

The large-scale language model's answer to the user's question is presented so that the user can judge its validity. [Solution] A context information search unit (23) searches a document database (13a) in which text groups obtained by dividing a document, feature vectors of the text groups, and page numbers of the text groups are registered in association with each other, using the feature vector of the question, and acquires, as context information, a text group whose feature vector similarity satisfies a predetermined condition. A prompt generation unit (24) generates a prompt including the question and its context information, and an answer acquisition unit (25) acquires an answer corresponding to the prompt from a large-scale language model (4a). An answer output unit (26) outputs, as an answer to the question, the above-mentioned context information and the page number associated with the context information, along with the answer corresponding to the prompt.

Description

[Technical Field]

[0001] The present invention relates to an information retrieval system. [Background technology]

[0002] In recent years, large-scale language models (LLMs) such as OpenAI's GPT and Google's PaLM2 have been put into practical use, making it possible to perform tasks such as question-and-answering in natural language.

[0003] A certain sentence generation device generates a prompt to be input into a large-scale language model by (a) generating another question sentence corresponding to an input question sentence based on question generation examples and conversation history, (b) calculating feature vectors of sentences generated from the original question sentence and the other question sentence, (c) retrieving sentences with high similarity from a database based on the feature vector, and (d) adding additional sentences generated from the retrieved sentences as reference information to the original question sentence (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent No. 7325152 specification Summary of the Invention [Problem to be solved by the invention]

[0005] However, large-scale language models such as those described above have a problem called hallucination, which can generate inappropriate answers (answers that are not factual or based on fictitious facts, etc.). Some users may believe that such inappropriate answers are correct when presented to them.

[0006] The present invention has been made in consideration of the above-mentioned problems, and aims to provide an information retrieval system that presents answers from a large-scale language model to a user's question so that the user can judge the validity of the answers. [Means for solving the problem]

[0007] The information retrieval system of the present invention is an information retrieval system that provides answers to questions using a large-scale language model, and includes: a question receiving unit that receives the question; a context information search unit that searches a document database in which text groups obtained by dividing a document, feature vectors of the text groups, and page numbers of the text groups are registered in association with each other, using the feature vector of the question, and acquires text groups whose similarity of the feature vectors satisfies a predetermined condition as context information; a prompt generation unit that generates a prompt including the question and the context information; an answer acquisition unit that acquires an answer corresponding to the prompt using the large-scale language model; and an answer output unit that outputs the context information and the page number associated with the context information together with the answer corresponding to the prompt as an answer to the question. [Effects of the Invention]

[0008] According to the present invention, an information retrieval system is provided that presents answers from a large-scale language model to a user's question so that the user can judge the validity of the answers.

[0009] The above and other objects, features and advantages of the present invention will become more apparent from the following detailed description taken in conjunction with the accompanying drawings. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a block diagram showing the configuration of an information retrieval system according to an embodiment of the present invention. [Figure 2] FIG. 2 is a diagram showing an example of a document. [Figure 3]FIG. 3 is a diagram illustrating a text group extracted from the document shown in FIG. 2 in the first embodiment. [Figure 4] FIG. 4 is a diagram illustrating the correspondence between the text groups and page numbers shown in FIG. [Figure 5] FIG. 5 is a diagram showing an example of a template for a prompt. [Figure 6] FIG. 6 is a diagram showing an example of a response to a question. [Figure 7] FIG. 7 is a flowchart illustrating the registration of a document in the information retrieval system shown in FIG. [Figure 8] FIG. 8 is a flowchart illustrating an information search in the information search system shown in FIG. [Figure 9] FIG. 9 is a diagram illustrating document registration in the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0012] Embodiment 1

[0013] Fig. 1 is a block diagram showing the configuration of an information retrieval system according to an embodiment of the present invention. The information retrieval system 1 shown in Fig. 1 is an information retrieval system that uses a large-scale language model 4a to provide answers to questions from users, and includes a processing unit 11 as a computer, a communication unit 12, and a storage unit 13. Note that, although the information retrieval system 1 is implemented in a single computer device here, it may instead be implemented in a distributed manner across multiple computer devices.

[0014] The communication device 12 is a device (such as a network interface) capable of data communication with other devices (here, a user terminal device 3, a server 4, etc.) via a computer network 2 such as the Internet or an intranet. The user terminal device 3 is a device capable of network communication, such as a personal computer or smartphone operated by a user. The server 4 has a built-in large-scale language model 4a, and upon receiving a prompt, generates an answer corresponding to the prompt using the large-scale language model 4a and transmits it as a response to the prompt.

[0015] The storage device 13 is a non-volatile storage device such as a hard disk drive or flash memory, and stores programs, data, etc. The storage device 13 stores a document database 13a, template data 13b, etc., which will be described later.

[0016] Here, the arithmetic processing unit 11 executes the program in the storage device 13, It operates as a document registration unit 21, a question reception unit 22, a context information search unit 23, a prompt generation unit 24, an answer acquisition unit 25, and an answer verification unit 26.

[0017] The document database 13a stores text groups obtained by dividing a document, the feature vectors of the text groups, and the page numbers of the text groups, all associated with one another. These documents may be specific internal documents such as company rules, or may be publicly available documents.

[0018] The document registration unit 21 registers a document specified by a user in the document database 13a. Specifically, the document registration unit 21 (a) divides the above-mentioned document to generate text groups, (b) identifies page numbers of the text groups based on the document, (c) derives feature vectors for the generated text groups, and (d) associates the generated text groups, the identified page numbers, and the derived feature vectors with each other and registers them in the document database. The feature vectors are derived from the text groups using existing embedding processing.

[0019] 2 is a diagram showing an example of a document. The document is, for example, a structured PDF (Portable Document Format) file containing text for each page, and the text and page number of each page are extracted from the PDF file using an existing method.

[0020] Fig. 3 is a diagram illustrating a text group extracted from the document shown in Fig. 2 in the first embodiment. Fig. 4 is a diagram illustrating the correspondence between the text group and the page numbers shown in Fig. 3.

[0021] The document registration unit 21 divides the document into groups of a predetermined number of characters (for example, 1,000 characters) as shown in FIG. 3, generates text groups #1 to #n, and as shown in FIG. 4, identifies the page number corresponding to each text group #i, and associates the text groups, page numbers, and feature vectors with each other and registers them in the document database 13a.

[0022] In this case, for a text group that spans two pages, such as text group #2 in Figure 3, the page number of the page to which the beginning of the text group belongs is identified as the page number corresponding to the text group, as shown in Figure 4.

[0023] The question receiving unit 22 receives a question. Specifically, the question receiving unit 22 receives a question (text data) transmitted from the user terminal device 3 using the communication device 12.

[0024] The context information search unit 23 searches the document database 13a using the feature vector of the received question, and obtains a group of texts (text data) from the document database 13a as context information, where the similarity (such as cosine similarity) between the feature vectors of both satisfy a predetermined condition.

[0025] The prompt generation unit 24 generates a prompt including the above-mentioned question and the above-mentioned context information. Specifically, the prompt generation unit 24 (a) acquires a template (text data) for the prompt by referring to the template data 13b, and (b) generates the prompt by inserting the above-mentioned question and the above-mentioned context information into the template.

[0026] FIG. 5 is a diagram showing an example of a template for a prompt. The prompt includes a command section, a context information section, and a question section. The command section is text indicating a command to the large-scale language model 4a, and the context information section is a section in which the above-mentioned context information is written, and in the template, includes a parameter {context} to be replaced with the context information. The question section is a section in which the above-mentioned question is written, and in the template, includes a parameter {question} to be replaced with the question.

[0027] The answer acquisition unit 25 acquires an answer corresponding to a prompt in the large-scale language model 4 a. Specifically, the answer acquisition unit 25 uses the communication device 12 to (a) send a prompt to the server 4 of the large-scale language model 4 a and receive an answer corresponding to the prompt from the server 4.

[0028] The answer output unit 26 outputs the answer to the question, together with the answer to the prompt, the context information and the page number associated with the context information. At this time, the answer to the question is transmitted by the answer output unit 26 to the user terminal device 3 using the communication device 12 and presented to the user.

[0029] Fig. 6 is a diagram showing an example of an answer to a question. For example, as shown in Fig. 6, answer output unit 26 displays to the user, as an answer to the above-mentioned question, an answer corresponding to the prompt, the above-mentioned context information ("reference information" in Fig. 6), and the above-mentioned page number ("page" in Fig. 6) on a single screen on a predetermined display device (the display device of user terminal device 3).

[0030] Next, the operation of the information retrieval system according to the first embodiment will be described.

[0031] (a) Document registration

[0032] FIG. 7 is a flowchart illustrating the registration of a document in the information retrieval system shown in FIG.

[0033] When the document registration unit 21 receives a document (such as a PDF file) along with a document registration request (step S1), it extracts text and page numbers for each page from the document (step S2), divides the text of the series of pages to generate text groups (step S3), derives feature vectors for the generated text groups (step S4), and for each generated text group, associates the text group, the identified page number, and the derived feature vector with each other and registers them in the document database 13a (step S5).

[0034] (b) Information search

[0035] FIG. 8 is a flowchart illustrating an information search in the information search system shown in FIG.

[0036] When the question receiving unit 22 receives a question (step S21), the context information search unit 23 derives a feature vector of the question, searches the document database 13a using the feature vector, and acquires corresponding context information and page numbers (step S22).

[0037] Next, the prompt generator 24 generates a prompt including the above-mentioned question and the above-mentioned context information (step S23), and the answer acquirer 25 acquires an answer corresponding to the first prompt from the large-scale language model 4a (step S24).

[0038] The answer output unit 26 outputs the answer corresponding to the prompt as well as the above-mentioned context information and page number as an answer corresponding to the question (step S25).

[0039] As described above, according to the first embodiment, the context information search unit 23 searches the document database 13a, in which text groups obtained by dividing a document, feature vectors of the text groups, and page numbers of the text groups are registered in association with each other, using the feature vector of the question, and acquires, as context information, a text group whose feature vector similarity satisfies a predetermined condition. The prompt generation unit 24 generates a prompt including the question and its context information, and the answer acquisition unit 25 acquires an answer corresponding to the prompt from the large-scale language model 4a. The answer output unit 26 outputs, as an answer to the question, the above-mentioned context information and the page number associated with the context information, along with the answer corresponding to the prompt.

[0040] This allows the user to determine the appropriateness of the large-scale language model's answer to the user's question by referring to the context information and page number along with the answer corresponding to the prompt. Specifically, the user can appropriately determine the appropriateness by referring to the presented context information, and can also more appropriately determine the appropriateness by referring to the page number in the original document based on the presented page number.

[0041] Embodiment 2

[0042] FIG. 9 is a diagram illustrating document registration in the second embodiment.

[0043] In the second embodiment, for example, as shown in FIG. 9, the document database 13a includes a first database 41 and a second database .

[0044] In the first database 41, text groups obtained by dividing a document into pages, the feature vectors of the text groups, and the page numbers are registered in association with each other.

[0045] In the second database 42, text groups obtained by dividing a document into a predetermined number of characters within each page and the feature vectors of the text groups are registered in association with each other.

[0046] That is, for a specified document, the document registration unit 21 (a) derives text groups obtained by dividing the document into pages, the feature vectors of the text groups, and the page numbers, and registers these in the first database 41, and (b) derives text groups obtained by dividing the document into a predetermined number of characters (for example, 1,000 characters) within each page, and the feature vectors, and registers these in the second database 42. Note that with regard to the text groups obtained by dividing each page into a predetermined number of characters, the number of characters in the text groups at the end of the page may be less than the predetermined number.

[0047] In the second embodiment, the context information search unit 23 searches the second database 42 using the feature vector of the question, and acquires, as context information, a group of texts whose feature vector similarity satisfies a predetermined condition.

[0048] Furthermore, the page number of the answer corresponding to the question is identified based on the first database 41. Here, the context information search unit 23 or the answer output unit 26 searches the first database 41 with the feature vector of the text group acquired as context information, and acquires the page number associated with the text group whose feature vector similarity satisfies a predetermined condition as the page number of the answer corresponding to the question.

[0049] That is, the first database 41 is a database for identifying page numbers of answers corresponding to questions, and the second database 42 is a database for identifying text groups (context information) corresponding to questions.

[0050] The other configurations and operations of the information retrieval system according to the second embodiment are the same as those of the first embodiment, and therefore the description thereof will be omitted.

[0051] It should be noted that various changes and modifications to the above-described embodiments will be apparent to those skilled in the art. Such changes and modifications may be made without departing from the spirit and scope of the subject matter and without diminishing its intended advantages. In other words, it is intended that such changes and modifications be included within the scope of the claims.

[0052] For example, in the above embodiment, the document name may be obtained based on the file name of the document data file or user input, and the document name may be associated with a group of text extracted from the document along with feature vectors, page numbers, etc., and these may be registered in the document database 13a, and when an answer is output, the corresponding document name may be output along with the page number. [Industrial Applicability]

[0053] The present invention is applicable, for example, to information retrieval systems that utilize large-scale language models. [Explanation of symbols]

[0054] 1 Information retrieval system 4a Large-scale language models 13a Document Database 21 Document Registration Department 22 Question Reception Department 23 Context Information Search Unit 24 Prompt Generation Unit 25 Answer acquisition part 26 Answer output section 41 Database 1 42 Second Database

Claims

1. An information retrieval system that uses large-scale language models to provide answers corresponding to questions, a question receiving unit that receives the question; a context information search unit that searches a document database in which text groups obtained by dividing a document, feature vectors of the text groups, and page numbers of the text groups are registered in association with each other, using the feature vector of the question, and acquires, as context information, text groups whose feature vector similarities satisfy a predetermined condition; a prompt generator that generates a prompt including the question and the context information; an answer acquisition unit that acquires an answer corresponding to the prompt using a large-scale language model; an answer output unit that outputs, as an answer to the question, an answer to the prompt, together with the context information and a page number associated with the context information; An information retrieval system comprising:

2. (a) dividing the document to generate the text group; (b) identifying page numbers of the text group based on the document; (c) deriving feature vectors of the text group; and (d).

2. The information retrieval system according to claim 1, further comprising a document registration unit that associates the generated text group, the identified page number, and the derived feature vector with each other and registers them in the document database.

3. the document database includes a first database in which text groups obtained by dividing the document into pages, feature vectors of the text groups obtained by dividing the document into pages, and page numbers of the text groups are registered in association with one another, and a second database in which text groups obtained by dividing the document into pages by a predetermined number of characters, and feature vectors of the text groups obtained by dividing the document into pages by a predetermined number of characters are registered in association with one another; the context information search unit searches the second database using the feature vector of the question, and acquires, as context information, a group of texts whose similarity with the feature vector satisfies a predetermined condition; the page number is identified based on the first database; 3. The information retrieval system according to claim 1, wherein:

4. 2. The information retrieval system according to claim 1, wherein the answer output unit displays the context information and the page number associated with the context information on a single screen on a predetermined display device, together with the answer corresponding to the prompt, as the answer corresponding to the question.

Images