Device and method
The apparatus improves information extraction from documents by using generative AI to acquire multiple responses and employ diverse output settings, ensuring reliable and flexible extraction of relevant document portions.
Patent Information
- Application Number
- PCT/JP2025/017055
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2025-05-09
- Publication Date
- 2026-11-12
AI Technical Summary
Existing document creation and proofreading support apparatuses are unable to effectively extract portions related to input information from documents.
An apparatus and method that utilize a generative AI to acquire multiple responses to the same input information and an extraction unit to extract relevant portions from documents based on these responses, employing techniques like setting higher randomness or diversity in the AI's output to enhance extraction accuracy.
Enhances the reliability and flexibility of extracting relevant document portions, reducing oversights and enabling more accurate information retrieval and response generation.
Smart Images

Figure JP2025017055_12112026_PF_FP_ABST
Abstract
Description
Apparatus and Method
[0001] One aspect of the present disclosure relates to an apparatus and a method for extracting a portion related to input information from a document.
[0002] Patent Document 1 below discloses a document creation and proofreading support apparatus that extracts portions related to numbers from Japanese sentences.
[0003] Japanese Unexamined Patent Publication No. 63-163958
[0004] However, the above document creation and proofreading support apparatus cannot extract a portion related to input information from a document. Accordingly, it is desired to extract the portion related to the input information from the document.
[0005] An apparatus according to one aspect of the present disclosure includes an acquisition unit that acquires a plurality of responses obtained by inputting the same input information to generative AI a plurality of times, and an extraction unit that extracts a portion related to the input information from a document based on the plurality of responses.
[0006] In such an aspect, the portion related to the input information in the document is extracted based on the plurality of responses obtained by inputting the same input information to the generative AI a plurality of times. Thereby, the portion related to the input information in the document can be extracted.
[0007] According to one aspect of the present disclosure, the portion related to the input information in the document can be extracted.
[0008] This figure shows an example of the system configuration of a system including the device according to the embodiment. This figure shows an example of the relationship between the device according to the embodiment and the generated AI. This figure shows a modified version of the relationship in Figure 2. This figure shows an example of the functional configuration of the device according to the embodiment. This flowchart shows an example of the processing performed by the device according to the embodiment. This flowchart shows an example of the specific processing performed by the device according to the embodiment. This is a sequence diagram (1) showing an example of the specific processing performed by the device according to the embodiment. This figure shows an example of a query generation prompt. This figure shows an example of the output based on the query generation prompt. This is a sequence diagram (2) showing an example of the specific processing performed by the device according to the embodiment. This figure shows an example of a word conversion prompt. This figure shows an example of the output based on the word conversion prompt. This figure shows an example of a document acquisition program. This figure shows an example of the output based on the document acquisition program. This is a sequence diagram (3) showing an example of the specific processing performed by the device according to the embodiment. This figure shows an example of a sampling response acquisition prompt. This figure shows an example of the output based on the sampling response acquisition prompt. This is a sequence diagram (4) showing an example of the specific processing performed by the device according to the embodiment. This figure shows an example of a document extraction prompt (1). This figure shows an example of a document extraction prompt (2). This figure shows an example of the output based on the document extraction prompt. This figure shows an example of a response output prompt. This figure shows an example of the output based on the response output prompt. This figure shows an example of a prompt for extracting documents in the prior art. This figure shows an example of output based on the prompt shown in Figure 24. This figure shows an example of the hardware configuration of a computer used in the apparatus according to the embodiment.
[0009] The embodiments of this disclosure will be described in detail below with reference to the drawings. In the description of the drawings, the same elements will be denoted by the same reference numerals, and redundant descriptions will be omitted. Furthermore, the embodiments of this disclosure described below are specific examples of the present invention and are not limited to these embodiments unless otherwise stated to limit the present invention.
[0010] Figure 1 is a diagram showing an example of the system configuration of a system 3 including a device 1 according to an embodiment. As shown in Figure 1, system 3 is composed of a device 1 and a terminal 2. The device 1 and the terminal 2 are connected to each other by a network such as a mobile communication network, and can send and receive information to each other. In system 3, there may be multiple terminals 2 (each used by a different user), in which case the device 1 and each terminal 2 are similarly connected to each other and can send and receive information to each other.
[0011] Device 1 is a computer device (server) that extracts the portion of a document related to a question (input information). In this embodiment, "question" may be replaced with "input information". The input information includes information about any input (user input, or input to Device 1 or the generating AI, etc.), such as questions, commands, or queries. In this embodiment, "question" will be mainly used for explanation, but it is not limited to this and may be replaced with "input information" as described above. Device 1 may also have the function of RAG (Retrieval-Augmented Generation). Device 1 may perform various processes using the generating AI LLM40 and prompts.
[0012] Generative AI is a technology that, in response to a prompt containing input information, generates content according to one or a combination of the instructions, context, questions, and output formats indicated by the prompt, and returns that content as response information. The prompt can also contain input information, in which case the generative AI generates response information based on the input information. The generative AI may be a conversational AI that includes, for example, a Large Language Model (LLM) and a User Interface (UI) for interaction with the user, enabling text chat or voice chat with the user. Examples of such generative AI include tsuzumi®, ChatGPT, GPT®-3.5, GPT-4V, and PaLM2.
[0013] The generated AI may be stored in another device connected to device 1 via a network, and configured to allow information exchange. Figure 2 shows an example of the relationship between device 1 and the generated AI. In the relationship shown in Figure 2, the generated AI, LLM40, is stored in an external server 4 connected to device 1 via a network. When needed, device 1 has LLM40 perform processing via the network, receives the execution results, and uses them.
[0014] The generated AI may be stored within device 1. Figure 3 shows a modified example of the relationship shown in Figure 2. In the modified example shown in Figure 3, the generated AI, LLM40, is stored within device 1. When necessary, device 1 executes processing on the LLM40 stored within device 1 and uses the execution result.
[0015] A prompt is information indicating instructions or questions to be entered by a user in an interactive system such as a conversation with a generating AI or a command-line interface (CLI). For example, a prompt may express, in text form, the command to be executed by the generating AI, the task to be performed by the generating AI, the background / context that the generating AI should consider (e.g., role, conditions), the question that the generating AI should answer, and the output format of the response information from the generating AI. In addition, information about the input that will be the target of the command / task to be executed by the generating AI may be added to the prompt. Such input information includes data files with filenames containing predetermined extensions, such as text data, image data, application-related data, audio data, video data, and still image data. Application-related data refers to data such as document data, tabular data, and graph data that can be processed by a default application program.
[0016] Details of device 1 will be described later.
[0017] Terminal 2 is a mobile communication terminal or a computer device such as a laptop computer that performs mobile communication. In this embodiment, terminal 2 is assumed to be a smartphone, but is not limited to this. Terminal 2 is carried by the user of terminal 2. Terminal 2 may be equipped with other sensors or functions that are typical of a smartphone.
[0018] Figure 4 shows an example of the functional configuration of the apparatus 1 according to the embodiment. As shown in Figure 4, the apparatus 1 is composed of a storage unit 10, a setting unit 11, an acquisition unit 12 (acquisition unit), a generation unit 13, a conversion unit 14, an extraction unit 15 (extraction unit), and an answer unit 16 (answer unit).
[0019] Each functional block of Device 1 is intended to function within Device 1, but is not limited to this. For example, some of the functional blocks of Device 1 may function within a computer device separate from Device 1, connected to Device 1 via a network, while appropriately sending and receiving information with Device 1. Furthermore, some functional blocks of Device 1 may be omitted, multiple functional blocks may be integrated into a single functional block, or a single functional block may be broken down into multiple functional blocks.
[0020] The following describes the functions of the device 1 shown in Figure 4.
[0021] The storage unit 10 stores arbitrary information used in calculations in the device 1, as well as the results of calculations in the device 1. The information stored in the storage unit 10 may be referenced as appropriate by each function of the device 1. The information stored in the storage unit 10 may be pre-set information, information purchased in advance, or information acquired at any time by the acquisition unit 12 described later. The storage unit 10 may also store the LLM 40 described above.
[0022] The setting unit 11 performs any settings related to the device 1 or LLM 40.
[0023] The setting unit 11 may set the randomness of the LLM40's output. The setting unit 11 may set the randomness of the LLM40's output lower than the normal setting (default setting, general setting) or higher than the normal setting. The lower the randomness setting of the LLM40 is set, the more consistent and predictable the LLM40 will produce (similar output). On the other hand, the higher the randomness setting of the LLM40 is set, the more random and unpredictable the LLM40 will produce (diverse output).
[0024] The setting unit 11 may also set Temperature, a parameter that controls the randomness of the output of the LLM 40. Temperature is represented by a real value between "0.0" and "1.0", for example, and the usual setting is "0.5". When Temperature is set to a value between "0.0" and "0.5", the LLM 40 produces a more consistent and predictable output, as described above. On the other hand, when Temperature is set to a value greater than "0.5" and less than or equal to "1.0", the LLM 40 produces a more random and unpredictable output, as described above.
[0025] The acquisition unit 12 acquires arbitrary information. The acquisition unit 12 may acquire arbitrary information used in calculations in the device 1 from other functional blocks of the device 1, including the storage unit 10, or it may acquire (receive) it from other devices via a network or the like. The acquisition unit 12 may acquire arbitrary information in advance prior to any calculations in the device 1, or it may acquire the information when it becomes necessary, in accordance with the requests of each functional block of the device 1. The acquisition unit 12 may store the acquired information in the storage unit 10, or it may output it to other functional blocks of the device 1.
[0026] The acquisition unit 12 acquires multiple answers obtained by inputting the same question multiple times to the LLM 40. More specifically, device 1 or another device may input the same question multiple times to the LLM 40, and the acquisition unit 12 may acquire the multiple answers output by the LLM 40 as responses. Alternatively, the acquisition unit 12 may input the same question multiple times to the LLM 40, and the acquisition unit 12 may acquire the multiple answers output by the LLM 40 as responses.
[0027] The question may be text or a prompt composed of text. Upon receiving a question, the LLM40 outputs an answer to that question based on the knowledge it stores internally. As described above, the randomness setting of the LLM40's output or the temperature setting of the LLM40 will determine whether the output is a consistent and predictable output or a random and unpredictable output.
[0028] Multiple answers may consist of the answers output each time the same question is input to the LLM 40 multiple times. That is, the acquisition unit 12 may acquire multiple answers consisting of the answers output each time the same question is input to the LLM 40 multiple times. More specifically, device 1 or another device may input the same question to the LLM 40 multiple times, and the acquisition unit 12 may acquire multiple answers (for those multiple inputs) consisting of the answers output by the LLM 40 as a response each time. Alternatively, the acquisition unit 12 may input the same question to the LLM 40 multiple times, and the acquisition unit 12 may acquire multiple answers (for those multiple inputs) consisting of the answers output by the LLM 40 as a response each time.
[0029] Multiple responses may be output by an LLM40 with a higher randomness setting than the normal setting. That is, the acquisition unit 12 may acquire multiple responses output by an LLM40 with a higher randomness setting than the normal setting (for example, a Temperature setting of "0.8") than the normal setting (for example, a Temperature setting of "0.5").
[0030] The acquisition unit 12 may acquire inquiries from a user (terminal 2) or other devices. The acquisition unit 12 may acquire questions (queries). The acquisition unit 12 may acquire documents. The acquisition unit 12 may perform a search and acquire the search results. The acquisition unit 12 may perform a search based on a question and acquire the search results. The acquisition unit 12 may perform a document search and acquire documents that are the search results. The acquisition unit 12 may perform a document search based on a question and acquire documents that are the search results. The acquisition unit 12 may acquire sampled responses. Details will be explained in the specific examples below.
[0031] The generation unit 13 generates queries based on the inquiry. Details will be explained in the specific examples below. The generation unit 13 may store the generated queries in the storage unit 10, output them to other functional blocks of the device 1, or output (transmit) them to other devices via the network.
[0032] The conversion unit 14 converts the query into words. Details will be explained in the specific examples below. The conversion unit 14 may store the converted words in the storage unit 10, output them to other functional blocks of the device 1, or output (transmit) them to other devices via the network.
[0033] The extraction unit 15 extracts the portion of the document related to the question based on multiple answers. More specifically, the extraction unit 15 may extract the portion of the document acquired by the acquisition unit 12 that is related to the question acquired by the acquisition unit 12, based on multiple answers obtained by inputting the same question multiple times into the LLM 40.
[0034] The documents may be obtained through a search based on a question. For example, the extraction unit 15 may extract the portion of the documents obtained by the acquisition unit 12 through a search based on a question that is relevant to the question, based on multiple answers (obtained based on the question).
[0035] The extraction by the extraction unit 15 may be based on the input of a prompt to the LLM 40 that includes an instruction to extract the portion of the document relevant to the question based on multiple answers. More specifically, device 1 or another device may input a prompt to the LLM 40 that includes an instruction to extract the portion of the document relevant to the question based on multiple answers, and the extraction unit 15 may output the portion output by the LLM 40 as the extraction result. Alternatively, the extraction unit 15 may input a prompt to the LLM 40 that includes an instruction to extract the portion of the document relevant to the question based on multiple answers, and the extraction unit 15 may output the portion output by the LLM 40 as the extraction result.
[0036] A prompt that includes instructions to extract the relevant portion of a document based on multiple answers may be generated by the extraction unit 15, or by device 1 (or any functional block thereof) or another device.
[0037] The extraction by the extraction unit 15 may be based on the input of a prompt to the LLM 40 that includes an instruction to extract the portion of the document relevant to the question after focusing on multiple answers. More specifically, device 1 or another device may input a prompt to the LLM 40 that includes an instruction to extract the portion of the document relevant to the question after focusing on multiple answers, and the extraction unit 15 may output the portion output by the LLM 40 as the extraction result. Alternatively, the extraction unit 15 may input a prompt to the LLM 40 that includes an instruction to extract the portion of the document relevant to the question after focusing on multiple answers, and the extraction unit 15 may output the portion output by the LLM 40 as the extraction result.
[0038] A prompt that includes instructions to focus on multiple answers and then extract the relevant portion of the document from the document may be generated by the extraction unit 15, or by device 1 (or any functional block thereof) or another device.
[0039] The extraction unit 15 may further extract the parts of the document surrounding the part related to the question. For example, the extraction unit 15 may further extract the n sentences before (n is a natural number of 1 or more) the part of the document related to the question, the n sentences after it, or the n sentences before and after the part related to the question.
[0040] The extraction unit 15 may cause the storage unit 10 to store the extraction result (the part of the document related to the question, the periphery of said part, etc.), may output the extraction result to another functional block of the apparatus 1, or may output (transmit) the extraction result to another apparatus via a network.
[0041] The answer unit 16 generates an answer to the question based on the part (related to the question in the document) extracted by the extraction unit 15. The answer unit 16 may generate the answer to the question based on the question and the part extracted by the extraction unit 15.
[0042] The answer unit 16 may generate the answer to the question based on the input of a prompt including an instruction to generate the answer to the question based on the part extracted by the extraction unit 15 to the LLM 40. More specifically, the apparatus 1 or another apparatus may input a prompt including an instruction to the LLM 40 to generate the answer to the question based on the part extracted by the extraction unit 15, and the answer unit 16 may output the answer output by the LLM 40 as a response thereto as a generation result. Furthermore, the answer unit 16 may input a prompt including an instruction to the LLM 40 to generate the answer to the question based on the part extracted by the extraction unit 15, and the answer unit 16 may output the answer output by the LLM 40 as a response thereto as a generation result.
[0043] The prompt including the instruction to generate the answer to the question based on the part extracted by the extraction unit 15 may be generated by the answer unit 16, or may be generated by (any functional block of) the apparatus 1 or another apparatus.
[0044] The answer unit 16 may cause the storage unit 10 to store the generated answer, may output the generated answer to another functional block of the apparatus 1, or may output (transmit) the generated answer to another apparatus (including the terminal 2) via a network.
[0045] Next, an example of extraction processing executed by the apparatus 1 will be described with reference to Figure 5. Figure 5 is a flowchart showing an example of processing executed by the apparatus 1.
[0046] First, the acquisition unit 12 acquires a plurality of responses obtained by inputting the same question to the LLM 40 (generative AI) a plurality of times (step S1, acquisition step). Next, the extraction unit 15 extracts a part related to the question from the document based on the plurality of responses acquired in step S1 (step S2, extraction step).
[0047] Subsequently, an example of specific extraction processing and response processing executed by the device 1 will be described with reference to FIG. 6. FIG. 6 is a flowchart illustrating an example of specific processing executed by the device 1.
[0048] Note that the aforementioned "question" and "document" respectively correspond to the subsequent "query" and "document".
[0049] First, the LLM 40 inputs a prompt that is an inquiry (step S10). More specifically, the acquisition unit 12 acquires the inquiry transmitted from the user (the user's terminal 2), converts the acquired inquiry into a prompt format, transmits the converted prompt to the LLM 40, and the LLM 40 acquires (inputs) the prompt.
[0050] Next, the prompt input in step S10 is transmitted to the generation unit 13, and the generation unit 13 generates a query with the LLM 40 based on the acquired prompt (step S11). More specifically, the generation unit 13 generates a query to be used for searching external information from the prompt input in step S10. Details of the query will be described later.
[0051] Next, the acquisition unit 12 acquires a document through an external search (step S12). More specifically, based on the query generated in step S11, the acquisition unit 12 (searches for and) acquires a document in which information related to the query is described from an external information source.
[0052] Next, the acquisition unit 12 performs sampling acquisition of query responses (step S13). More specifically, the acquisition unit 12 inputs the query generated in step S11 to the LLM 40 a plurality of times, and acquires candidate responses (sampling responses) in advance.
[0053] Next, the extraction unit 15 extracts relevant parts from the document (step S14). More specifically, the extraction unit 15 extracts the parts containing the elements necessary for the search from the document obtained in step S12. During this extraction, the extraction unit 15 uses a prompt into which the candidate answers obtained in step S13 have been inserted.
[0054] Next, the response unit 16 generates the query response (step S15). More specifically, the response unit 16 generates a response to the external search query generated in step S11 from the portion extracted in step S14.
[0055] Next, the response unit 16 outputs the final answer (step S16). More specifically, once the search is complete, the response unit 16 responds to the prompt entered in step S1 based on the search results (answers) obtained in step S15.
[0056] Next, with reference to Figures 7 to 23, we will explain from another perspective specific examples of the extraction and response processes performed by the device 1.
[0057] Figure 7 is a sequence diagram (part 1) showing an example of a specific process performed by device 1.
[0058] First, terminal 2 sends a query to device 1 (step S20). The content of the query is "What was the first drama series in which Ooshiba Uka played the lead role?" Next, the generation unit 13 of device 1 generates a query generation prompt based on the query received in step S20 (step S21).
[0059] Figure 8 shows an example of a query generation prompt. In the query generation prompt shown in Figure 8, the "{question}" is entered as the query acquired by the device 1 (its acquisition unit 12) in step S20: "What was Ooshiba Uka's first starring role in a television drama series?"
[0060] Next, the generation unit 13 sends the query generation prompt generated in step S21 to the LLM 40 (step S22). The LLM 40 then executes the query generation prompt received in step S22 by inputting it, and outputs the output query (hereinafter simply referred to as "query") "What was Ooshiba Uka's first starring role in a TV drama series?". Figure 9 shows an example of the output based on the query generation prompt. Next, the LLM 40 sends the output query to the device 1 (generation unit 13) (as a response to the transmission in step S22) (step S23).
[0061] In this example, the query in step S20 and the query in step S23 are identical because the query is simple, but different outputs would be obtained if the query were more complex. The query in step S20 is the question the user wants to know. The query in step S23 is a question that searches for the information necessary to answer that question.
[0062] Let's look at an example where the inquiry in step S20 and the query in step S23 are different. If the inquiry (user question) is "If we give 1,000 yen to every resident of Shizuoka Prefecture, will 100 million yen be enough?", then knowing the number of residents of Shizuoka Prefecture will allow us to answer the above question. Therefore, using the query generation prompt to output the query will output "What is the number of residents of Shizuoka Prefecture?".
[0063] Figure 10 is a sequence diagram (part 2) showing an example of a specific process performed by device 1. The sequence diagram in Figure 10 is a continuation of the sequence diagram in Figure 7. That is, following step S23 in the sequence diagram in Figure 7, step S30 in the sequence diagram in Figure 10 is executed.
[0064] First, the conversion unit 14 generates a word conversion prompt (step S30) based on the query acquired (by the acquisition unit 12) in step S23.
[0065] FIG. 11 is a diagram illustrating an example of a word conversion prompt. In "{query}" of the word conversion prompt shown in FIG. 11, "What is the first lead drama work of Urara Oshiba?", which is the query acquired in step S23, is input.
[0066] Next, the conversion unit 14 transmits the word conversion prompt generated in step S30 to the LLM 40 (step S31). Next, the LLM 40 executes processing by inputting the word conversion prompt received in step S31, and outputs "Urara Oshiba" as a word. FIG. 12 is a diagram illustrating an example of an output based on the word conversion prompt. Next, the LLM 40 transmits the output word to the apparatus 1 (the conversion unit 14 thereof) (as a response to the transmission in step S31) (step S32).
[0067] Note that in this specific example, since the API of Wikipedia (registered trademark) is used, a method of performing search on a word basis is assumed, and thus the word conversion prompt is used, but the present disclosure is not limited thereto. For example, any other arbitrary method such as a method of calculating the similarity between a document and a query after vectorization may be used.
[0068] Further, the processing from step S30 to step S32 is processing for facilitating search, and even if an accurate Wikipedia (registered trademark) page name cannot be obtained here, a page close to the accurate page can be acquired when the API is used.
[0069] Next, the acquisition unit 12 executes the Wikipedia (registered trademark) API based on the word acquired in step S32 (step S33), and receives, as a response therefrom, a document that is the full text of a Wikipedia (registered trademark) page related to the word from Wikipedia (registered trademark) (step S34).
[0070] FIG. 13 is a diagram illustrating an example of a document acquisition program. The document acquisition program illustrated in FIG. 13 is written in Python (registered trademark). The word acquired in step 32 is assigned to "word" of the document acquisition program illustrated in FIG. 13, and a document is assigned to "line".
[0071] FIG. 14 is a diagram showing an example of output based on a document acquisition program. The output shown in FIG. 14 presents a document (the full text of a Wikipedia (registered trademark) page) related to the word "Yuka Oshiba" acquired in step 32.
[0072] FIG. 15 is a sequence diagram (part 3) showing an example of specific processing executed by the device 1. The sequence diagram shown in FIG. 15 is a continuation of the sequence diagram shown in FIG. 10. That is, following step S34 of the sequence diagram shown in FIG. 10, step S40 of the sequence diagram shown in FIG. 15 is executed.
[0073] First, the generation unit 13 generates a sampling answer acquisition prompt based on the query acquired (by the acquisition unit 12) in step S23 (step S40).
[0074] FIG. 16 is a diagram showing an example of the sampling answer acquisition prompt. Into "{query}" in the sampling answer acquisition prompt shown in FIG. 16, "What is the first TV drama series starring Yuka Oshiba?" which is the query acquired in step S23, is input.
[0075] Next, the generation unit 13 transmits the sampling answer acquisition prompt generated in step S40 to the LLM 40 (step S41). Next, the LLM 40 executes (for the first time) by inputting the sampling answer acquisition prompt received in step S41, and outputs "Sampling Answer: 0" as 'The first TV drama series starring Yuka Oshiba is "Shuriken x Monk Maji Maji Ninja!"'. Next, the LLM 40 transmits the output "Sampling Answer: 0" (as a response to the transmission in step S41) to (the generation unit 13 of) the device 1 (step S42).
[0076] Next, the generation unit 13 sends the sampling response acquisition prompt generated in step S40 to the LLM 40 (step S43). Next, the LLM 40 executes (for the second time) by inputting the sampling response acquisition prompt received in step S43, and outputs "Oshiba Yuka's first starring role in a TV drama series is 'Tokutokugigigi'." as "Sampling Response: 1". Next, the LLM 40 sends the outputted "Sampling Response: 1" to the device 1 (generation unit 13) (as a response to the transmission in step S43) (step S44).
[0077] The generation unit 13 may also obtain 10 sampled responses by repeatedly inputting a sampling response acquisition prompt to the LLM 40 by raising the Temperature of the LLM 40 (for example, to "0.8") 10 times.
[0078] Figure 17 shows an example of output based on the sampling response acquisition prompt. In the output shown in Figure 17, each sampling response acquired by the generation unit 13 from the LLM 40 is arranged in the order it was acquired.
[0079] Figure 18 is a sequence diagram (part 4) showing an example of a specific process performed by device 1. The sequence diagram in Figure 18 is a continuation of the sequence diagram in Figure 15. That is, following step S44 of the sequence diagram in Figure 15 (more precisely, after device 1 has acquired all the sampling responses (for example, 10 times)), step S50 of the sequence diagram in Figure 18 is executed.
[0080] First, the generation unit 13 generates a document extraction prompt (step S50) based on the query obtained in step S23 (by the acquisition unit 12), the document obtained in step S34 (by the acquisition unit 12), and the sampled response obtained in steps S42 and S44 (by the generation unit 13).
[0081] Figure 19 shows an example of a document extraction prompt (part 1). Figure 20 shows an example of a document extraction prompt (part 2). The document extraction prompt shown in Figures 19 and 20 is a single entity, with the first part shown in Figure 19 and the second part in Figure 20. In other words, the document extraction prompt shown in Figure 20 is a continuation of the document extraction prompt shown in Figure 19.
[0082] In the document extraction prompt shown in Figure 20, the query "What was the first TV drama series in which Yuka Oshiba played the lead role?" is entered into "{query}", which was obtained in step S23. The document obtained in step S34 and the sampled answers obtained in steps S42 and S44, etc., are already embedded in the document extraction prompt shown in Figure 20 (see the "# Wikipedia page" and "# sampled answer" parts in the figure, respectively).
[0083] Next, the generation unit 13 sends the document extraction prompt generated in step S50 to the LLM 40 (step S51). Next, the LLM 40 executes the document extraction prompt received in step S51 by inputting it, and outputs the extracted portion (see Figure 21, described later). Next, the LLM 40 sends the outputted extracted portion to the device 1 (generation unit 13) (as a response to the transmission in step S51) (step S52).
[0084] Figure 21 shows an example of output based on the document extraction prompt. In the output shown in Figure 21, the portion of the document obtained in step S34 that relates to the query obtained in step S23 (the extracted portion) has been extracted.
[0085] Next, the generation unit 13 generates a response output prompt (step S53) based on the query obtained in step S23 (by the acquisition unit 12) and the extracted portion obtained in step S52.
[0086] Figure 22 shows an example of a response output prompt. In the response output prompt shown in Figure 22, the query obtained in step S23, "What was the first drama series in which Ooshiba Uka played the lead role?", is entered into "{query}". The extracted portion obtained in step S52 is already embedded in the response output prompt shown in Figure 22 (see "# Extraction Result" in the figure).
[0087] Next, the generation unit 13 sends the response output prompt generated in step S53 to the LLM 40 (step S54). Next, the LLM 40 executes by inputting the response output prompt received in step S54 and outputs the response, "Tokutokugigigi". Next, the LLM 40 sends the outputted response to the device 1 (generation unit 13) (as a response to the transmission in step S54) (step S55).
[0088] Figure 23 shows an example of output based on the response output prompt. In the output shown in Figure 23, the (correct) response to the query obtained in step S23 is "Tokutokugigigi".
[0089] Next, the response unit 16 sends the response obtained in step S55 (by the generation unit 13) to the terminal 2 (as a response to the inquiry in step S20 of Figure 7) (step S56), and the response is displayed to the user, for example, on the screen of the terminal 2. The displayed response is the (correct) response to the inquiry in step S20 of Figure 7.
[0090] Next, the processing of the prior art will be briefly explained with reference to Figures 24 and 25. Figure 24 shows an example of a prompt for extracting documents in the prior art. In the prompt shown in Figure 24, there are no sampling responses that appeared in the processing of device 1. Figure 25 shows an example of output based on the prompt shown in Figure 24. As shown in the output in Figure 25, the LLM was unable to find the part of the document related to the query, and "Not described." was output.
[0091] Next, the operation and effects of the apparatus 1 according to this embodiment will be described.
[0092] Device 1 comprises an acquisition unit 12 that acquires multiple answers obtained by inputting the same question multiple times to LLM 40 (Generating AI), and an extraction unit 15 that extracts the portion of a document related to the question based on the multiple answers. With this configuration, the portion of a document related to the question is extracted based on the multiple answers obtained by inputting the same question multiple times to LLM 40. This makes it possible to extract the portion of a document related to the question. Furthermore, because the extraction is based on multiple answers obtained by inputting the same question multiple times to LLM 40, the extraction can be performed with greater reliability. In addition, for example, the LLM 40 can be guided by the multiple answers obtained by inputting the same question multiple times to LLM 40, which can suppress oversights and make the extraction more reliable. By extracting the portion of a document related to the question, for example, a user can easily and quickly find information related to the question in the document. Furthermore, by extracting the portion of a document related to the question, for example, another process using that portion can be performed (automatically).
[0093] In device 1, the multiple responses may consist of the responses output each time the same question is input to the LLM 40 multiple times. With this configuration, for example, the multiple responses consisting of the responses output each time the same question is input to the LLM 40 multiple times will be diverse responses, and therefore extraction that takes diversity into account can be performed.
[0094] In device 1, multiple responses may be output by an LLM40 set to have a higher randomness of output than the normal setting. With this configuration, the multiple responses will be more random (or diverse) than with the normal setting, and therefore, sampling that takes randomness (or diversity) into account can be performed.
[0095] In device 1, the documents may be obtained through a search based on a question. With this configuration, for example, the portion of the document related to the question obtained through the search that is relevant to that question can be extracted, thus allowing the portion relevant to the question to be extracted.
[0096] The extraction by the extraction unit 15 of the device 1 may be based on inputting a prompt to the LLM 40 that includes an instruction to extract the portion of the document relevant to the question based on multiple answers. This configuration allows for more flexible extraction, for example, by utilizing the flexibility of the LLM 40.
[0097] The extraction by the extraction unit 15 of the device 1 may be based on input to the LLM 40 of a prompt that includes an instruction to extract the part of the document related to the question after focusing on multiple answers. This configuration allows for more flexible extraction, for example, by utilizing the flexibility of the LLM 40. Furthermore, by having the LLM 40 focus on multiple answers, for example, the LLM 40 can be directed to multiple answers, thereby suppressing oversights by the LLM 40 and enabling more reliable extraction.
[0098] The extraction unit 15 of the device 1 may further extract information surrounding the portion of the document related to the question. Since the area surrounding the portion of the document related to the question is likely to contain information that is relevant to that portion, this configuration allows for the extraction of, for example, more useful information.
[0099] The device 1 may further include an answering unit 16 that generates an answer to a question based on the portion extracted by the extraction unit 15. With this configuration, since the answer to the question is generated based on the portion of the document relevant to the question, it is possible to generate a more appropriate answer compared to simply generating an answer to a question, for example, because it is based on the portion of the document relevant to the question.
[0100] The device 1 may further include an answering unit 16 that generates an answer to a question based on the input of a prompt to the LLM 40, which includes an instruction to generate an answer to the question based on the portion extracted by the extraction unit 15. This configuration allows for the generation of more flexible answers, for example, by taking advantage of the flexibility of the LLM 40.
[0101] As background, while LLM can store knowledge within the model, it faces the challenge of hallucination occurring when there is no knowledge about the input to the LLM, or when there is incorrect knowledge. Hallucination is extremely harmful because it outputs incorrect knowledge and answers to the user. Therefore, conventional methods have been used to suppress hallucination by obtaining information from external sources. In this case, when extracting information related to questions and inferences from external sources in the LLM, it is necessary to extract the relevant parts if the obtained document is long.
[0102] On the other hand, a challenge is that, in the case of long texts, even if information related to the question and inference is included in the text, there are cases where LLM fails to extract it correctly.
[0103] Device 1 is a summarization support system that uses sampling. When extracting necessary parts from long texts obtained using queries, such as in external searches, Device 1 uses a method that facilitates extraction by obtaining candidate answers using sampling. Device 1 provides information extraction support by setting a high value (e.g., 1.0) to the Temperature of the Generating AI (LLM) to obtain multiple answers to a question.
[0104] Device 1, when extracting information related to a question using a generating AI (LLM), employs a method (repeated sampling) that obtains multiple answers to a question by setting a high value (e.g., 1.0) for Temperature. By acquiring potentially relevant sentences as candidates, the device focuses on those parts for extraction, thereby improving the success rate of information extraction compared to conventional methods.
[0105] According to device 1, by sampling answers to questions using the Generative AI (LLM), the Generative AI (LLM) can focus on the sampled answers while processing long texts. This makes it possible to extract appropriate information even in cases where conventional methods fail to extract information effectively.
[0106] The apparatus 1 of this disclosure may have the following configuration.
[0107] [1] An apparatus comprising: an acquisition unit that acquires multiple responses obtained by inputting the same input information multiple times to a generating AI; and an extraction unit that extracts a portion of a document related to the input information based on the multiple responses.
[0108] [2] The apparatus according to [1], wherein the plurality of responses consist of responses output each time by inputting the same input information to the generating AI multiple times.
[0109] [3] The apparatus according to [1] or [2], wherein the multiple responses are output by the generating AI, which is set to have a higher randomness of output than the normal setting.
[0110] [4] The document is obtained by a search based on the input information, and is the apparatus described in any one of [1] to [3].
[0111] [5] The apparatus according to any one of [1] to [4], wherein the extraction of the extraction unit is based on input to a prompt generation AI that includes an instruction to extract the portion of the document related to the input information based on the multiple answers.
[0112] [6] The apparatus according to any one of [1] to [5], wherein the extraction of the extraction unit is based on input to a prompt generation AI that includes an instruction to extract the portion of the document related to the input information after focusing on the multiple answers.
[0113] [7] The apparatus according to any one of [1] to [6], wherein the extraction unit further extracts the area surrounding the portion of the document related to the input information.
[0114] [8] The apparatus according to any one of [1] to [7], further comprising a response unit that generates a response to the input information based on the portion extracted by the extraction unit.
[0115] [9] The apparatus according to any one of [1] to [8], further comprising a response unit that generates a response to the input information based on input to a prompt generation AI that includes an instruction to generate a response to the input information based on the portion extracted by the extraction unit.
[0116] The block diagrams used in the description of the above embodiments show functional units. These functional blocks (components) are realized by any combination of at least one of hardware and software. Furthermore, the method of realizing each functional block is not particularly limited. That is, each functional block may be realized using one device that is physically or logically coupled, or it may be realized using two or more physically or logically separated devices that are directly or indirectly connected (for example, using wired or wireless connections). A functional block may also be realized by combining software with the one or more of the above devices.
[0117] Functions include, but are not limited to, judgment, decision, determination, calculation, calculation, processing, derivation, investigation, exploration, confirmation, reception, transmission, output, access, resolution, selection, selection, establishment, comparison, assumption, expectation, assumption, broadcasting, notifying, communicating, forwarding, configuring, reconfiguring, allocating (mapping), and assigning. For example, a functional block (configuration part) that enables transmission is called a transmitting unit or transmitter. In all cases, as mentioned above, the method of implementation is not particularly limited.
[0118] For example, Apparatus 1 in one embodiment of the present disclosure may function as a computer that processes the extraction method and response method of the present disclosure. Figure 26 is a diagram showing an example of the hardware configuration of Apparatus 1 according to one embodiment of the present disclosure. Physically, Apparatus 1 may be configured as a computer device including a processor 1001, memory 1002, storage 1003, communication device 1004, input device 1005, output device 1006, bus 1007, etc.
[0119] In the following explanation, the term "device" can be replaced with "circuit," "device," "unit," etc. The hardware configuration of device 1 may include one or more of the devices shown in the figure, or it may be configured to omit some of the devices.
[0120] Each function in device 1 is realized by loading predetermined software (programs) onto hardware such as the processor 1001 and memory 1002, which allows the processor 1001 to perform calculations, control communication by the communication device 1004, and control at least one of data reading and writing in the memory 1002 and storage 1003.
[0121] The processor 1001 controls the entire computer, for example, by running an operating system. The processor 1001 may be composed of a central processing unit (CPU) that includes interfaces with peripheral devices, control devices, arithmetic units, registers, etc. For example, the setting unit 11, acquisition unit 12, generation unit 13, conversion unit 14, extraction unit 15, and answer unit 16 described above may be implemented by the processor 1001.
[0122] Furthermore, the processor 1001 reads programs (program code), software modules, data, etc., from at least one of the storage 1003 and the communication device 1004 into the memory 1002 and executes various processes accordingly. The program used is one that causes the computer to execute at least a part of the operations described in the above embodiment. For example, the setting unit 11, acquisition unit 12, generation unit 13, conversion unit 14, extraction unit 15, and answer unit 16 may be stored in the memory 1002 and implemented by a control program that operates on the processor 1001, and other functional blocks may be implemented similarly. The above-described various processes have been explained as being executed by one processor 1001, but they may be executed simultaneously or sequentially by two or more processors 1001. The processor 1001 may be implemented by one or more chips. The program may also be transmitted from a network via a telecommunications line.
[0123] The memory 1002 is a computer-readable recording medium and may consist of at least one of the following: ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable ROM), RAM (Random Access Memory), etc. The memory 1002 may also be called a register, cache, main memory, etc. The memory 1002 can store executable programs (program code), software modules, etc., for carrying out a wireless communication method according to one embodiment of the present disclosure.
[0124] The storage 1003 is a computer-readable recording medium and may consist of at least one of the following: an optical disc such as a CD-ROM (Compact Disc ROM), a hard disk drive, a flexible disk, a magneto-optical disk (e.g., Compact Disc, Digital Multipurpose Disc, Blu-ray® Disc), a smart card, flash memory (e.g., a card, stick, key drive), a floppy® disk, a magnetic strip, etc. The storage 1003 may also be called an auxiliary storage device. The above-mentioned storage medium may be, for example, a database, server, or other suitable medium including at least one of memory 1002 and storage 1003.
[0125] The communication device 1004 is hardware (transceiver / receiver device) for communicating between computers via at least one of a wired network and a wireless network, and is also referred to as a network device, network controller, network card, communication module, etc. The communication device 1004 may be configured to include, for example, a high-frequency switch, duplexer, filter, frequency synthesizer, etc., in order to implement at least one of frequency division duplex (FDD) and time division duplex (TDD). For example, the setting unit 11, acquisition unit 12, generation unit 13, conversion unit 14, extraction unit 15, and response unit 16 described above may be implemented by the communication device 1004.
[0126] The input device 1005 is an input device that accepts input from an external source (e.g., a keyboard, mouse, microphone, switch, button, sensor, etc.). The output device 1006 is an output device that outputs to an external source (e.g., a display, speaker, LED lamp, etc.). The input device 1005 and the output device 1006 may be configured as an integrated unit (e.g., a touch panel).
[0127] Furthermore, each device, such as the processor 1001 and memory 1002, is connected by a bus 1007 for communicating information. The bus 1007 may be configured using a single bus, or different buses may be configured for each device.
[0128] Furthermore, the device 1 may be configured to include hardware such as a microprocessor, a digital signal processor (DSP), an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), and an FPGA (Field Programmable Gate Array), and some or all of each functional block may be realized by such hardware. For example, the processor 1001 may be implemented using at least one of these hardware components.
[0129] The notification of information is not limited to the manner / embodiments described herein and may be carried out by other means.
[0130] Each aspect / embodiment described in this disclosure may be applied to at least one of the following systems: LTE (Long Term Evolution), LTE-A (LTE-Advanced), SUPER 3G, IMT-Advanced, 4G (4th generation mobile communication system), 5G (5th generation mobile communication system), FRA (Future Radio Access), NR (new Radio), W-CDMA®, GSM®, CDMA2000, UMB (Ultra Mobile Broadband), IEEE 802.11 (Wi-Fi®), IEEE 802.16 (WiMAX®), IEEE 802.20, UWB (Ultra-WideBand), Bluetooth®, and other appropriate systems, as well as next-generation systems extended based thereon. Furthermore, multiple systems may be applied in combination (for example, a combination of at least one of LTE and LTE-A with 5G).
[0131] The processing procedures, sequences, flowcharts, etc., of each aspect / embodiment described in this disclosure may be reordered, provided they do not contradict each other. For example, the methods described in this disclosure present various step elements using exemplary order and are not limited to the specific order presented.
[0132] Input and output information may be stored in a specific location (e.g., memory) or managed using a management table. Input and output information may be overwritten, updated, or appended to. Output information may be deleted. Input information may be transmitted to other devices.
[0133] The determination may be made by a value represented by one bit (0 or 1), by a boolean value (true or false), or by a numerical comparison (for example, a comparison with a predetermined value).
[0134] Each aspect / embodiment described in this disclosure may be used individually, in combination, or switched between as needed during implementation. Furthermore, notification of specific information (e.g., notification that "X is") is not limited to explicit notification, but may also be implicit (e.g., by not providing such notification).
[0135] Although the present disclosure has been described in detail above, it will be clear to those skilled in the art that the present disclosure is not limited to the embodiments described herein. The present disclosure can be implemented in modified and altered forms without departing from the intent and scope of the present disclosure as defined by the claims. Therefore, the descriptions in the present disclosure are illustrative and not intended to be restrictive in any way.
[0136] Software should be broadly interpreted to mean instructions, instruction sets, code, code segments, program code, programs, subprograms, software modules, applications, software applications, software packages, routines, subroutines, objects, executable files, execution threads, procedures, functions, and so on, whether they are called software, firmware, middleware, microcode, hardware description languages, or by any other name.
[0137] Furthermore, software, instructions, information, etc., may be transmitted and received via a transmission medium. For example, if software is transmitted from a website, server, or other remote source using at least one of wired technology (such as coaxial cable, fiber optic cable, twisted pair, or digital subscriber line (DSL)) and wireless technology (such as infrared or microwave), then at least one of these wired and wireless technologies is included in the definition of a transmission medium.
[0138] The information, signals, etc. described in this disclosure may be represented using any of the various different techniques. For example, the data, instructions, commands, information, signals, bits, symbols, chips, etc. that may be referred to throughout the above description may be represented by voltage, current, electromagnetic waves, magnetic fields or magnetic particles, optical fields or photons, or any combination thereof.
[0139] In addition, terms used in this disclosure and terms necessary for understanding this disclosure may be replaced with terms having the same or similar meaning.
[0140] The terms “system” and “network” as used in this disclosure are interchangeable.
[0141] Furthermore, the information, parameters, etc., described in this disclosure may be expressed using absolute values, relative values from a predetermined value, or corresponding other information.
[0142] The names used for the parameters described above are not restrictive in any way. Furthermore, the formulas and other expressions using these parameters may differ from those expressly disclosed in this disclosure.
[0143] As used in this disclosure, the terms “determining” and “decision” may encompass a wide variety of actions. “Determining” may include, for example, judging, calculating, computing, processing, deriving, investigating, looking up, search, inquiry (e.g., searching in tables, databases or other data structures), and ascertaining. “Determining” may also include receiving (e.g., receiving information), transmitting (e.g., sending information), input, output, and accessing (e.g., accessing data in memory). Furthermore, “determining” may include resolving, selecting, choosing, establishing, and comparing. In other words, "judgment" and "decision" can include considering that some action has been "judged" or "decided." Also, "judgment (decision)" can be reinterpreted as "assuming," "expecting," or "considering."
[0144] The terms “connected,” “coupled,” and any variations thereof mean any direct or indirect connection or coupling between two or more elements, and may include the presence of one or more intermediate elements between two elements that are “connected” or “coupled” with each other. The coupling or connection between elements may be physical, logical, or a combination thereof. For example, “connection” may be reinterpreted as “access.” As used in this disclosure, two elements may be considered to be “connected” or “coupled” with each other using at least one of one or more wires, cables, and printed electrical connections, and, in some non-limiting and non-exclusive examples, electromagnetic energy having wavelengths in the radio frequency domain, microwave domain, and optical (both visible and invisible) domain.
[0145] In this disclosure, the phrase "based on" does not mean "based solely on" unless otherwise specified. In other words, the phrase "based on" means both "based solely on" and "based at least on."
[0146] Any reference to elements using the designations “first,” “second,” etc., as used in this disclosure does not generally limit the quantity or order of those elements. These designations may be used in this disclosure as a convenient way to distinguish between two or more elements. Accordingly, references to the first and second elements do not imply that only two elements may be employed, or that the first element must precede the second element in any way.
[0147] In the configuration of each of the above devices, "means" may be replaced with "part," "circuit," "device," etc.
[0148] Where the terms “include,” “including,” and variations thereof are used in this disclosure, these terms are intended to be inclusive, as is the term “comprising.” Furthermore, the term “or” as used in this disclosure is not intended to mean exclusive OR.
[0149] In this disclosure, if articles are added by translation, such as a, an, and the in English, this disclosure may include the fact that the noun following these articles is plural.
[0150] In this disclosure, the term "A and B are different" may mean "A and B are different from each other." The term may also mean "A and B are each different from C." Terms such as "separate" and "combine" may be interpreted similarly to "different."
[0151] 1...Device, 2...Terminal, 3...System, 4...External Server, 10...Storage Unit, 11...Setting Unit, 12...Acquisition Unit, 13...Generation Unit, 14...Conversion Unit, 15...Extraction Unit, 16...Response Unit, 40...LLM, 1001...Processor, 1002...Memory, 1003...Storage, 1004...Communication Device, 1005...Input Device, 1006...Output Device, 1007...Bus.
Claims
1. An apparatus comprising: an acquisition unit that acquires multiple responses obtained by inputting the same input information multiple times to a generating AI; and an extraction unit that extracts a portion of a document related to the input information based on the multiple responses.
2. The apparatus according to claim 1, wherein the plurality of responses consist of responses output each time the same input information is input multiple times to the generating AI.
3. The apparatus according to claim 1, wherein the multiple responses are output by the generating AI, which is set to have a higher randomness of output than the normal setting.
4. The apparatus according to claim 1, wherein the document is obtained by a search based on the input information.
5. The apparatus according to claim 1, wherein the extraction of the extraction unit is based on input to a prompt generation AI that includes an instruction to extract the portion of the document related to the input information based on the plurality of answers.
6. The apparatus according to claim 1, wherein the extraction of the extraction unit is based on input to a prompt generation AI that includes an instruction to extract the portion of the document related to the input information after focusing on the plurality of answers.
7. The apparatus according to claim 1, wherein the extraction unit further extracts the area surrounding the portion of the document related to the input information.
8. The apparatus according to claim 1, further comprising a response unit that generates a response to the input information based on the portion extracted by the extraction unit.
9. The apparatus according to claim 1, further comprising a response unit that generates a response to the input information based on input to a prompt generation AI that includes an instruction to generate a response to the input information based on the portion extracted by the extraction unit.
10. A method performed by a computer, comprising: an acquisition step of obtaining multiple responses obtained by inputting the same input information multiple times to a generating AI; and an extraction step of extracting a portion of a document related to the input information based on the multiple responses.
