Information display device, information display method, and program

Abstract

To provide technology that can provide highly accurate answers to prompts. [Solution] An information presentation device according to one aspect of the present disclosure includes: an evaluation unit that evaluates the degree to which a highly accurate answer can be obtained from a first prompt given by a user; a search unit that searches for a second prompt to improve the first prompt if the evaluation value representing the result of the evaluation is below a predetermined threshold; an improvement unit that creates a third prompt which is an improved version of the first prompt based on the second prompt so as to provide a more accurate answer than the first prompt; an answer generation unit that generates an answer to the third prompt based on the third prompt and a large-scale language model; and a presentation unit that presents the answer to the user.

Description

Technical Field

[0004] ,

[0006] , , , , , ,

[0005] , , ,

[0003] , , , ,

[0001] The present disclosure relates to an information presentation device, an information presentation method, and a program.

Background Art

[0002] In recent years, language models called large language models (LLMs) have been utilized in various business fields. For example, Patent Document 1 discloses a technique in which a large language model is utilized for an automatic voice response used in a call center or the like, a response text for an input text is generated by the large language model, and then the response text is further corrected.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, when generating an accurate answer to a certain question, instruction, or command (these are also called "prompts") using a large language model, it is necessary to create a prompt that clearly includes the content of the question, instruction, or command. Therefore, when the user's knowledge, skills, and proficiency regarding prompts are low, an accurate answer may not be obtained.

[0005] <00​​​​​​​An information presentation device according to one aspect of the present disclosure includes: an evaluation unit that evaluates the degree to which a first prompt given by a user can provide a more accurate answer; a search unit that searches for a second prompt to improve the first prompt if the evaluation value representing the result of the evaluation is below a predetermined threshold; an improvement unit that creates a third prompt based on the second prompt so as to provide a more accurate answer than the first prompt; an answer generation unit that generates an answer to the third prompt based on the third prompt and a large-scale language model; and a presentation unit that presents the answer to the user. [Effects of the Invention]

[0007] A technology is provided that can provide highly accurate answers to prompts. [Brief explanation of the drawing]

[0008] [Figure 1] This figure shows an example of the overall configuration of the information presentation system according to this embodiment. [Figure 2] This figure shows an example of the hardware configuration of the information display device according to this embodiment. [Figure 3] This figure shows an example of the functional configuration of the information display device according to this embodiment. [Figure 4] This flowchart shows an example of the improved model construction process according to this embodiment. [Figure 5] This figure shows an example of a prompt for identifying the topic to which the input data belongs. [Figure 6] This figure shows an example of a prompt for evaluating input data belonging to a certain topic. [Figure 7] This figure shows an example of a knowledge graph. [Figure 8] This flowchart shows an example of information presentation processing according to this embodiment. [Figure 9] This figure shows an example of prompts for refining the input data. [Modes for carrying out the invention]

[0009] The following describes one embodiment of the present invention in detail with reference to the drawings. In the following embodiment, an information presentation system 1 is described that can provide accurate answers to prompts entered by a user, using a technology called RAG (Retrieval-Augmented Generation) which combines a large-scale language model and search. An accurate answer is an answer that contains the information desired by the user who entered the prompt. However, targeting RAG is just one example and is not limited to it.

[0010] A large-scale language model is a language model that has been trained using a machine learning or statistical model, including neural networks, with a large amount of text data and deep learning techniques.

[0011] <Example of the overall configuration of Information Presentation System 1> Figure 1 shows an example of the overall configuration of the information presentation system 1 according to this embodiment. As shown in Figure 1, the information presentation system 1 according to this embodiment includes an information presentation device 10, an administrator terminal 20, and one or more user terminals 30. The information presentation device 10, the administrator terminal 20, and each user terminal 30 are connected to each other via a communication network 40, such as the Internet.

[0012] The information presentation device 10 generates a response to an input prompt (hereinafter also referred to as an "input prompt") given by a user using the user terminal 30, using RAG. At this time, the information presentation device 10 improves (or "enhances") the input prompt as necessary, and then generates a response using the improved input prompt. The information presentation device 10 then presents the response to the user. Here, improving (or enhancing) a prompt means modifying, changing, or recreating the prompt in order to obtain a more accurate response. A prompt is a natural language sentence that represents a question, instruction, command, etc., to a large-scale language model.

[0013] The administrator terminal 20 is a type of terminal used by the administrator of the information display device 10. For example, the administrator terminal 20 can be a PC (personal computer), smartphone, tablet, wearable device, or other type of terminal device.

[0014] The user terminal 30 is a type of terminal used by the user of the information display device 10. For example, the user terminal 30 can be a PC, smartphone, tablet, wearable device, game console, digital home appliance, or in-vehicle device.

[0015] Note that the overall configuration of the information presentation system 1 shown in Figure 1 is just one example and is not limited to it. For example, the information presentation system 1 may include multiple administrator terminals 20, or at least one user terminal 30 may function as an administrator terminal 20.

[0016] <Example of hardware configuration for information display device 10> FIG. 2 is a diagram showing an example of the hardware configuration of the information presentation device 10 according to the present embodiment. As shown in FIG. 2, the information presentation device 10 according to the present embodiment includes an input device 101, a display device 102, an external I / F 103, a communication I / F 104, a RAM (Random Access Memory) 105, a ROM (Read Only Memory) 106, an auxiliary storage device 107, and a processor 108. These hardware components are communicably connected to each other via a bus 109.

[0017] The input device 101 is, for example, a keyboard, a mouse, a touch panel, a physical button, or the like. The display device 102 is, for example, a display, a display panel, or the like. Note that the information presentation device 10 may not have at least one of the input device 101 and the display device 102.

[0018] The external I / F 103 is an interface with an external device such as a recording medium 103a. Examples of the recording medium 103a include a CD (Compact Disc), a DVD (Digital Versatile Disk), an SD memory card (Secure Digital memory card), a USB (Universal Serial Bus) memory card, and the like.

[0019] The communication I / F 104 is an interface for connecting to a communication network 40. The RAM 105 is a volatile semiconductor memory (storage device) that temporarily holds programs and data. The ROM 106 is a non-volatile semiconductor memory (storage device) that can hold programs and data even when the power is turned off. The auxiliary storage device 107 is a non-volatile storage device such as, for example, an HDD (Hard Disk Drive), an SSD (Solid State Drive), or a flash memory. The processor 108 is an arithmetic device such as, for example, a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit).

[0020] Note that the hardware configuration shown in Figure 2 is just one example, and the hardware configuration of the information display device 10 is not limited to this. For example, the information display device 10 may have multiple auxiliary storage devices 107 or multiple processors 108, it may not have some of the hardware shown, or it may have various other hardware components besides the hardware shown.

[0021] <Example of the functional configuration of the information display device 10> Figure 3 is a diagram showing an example of the functional configuration of the information presentation device 10 according to this embodiment. As shown in Figure 3, the information presentation device 10 according to this embodiment has an improved model construction unit 110 and an information presentation unit 120. Each of these units is realized, for example, by a process in which one or more programs installed in the information presentation device 10 are executed by a processor 108 or the like. The information presentation device 10 according to this embodiment also has a prompt storage unit 130, an improved model storage unit 140, and a business information storage unit 150. Each of these storage units is realized, for example, by a storage area such as an auxiliary storage device 107. However, at least one of these storage units may be realized by a storage area such as a storage device (e.g., a storage device provided by a database server) that is communicably connected to the information presentation device 10.

[0022] The improved model construction unit 110 constructs a model (hereinafter also called the "improved model") for searching for prompts used to improve input prompts (hereinafter also called "improvement prompts") in response to construction instructions given from the administrator terminal 20. The improved model construction unit 110 includes a construction instruction acquisition unit 111, a history acquisition unit 112, an evaluation unit 113, and a model construction unit 114.

[0023] The construction instruction acquisition unit 111 acquires construction instructions given from the administrator terminal 20. When the construction instruction acquisition unit 111 acquires construction instructions, the history acquisition unit 112 acquires prompts stored in the prompt storage unit 130 (i.e., a history of past input prompts). The evaluation unit 113 evaluates the degree to which a more accurate response can be obtained from the prompts acquired by the history acquisition unit 112. The model construction unit 114 constructs an improved model based on the prompts acquired by the history acquisition unit 112 and the evaluation by the evaluation unit 113. The model construction unit 114 also saves the improved model in the improved model storage unit 140.

[0024] The information presentation unit 120 generates and presents an answer to an input prompt given by the user terminal 30. At this time, the information presentation unit 120 improves the input prompt as necessary and generates an answer using the improved input prompt. The information presentation unit 120 includes a prompt acquisition unit 121, an evaluation unit 122, a prompt search unit 123, an improvement unit 124, a business information search unit 125, an answer generation unit 126, and a presentation unit 127.

[0025] The prompt acquisition unit 121 acquires the input prompt given by the user terminal 30. The prompt acquisition unit 121 also stores the input prompt in the prompt storage unit 130. The evaluation unit 122 evaluates the degree to which an accurate response can be obtained from the input prompt acquired by the prompt acquisition unit 121. The prompt search unit 123 searches for an improved prompt based on the improved model stored in the improved model storage unit 140. The improvement unit 124 improves the input prompt acquired by the prompt acquisition unit 121 based on the improved prompt found by the prompt search unit 123. The business information search unit 125 searches the business information storage unit 150 for business information related to the input prompt or the improved input prompt based on the input prompt acquired by the prompt acquisition unit 121 or the improved input prompt by the improvement unit 124. The response generation unit 126 generates a response to the input prompt or the improved input prompt using a large-scale language model, based on the input prompt acquired by the prompt acquisition unit 121 or the improved input prompt by the improvement unit 124 and the business information retrieved by the business information retrieval unit 125. The presentation unit 127 transmits the response generated by the response generation unit 126 to the user terminal 30. The large-scale language model may be owned by the information presentation device 10, or by another device (e.g., a cloud server that provides processing using the large-scale language model as a service) that is communicably connected to the information presentation device 10.

[0026] The prompt storage unit 130 stores the input prompts acquired by the prompt acquisition unit 121 as a history of past prompts.

[0027] The improved model storage unit 140 stores the improved model constructed by the model construction unit 114.

[0028] The business information storage unit 150 stores business information. Business information refers to information related to various types of knowledge (e.g., business manuals, company regulations, know-how, etc.) that belong to the tasks that the user is responsible for.

[0029] <Example of improved model construction process> Figure 4 is a flowchart showing an example of the improved model construction process according to this embodiment. In the following, it is assumed that a construction instruction is given from the administrator terminal 20 to the information display device 10. Note that the construction instruction may be given repeatedly at predetermined intervals (e.g., every 3 months).

[0030] The construction instruction acquisition unit 111 of the improved model construction unit 110 acquires the given construction instruction (step S101).

[0031] When the improved model construction unit 110 receives a construction instruction in step S101, the history acquisition unit 112 acquires the prompts stored in the prompt storage unit 130 as prompt history (step S102). Each prompt is associated with at least identification information (e.g., user ID) that identifies the user who entered that prompt. Hereinafter, as an example, the identification information that identifies the user will be assumed to be the user ID.

[0032] The evaluation unit 113 of the improved model construction unit 110 evaluates each of the prompts included in the prompt history acquired in step S102 above as the prompts to be evaluated (step S103). The evaluation unit 113 may, for example, identify the topic to which each prompt to be evaluated belongs, and then, for each topic, evaluate the degree to which a more accurate answer can be obtained with each prompt belonging to that topic. Specifically, the evaluation unit 113 may, for example, execute the following steps 1-1 to 1-2. Note that a topic refers to the field or subject matter that the prompt is targeting.

[0033] Procedure 1-1: The evaluation unit 113 identifies the topic to which each prompt to be evaluated belongs. For example, the evaluation unit 113 identifies the topic to which the input data belongs by providing the input data and a prompt for identifying the topic to which the input data belongs (hereinafter also referred to as the "topic identification prompt") to the large-scale language model, using the prompt to be evaluated as input data. Specifically, the evaluation unit 113 identifies the topic to which the input data belongs by providing the topic identification prompt 1100 shown in Figure 5 and the input data to the large-scale language model, using the prompt to be evaluated as input data. The topic identification prompt 1100 shown in Figure 5 is a prompt that instructs the large-scale language model to classify the input data into one of the predetermined topics: "Semiconductor Business," "Industry Business," "Energy Business," "Food Distribution Business," or "Other." This identifies the topic to which each prompt to be evaluated belongs. Note that the topic identification prompt is created in advance. Input data refers to data other than prompts that is input to the large-scale language model.

[0034] Procedure 1-2: The evaluation unit 113 evaluates each prompt to be evaluated according to the topic to which the prompt belongs. For example, the evaluation unit 113 evaluates the input data by providing the input data and a prompt for evaluating the input data of the topic to which the prompt belongs (hereinafter also referred to as the "evaluation prompt") to the large-scale language model, using the prompt to be evaluated as input data. To give a specific example, if the topic to which the input data belongs is "semiconductor business," the evaluation unit 113 evaluates the input data by providing the evaluation prompt 1200 shown in Figure 6 and the input data to the large-scale language model. The evaluation prompt 1200 shown in Figure 6 is a prompt that instructs the large-scale language model to evaluate on a 5-point scale the degree to which a good response can be obtained when the input data belonging to the predetermined topic "semiconductor business" is used as a prompt. As a result, for each prompt to be evaluated that belongs to a topic, an evaluation value representing the degree to which a good response can be obtained from that prompt is obtained as the evaluation result. Note that the evaluation prompt is created in advance for each topic.

[0035] By following steps 1-1 to 1-2 above, information in the format (prompt, topic, evaluation result) is obtained for each prompt included in the prompt history acquired in step S102 above.

[0036] The model building unit 114 of the improved model building unit 110 builds an improved model (step S104) based on the prompt history obtained in step S101 and the evaluation result in step S103. Below, we will describe the case in which an improved model is built using a knowledge graph and a prompt history, assuming that in step S103, information in the format of (prompt, topic, evaluation result) is obtained for each prompt included in the prompt history. In this case, the model building unit 114 can, for example, execute the following steps 2-1 to 2-6. However, the knowledge graph is just one example, and the improved model is not limited to a knowledge graph as long as it is a model that can search for prompts similar to or related to the input prompt given from the user terminal 30 and that can provide a highly accurate response as an improved prompt. For example, the improved model can also be implemented using a machine learning model such as a neural network.

[0037] Step 2-1: The model building unit 114 extracts predetermined keywords from the prompts included in the prompt history. Examples of keywords include proper nouns such as product names, task names, and procedure names.

[0038] Step 2-2: The model building unit 114 constructs a knowledge graph that associates the topic to which the prompts included in the prompt history belong, the keywords extracted from those prompts, and the user IDs associated with those prompts. This results in a knowledge graph in which topics, keywords, and user IDs are nodes, and their associations are edges (or links).

[0039] Step 2-3: For each user ID, the model building unit 114 calculates a label indicating whether the user of that user ID has a high level of knowledge, skill, and proficiency regarding the prompt, based on the evaluation results of the prompt associated with that user ID. For example, for each user ID, the model building unit 114 calculates the average value of the evaluation results of the prompt associated with that user ID, and if that average value exceeds a predetermined threshold, it calculates the label as "expert," otherwise it calculates the label as "unexpert."

[0040] Step 2-4: The model building unit 114 assigns the labels calculated in Step 2-3 to the user IDs in the knowledge graph built in Step 2-2. This results in a knowledge graph where topics, keywords, and user IDs are nodes, their associations are edges (or links), and the nodes representing user IDs are labeled as either "expert" or "inexperienced." For example, the knowledge graph 2100 shown in Figure 7 is obtained. The knowledge graph 2100 shown in Figure 7 is a knowledge graph where the topics are "Semiconductor Business," "Industry Business," "Energy Business," "Food Distribution Business," and "Other," the keywords are "Module A," ..., "Module D," and "Paid Leave Application," and the user IDs are "User A," "User B," and "User C."

[0041] Step 2-6: The model building unit 114 associates prompts included in the prompt history with the topic to which the prompt belongs and keywords extracted from the prompt. This results in an improved model realized with a knowledge graph and a prompt history consisting of prompts to which at least the user ID, topic, and keywords are associated.

[0042] The model construction unit 114 of the improved model construction unit 110 saves the improved model constructed in step S104 to the improved model storage unit 140 (step S105). If an improved model is already stored in the improved model storage unit 140, the model construction unit 114 can update the improved model stored in the improved model storage unit 140 using the improved model constructed in step S104.

[0043] <Example of information presentation processing> Figure 8 is a flowchart showing an example of information presentation processing according to this embodiment. In the following, it is assumed that an input prompt is provided from the user terminal 30.

[0044] The prompt acquisition unit 121 of the information presentation unit 120 acquires the given input prompt (step S201).

[0045] The prompt acquisition unit 121 of the information presentation unit 120 stores the input prompt acquired in step S201 in the prompt storage unit 130 (step S202). At this time, the prompt acquisition unit 121 stores the input prompt in the prompt storage unit 130 in association with the user ID of the user using the user terminal 30.

[0046] The evaluation unit 122 of the information presentation unit 120 evaluates the input prompt acquired in step S201 as the prompt to be evaluated (step S203). Similar to step S103 in Figure 4, the evaluation unit 122 may, for example, identify the topic to which the prompt to be evaluated belongs and then evaluate the degree to which a highly accurate answer can be obtained from the prompt to be evaluated. Specifically, the evaluation unit 122 may, for example, execute the following steps 3-1 to 3-2.

[0047] Step 3-1: The evaluation unit 122 identifies the topic to which the prompt to be evaluated belongs. Similar to Step 1-1 above, the evaluation unit 122 identifies the topic to which the input data belongs by providing the prompt to be evaluated as input data, along with a topic identification prompt and the input data, to a large-scale language model.

[0048] Step 3-2: The evaluation unit 122 evaluates the prompt to be evaluated according to the topic to which the prompt belongs. For example, the evaluation unit 122 evaluates the input data by providing the prompt to be evaluated as input data to a large-scale language model, along with an evaluation prompt for evaluating the input data of the topic to which the prompt to be evaluated belongs.

[0049] By following steps 3-1 to 3-2 above, information in the format (input prompt, topic, evaluation result) is obtained for the input prompt acquired in step S201 above.

[0050] The evaluation unit 122 of the information presentation unit 120 determines whether the evaluation result from step S203 exceeds a predetermined threshold (step S204). If the evaluation result exceeds the threshold, it means that the input prompt is a prompt from which a highly accurate answer can be obtained. On the other hand, if the evaluation result does not exceed the threshold, it means that the input prompt is a prompt from which a highly accurate answer cannot necessarily be obtained.

[0051] If the evaluation result in step S204 is determined to exceed a predetermined threshold, the business information retrieval unit 125 of the information presentation unit 120 retrieves business information related to the input prompt obtained in step S201 from the business information storage unit 150 (step S205). The business information retrieval unit 125 can use any known method to retrieve business information related to the input prompt from the business information storage unit 150. For example, the business information retrieval unit 125 can calculate the similarity between the vector data obtained by vectorizing the input prompt and the vector data obtained by vectorizing each piece of business information, and then retrieve the business information with the highest similarity from the business information storage unit 150.

[0052] The response generation unit 126 of the information presentation unit 120 uses the business information retrieved in step S205 as input data, and provides the input prompt obtained in step S201 and the input data to a large-scale language model to generate a response to the input prompt (step S206).

[0053] On the other hand, if the evaluation result in step S204 above is not determined to exceed a predetermined threshold, the prompt search unit 123 of the information presentation unit 120 searches for a prompt to be used for improvement based on the improved model stored in the improved model storage unit 140, which is similar to or related to the input prompt obtained in step S201 above and can provide a highly accurate response (step S207). For example, if the improved model is realized using a knowledge graph and a prompt history consisting of prompts to which the user ID, topic, and keyword are at least associated, the prompt search unit 123 can, for example, execute the following steps 4-1 to 4-3.

[0054] Step 4-1: The prompt search unit 123 extracts predetermined keywords from the input prompt, similar to step 2-1 above.

[0055] Step 4-2: The prompt search unit 123 identifies user IDs that have been assigned the label "Expert" based on the topic to which the input prompt belongs, the keywords extracted in Step 4-1 above, and the knowledge graph. Note that multiple user IDs may be identified at this time.

[0056] For example, suppose the topic to which the input prompt belongs is "Semiconductor Business," and the keyword extracted in step 4-1 above is "Module A." In this case, the user ID "User A" is identified by the knowledge graph 2100 shown in Figure 7.

[0057] Step 4-3: The prompt search unit 123 searches the prompt history for prompts that are associated with the user ID identified in Step 4-2 above, the topic to which the input prompt belongs, and the keywords extracted in Step 4-1 above, and uses these as improvement prompts. Note that multiple improvement prompts may be found at this time.

[0058] By following the above steps 4-1 to 4-3, a prompt belonging to the same topic as the input prompt, containing the same keywords, and entered by an expert is obtained as an improved prompt.

[0059] The improvement unit 124 of the information presentation unit 120 improves the input prompt obtained in step S201 based on the improvement prompt retrieved in step S207 (step S208). The improvement unit 124 uses the input prompt as input data and improves the input data by providing the input data and the improvement prompt to a large-scale language model, along with a prompt for improving the input data with reference to the improvement prompt (hereinafter also referred to as the "improvement instruction prompt"). Specifically, the improvement unit 124 uses the input prompt as input data and improves the input data by providing the improvement instruction prompt 1300 shown in Figure 9, along with the input data and the improvement prompt to a large-scale language model. This results in improved input data, i.e., an improved input prompt.

[0060] The business information retrieval unit 125 of the information presentation unit 120 retrieves business information related to the improved input prompt obtained in step S208 from the business information storage unit 150, similar to step S205 above (step S209).

[0061] The response generation unit 126 of the information presentation unit 120 uses the business information retrieved in step S209 as input data, and provides the improved input prompt obtained in step S208 and the input data to a large-scale language model to generate a response to the improved input prompt (step S210).

[0062] The presentation unit 127 of the information presentation unit 120 transmits the response generated in step S206 or step S210 above to the user terminal 30 (step S211). This presents the user using the user terminal 30 with the response to the input prompt.

[0063] <Summary> As described above, in the information presentation system 1 according to this embodiment, the degree to which an accurate response can be obtained from an input prompt given by the user is evaluated, and if the evaluation result is low, the input prompt is improved using a prompt that is similar to or related to the input prompt and has been evaluated as having a high degree to which an accurate response can be obtained. As a result, in the information presentation system 1 according to this embodiment, even if the user's knowledge, skills, or proficiency regarding the prompts is low, the user can obtain an accurate response (i.e., an answer that contains the information the user desires).

[0064] The present invention is not limited to the embodiments specifically disclosed above, and various modifications, changes, and combinations with known technologies are possible as long as they do not deviate from the spirit described in the claims. [Explanation of Symbols]

[0065] 1. Information Presentation System 10 Information presentation device 20 Administrator terminals 30 User terminals 40 Communication Networks 101 Input Device 102 Display device 103 External I / F 103a Recording medium 104 Communication I / F 105 RAM 106 ROM 107 Auxiliary storage 108 processors 109 Bus 110 Improved Model Construction Department 111 Construction instruction acquisition unit 112 History Acquisition Unit 113 Evaluation Department 114 Model Building Department 120 Information Presentation Department 121 Prompt acquisition section 122 Evaluation Department 123 Prompt Search Section 124 Improvement Department 125 Business Information Retrieval Department 126 Answer generation part 127 Presentation section 130 Prompt memory unit 140 Improved Model Memory Unit 150 Business information storage unit

Claims

1. An evaluation unit that evaluates the degree to which a highly accurate response can be obtained from a first prompt given by the user, If the evaluation value representing the result of the evaluation is below a predetermined threshold, a search unit searches for a second prompt to improve the first prompt, An improvement unit creates a third prompt that is an improved version of the first prompt, based on the second prompt, so as to provide a more accurate answer than the first prompt. A response generation unit that generates a response to the third prompt based on the third prompt and a large-scale language model, A display unit that presents the aforementioned answer to the user, An information display device having the following features.

2. The aforementioned search unit, The information presentation device according to claim 1, which searches for a second prompt that is related to or similar to the first prompt and can provide a highly accurate response, based on a prompt history representing the history of past prompts and a graph for searching for prompts included in the prompt history.

3. The aforementioned graph shows This is a knowledge graph that associates the topic to which the prompts included in the prompt history belong, the keywords included in the prompts, and the identification information of the user who gave the prompts. The information presentation device according to claim 2, wherein the node representing the identification information is assigned a label indicating whether or not the user identified by the identification information can create a prompt that provides an accurate response.

4. The aforementioned search unit, Based on the topic to which the first prompt belongs, the keywords included in the first prompt, and the knowledge graph, identify user identification information that can create prompts that yield accurate answers. The information presentation device according to claim 3, which searches for the second prompt from among prompts given by a user identified by the identified identification information, based on the topic to which the first prompt belongs, the keywords contained in the first prompt, and the prompt history.

5. An information presentation device according to any one of claims 2 to 4, having a construction unit that constructs the graph based on the prompt history at predetermined intervals.

6. An evaluation procedure for evaluating the degree to which a highly accurate response can be obtained from a first prompt given by the user, If the evaluation value representing the result of the evaluation is below a predetermined threshold, a search procedure for searching for a second prompt to improve the first prompt, An improvement procedure for creating a third prompt that improves upon the first prompt so that a more accurate answer can be obtained than the first prompt, based on the second prompt; A response generation procedure that generates a response to the third prompt based on the third prompt and a large-scale language model, A presentation procedure for presenting the aforementioned answer to the user, A method of presenting information that a computer performs.

7. An evaluation procedure for evaluating the degree to which a highly accurate response can be obtained from a first prompt given by the user, If the evaluation value representing the result of the evaluation is below a predetermined threshold, a search procedure for searching for a second prompt to improve the first prompt, An improvement procedure for creating a third prompt that improves upon the first prompt so that a more accurate answer can be obtained than the first prompt, based on the second prompt; A response generation procedure that generates a response to the third prompt based on the third prompt and a large-scale language model, A presentation procedure for presenting the aforementioned answer to the user, A program that causes a computer to execute something.

Images