Hallucination suppression system and program

Abstract

Provided are a hallucination suppression system and program capable of suppressing hallucination of interactive AI.　The hallucination suppression system 10 of interactive AI comprises: a custom prompt acquisition unit 104 for acquiring a custom prompt for eliciting an answer with higher accuracy, the prompt being generated by interactive AI 50 on the basis of a prompt from a user; a determination unit 106 for determining, on the basis of an evaluation by a machine-learning model, whether the answer of the interactive AI includes hallucination; and a transmission / reception unit 102 for transmitting the answer to a user terminal 20 when the determination unit 106 determines that the answer of the interactive AI 50 does not include hallucination.

Description

Hallucination suppression system and program

[0001] The present invention relates to a hallucination suppression system and program.

[0002] Non-Patent Document 1 describes how ChatGPT ("CHATGPT" is an internationally registered trademark) works and why it works so well, and describes how ChatGPT sometimes generates information that is not present in the actual data, i.e., hallucination.

[0003] "What Is ChatGPT Doing… and Why Does It Work?", [online], [Retrieved July 5, 2024], https: / / writings.stephenwolfram.com / 2023 / 02 / what-is-chatgpt-doing-and-why-does-it-work / , Internet

[0004] The present invention aims to provide a hallucination suppression system and program that can suppress hallucination in interactive AI.

[0005] A hallucination suppression system according to one embodiment of the present invention comprises a determination unit that determines, based on a machine learning model, whether or not hallucination is included in an interactive AI's response to a prompt sent by a user, and a transceiver unit that transmits the response to a user terminal if the determination unit determines that the interactive AI's response does not include hallucination, and the response that is determined not to include hallucination is transmitted to the user terminal.

[0006] According to the present invention, a hallucination suppression system and program can be provided that can suppress hallucination in interactive AI.

[0007] 1 is an explanatory diagram showing the configuration of a hallucination suppression system according to an embodiment of the present invention;

[0008] Next, embodiments of the present invention will be described with reference to the accompanying drawings to facilitate understanding of the present invention. Note that in the drawings, parts that are not relevant to the description may be omitted.

[0009] A hallucination suppression system 10 according to one embodiment of the present invention can suppress hallucination from being included in a conversational AI response to a prompt sent by a user. The conversational AI is, for example, ChatGPT ("CHATGPT" is an internationally registered trademark) provided by OpenAI, Inc., and is configured based on large language models.

[0010] As shown in FIG. 1, the hallucination suppression system 10 includes a transmitter / receiver 102, a custom prompt acquisition unit 104, and a determination unit 106.

[0011] The transmission / reception unit (an example of a transmission / reception means) 102 is connected to the user terminal 20 and the interactive AI 50 provided as a cloud service via a network. Therefore, the transmission / reception unit 102 can receive prompts input by the user from the user terminal 20 and transmit data such as answers to the prompts to the user terminal 20. The transmission / reception unit 102 can also transmit prompts to the interactive AI 50 and receive answers to the prompts from the interactive AI 50.

[0012] Here, the prompts sent to the conversational AI 50 include custom prompts. These custom prompts are generated by the conversational AI based on the user's prompts, and are prompts that elicit more accurate answers from the conversational AI 50.

[0013] Specifically, the custom prompt is written, for example, based on Markdown notation, and includes at least the following content: (1) Command The command specifies the specific content of the question or request by clearly indicating the information or task desired of the conversational AI 50. A specific example of a command is "Analyze this dataset and report key statistical indicators."

[0014] (2) Role The role indicates the role expected of the interactive AI 50 by indicating the position and perspective from which the interactive AI 50 should respond. A specific example of a role is "You are an expert in hydraulic technology."

[0015] (3) Output Restrictions Output restrictions restrict the format and scope of answers by specifying the length, detail, format, etc. Specific examples of output restrictions include "Please answer concisely in 200 characters or less" and "Please explain in a way that even a high school student can understand."

[0016] (4) Evaluation: Evaluation indicates that the answer of the interactive AI 50 is evaluated in a predetermined manner, and if the answer does not meet the predetermined evaluation criteria, it instructs the AI ​​50 to generate a new answer. A specific example of evaluation is, "Answers are always evaluated on a three-point scale. If the evaluation is 2 points or less, please come up with a different answer." This evaluation may be included in the above-mentioned command. Furthermore, the evaluation criteria are not limited to evaluation on multiple levels.

[0017] (5) Reference example The reference example shows an example of a desirable answer. A specific example of the reference example is "Includes an overview of the failure, investigation of the cause, and repair procedures."

[0018] (6) Execution Procedures The execution procedures detail the specific steps for executing a task.

[0019] In addition, at least a part of the contents of "(2) Role," "(3) Output Limitation," "(4) Evaluation," "(5) Reference Example," and "(6) Execution Procedure" may be described as "(1) Command." Additionally, data between the transmission / reception unit 102 and the interactive AI 50 is transmitted and received using an API.

[0020] The custom prompt acquisition unit (an example of a custom prompt acquisition means) 104 transmits a custom prompt generation prompt to the interactive AI 50 for generating a custom prompt based on the user's prompt received by the transmission / reception unit 102, and can acquire the custom prompt as an answer generated by the interactive AI 50. Data between the custom prompt acquisition unit 104 and the interactive AI 50 is transmitted and received using an API. The custom prompt acquired by the custom prompt acquisition unit 104 is transmitted from the transmission / reception unit 102 to the interactive AI 50.

[0021] The determination unit (an example of a determination means) 106 can determine whether hallucination is included in the answer of the interactive AI 50 to the custom prompt based on an evaluation by a machine learning model. The machine learning model has previously learned answers that include hallucination and can determine whether the generated answer includes hallucination. The machine learning model can output the degree of hallucination as a multi-level evaluation score (e.g., a 5-level evaluation score). The evaluation output by the machine learning model is not limited to a multi-level evaluation score, and may be any content that corresponds to the evaluation criteria specified in the "evaluation" of the custom prompt.

[0022] When the determination unit 106 determines that the answer of the interactive AI 50 does not contain hallucination, it can instruct the transceiver unit 102 to send the answer to the user terminal 20. On the other hand, when the determination unit 106 determines that the answer of the interactive AI 50 contains hallucination, it can generate a reconsideration prompt to cause the interactive AI 50 to reconsider the answer and instruct the transceiver unit 102 to send the reconsideration prompt to the interactive AI 50.

[0023] The reconsideration prompt is the evaluation content output by the machine learning model. As described above, if the "evaluation" included in the custom prompt does not satisfy the evaluation criteria, the interactive AI 50 is instructed to generate a new answer. Therefore, by simply sending the evaluation output by the machine learning model as a prompt, the interactive AI 50 reconsiders the answer and generates a new answer. The reconsideration prompt is not limited to the evaluation output by the machine learning model, and may be any content that causes the interactive AI 50 to reconsider the answer.

[0024] At least the transmission / reception unit 102, the custom prompt acquisition unit 104, and the determination unit 106 are realized by a computer program and function as a transmission / reception means, a custom prompt acquisition means, and a determination means, respectively. Furthermore, the transmission / reception unit 102, the custom prompt acquisition unit 104, and the determination unit 106 are not limited to being all included in a single computer; at least some of them may be configured separately as multiple computers connected to each other via a network, or at least some of the functions may be realized by cloud computing. Furthermore, the interactive AI that generates the custom prompt and the interactive AI that returns an answer to the custom prompt do not have to be a common interactive AI, but may be interactive AIs configured based on different large-scale language models.

[0025] Next, the operation of the hallucination suppression system 10 (hallucination suppression method) will be described with reference to FIG. 2. Hallucination is suppressed according to the following steps S1 to S10. However, if possible, the steps may be executed interchangeably or in parallel. For ease of understanding, a hypothetical example will be described in which a user obtains information about company A from the interactive AI 50.

[0026] (Step S1) The user sends a prompt from the user terminal 20. The content of the prompt is, for example, "Please tell me about Company A at the following URL: https: / / e-ryowa.com / page1.html."

[0027] (Step S2) The transceiver 102 of the hallucination suppression system 10 receives the user's prompt. The custom prompt acquisition unit 104 sends a custom prompt generation prompt to the interactive AI 50, instructing the interactive AI 50 to generate a custom prompt that will provide a more accurate answer to the received prompt. The interactive AI 50 returns an answer to the custom prompt generation prompt. In other words, the interactive AI 50 generates a custom prompt based on the user's prompt. The generated custom prompt is acquired by the custom prompt acquisition unit 104.

[0028] The custom prompt returned by the conversational AI 50 might, for example, be something like the following: (1) Command: "Crawl the following URL and summarize the company's content in approximately 200 characters: https: / / e-ryowa.com / page1.html" (2) Role: "You are a competent prompt engineer. Your goal is to create the best prompt that meets my needs." (3) Output restrictions: "Please explain in a way that even an elementary school student can understand." (4) Evaluation: "Answers are always scored on a 5-point scale. If your score is 3 or less, please think of a different answer." (5) Reference example: "Company name: {Company name} Title: {Title} Body: {Body}" (6) Execution procedure: "The user provides the URL of a company's website. You crawl the specified URL and extract relevant information about the company. Then, briefly summarize the company's content in approximately 200 characters."

[0029] The aforementioned "(4) Evaluation" indicates that the evaluation criteria are evaluation points on a 5-point scale, and the evaluation points (1 to 5 points) correspond to the evaluation points output by the machine learning model.

[0030] (Step S3) The transmission / reception unit 102 transmits the custom prompt acquired by the custom prompt acquisition unit 104 to the interactive AI 50. As a result, the interactive AI 50 outputs information about Company A as a response to the custom prompt.

[0031] (Step S4) The transmitter / receiver 102 receives the answer generated by the interactive AI 50.

[0032] (Step S5) The judgment unit 106 obtains an evaluation score representing the degree to which the answer contains hallucination from a machine learning model that has previously learned answers containing hallucination, and determines whether the answer generated by the interactive AI 50 contains hallucination based on this evaluation score.

[0033] Specifically, for example, if the evaluation score output by the machine learning model for the answer is 3 points or less, the determination unit 106 determines that hallucination is included in the answer regarding company A. On the other hand, for example, if the evaluation score output by the machine learning model is 4 points or more, the determination unit 106 determines that hallucination is not included in the answer regarding company A.

[0034] (Step S6) If the judgment unit 106 determines that the answer from the interactive AI 50 does not contain hallucination, step S7 is executed, and if it determines that the answer contains hallucination, step S8 is executed.

[0035] (Step S7) The transmitter / receiver 102 returns a response to the user terminal 20. That is, information about Company A is returned as a response to the prompt sent by the user in step S1. The transmitter / receiver 102 also transmits the evaluation score from the previous step S6 to the interactive AI 50.

[0036] (Step S8) If a predetermined condition is not satisfied, step S9 is executed, and if the predetermined condition is satisfied, step S10 is executed. Here, the predetermined condition is, for example, that the number of times Nr to repeat step S9 exceeds a predetermined number, for example, 10 times.

[0037] However, the judgment condition (predetermined condition) of step S8 is not limited to whether the number of times Nr to repeat step S9 exceeds a predetermined number. Another example of a predetermined condition is, for example, whether the time elapsed since any of steps S4 to S6 was executed exceeds a predetermined length. That is, the predetermined condition may be whether the number of times the process of repeating steps S4 (receiving an answer), S5, S6, S8, and S9 (sending a reconsideration prompt) is repeated exceeds a predetermined number, or whether the time required for the process exceeds a predetermined length.

[0038] (Step S9) The transmission / reception unit 102 sends a prompt indicating the evaluation score (an example of a reconsideration prompt) to the interactive AI 50. This prompt is, for example, the content of the evaluation by the machine learning model, such as "The evaluation score of the answer is 3 points." The interactive AI 50 understands the evaluation score based on the custom prompt initially input, and returns the result of reconsidering the answer. In other words, it generates an answer that is different from the previous answer.

[0039] (Step S10) The judgment unit 106 outputs an error, and outputs the answer of the interactive AI 50 at that time as training data for an answer including hallucination. The output error is notified to the user terminal 20. The output training data is used to train the machine learning model, contributing to improving the accuracy of hallucination judgment by the machine learning model.

[0040] Thus, the hallucination suppression system 10 includes a custom prompt acquisition unit 104 that acquires a custom prompt generated from a user's prompt and transmits the custom prompt to the interactive AI 50, so that the interactive AI 50 can be expected to return a more accurate answer with hallucination suppressed. Furthermore, if the answer of the interactive AI 50 contains hallucination, the hallucination suppression system 10 causes the interactive AI 50 to reconsider the answer until it is determined that it does not contain hallucination, thereby suppressing hallucination.

[0041] Although the embodiments of the present invention have been described above, the present invention is not limited to the above-described embodiments, and all changes in conditions that do not depart from the gist of the present invention are within the scope of application of the present invention.

[0042] 10 Hallucination suppression system 20 User terminal 50 Interactive AI 102 Transmitting / receiving unit 104 Custom prompt acquisition unit 106 Determination unit

Claims

1. A hallucination suppression system for an interactive AI comprising: a custom prompt acquisition unit that acquires custom prompts that are generated by the interactive AI based on user prompts to elicit more accurate answers; a judgment unit that determines whether the answer of the interactive AI contains hallucination based on an evaluation by a machine learning model; and a transceiver unit that transmits the answer to a user terminal when the judgment unit determines that the answer of the interactive AI does not contain hallucination.

2. A hallucination suppression system for an interactive AI as described in claim 1, wherein, when the judgment unit determines that the answer of the interactive AI contains hallucination, the transceiver unit sends a reconsideration prompt to the interactive AI to cause the interactive AI to reconsider the answer.

3. A hallucination suppression system for an interactive AI as described in claim 2, wherein the transmitting / receiving unit executes a repetitive process of receiving an answer and transmitting the reconsideration prompt until the judgment unit determines that the answer of the interactive AI does not contain hallucination.

4. The hallucination suppression system of an interactive AI according to claim 3, wherein an error is output when a predetermined condition is met.

5. A hallucination suppression system for an interactive AI as described in claim 4, wherein the judgment unit outputs the answer of the interactive AI when the error is output as training data for an answer including hallucination for the machine learning model.

6. An interactive AI hallucination suppression system as described in claim 4, wherein the predetermined condition is that the number of times the repetitive processing has been performed has exceeded a predetermined number of times or that the time required for the repetitive processing has exceeded a predetermined length.

7. An interactive AI hallucination suppression system as described in claim 5, wherein the content of the custom prompt includes an instruction to generate a new answer if the answer of the interactive AI does not satisfy predetermined evaluation criteria, and the content of the reconsideration prompt includes the evaluation content by the machine learning model.

8. An interactive AI hallucination suppression system according to claim 7, wherein the predetermined evaluation criteria are evaluation points based on a plurality of stages.

9. A hallucination suppression system using an interactive AI as described in any one of claims 1 to 8, wherein the interactive AI that generates the custom prompt and the interactive AI that returns an answer to the custom prompt are each constructed based on different large-scale language models.

10. A hallucination suppression system for an interactive AI, comprising: a judgment unit that judges based on a machine learning model whether or not hallucination is included in the response of an interactive AI to a prompt sent by a user; and a transceiver unit that, when the judgment unit determines that the response of the interactive AI does not contain hallucination, transmits the response to a user terminal; and, when the judgment unit determines that the response of the interactive AI contains hallucination, causes the interactive AI to reconsider the response until it is determined that it does not contain hallucination.

11. A hallucination suppression program for causing a computer to function as: a custom prompt acquisition means for acquiring a custom prompt that is generated by an interactive AI based on a user's prompt and is used to elicit a more accurate response; a determination means for determining based on a machine learning model whether the interactive AI's response to the custom prompt contains hallucination; and a transmission / reception means for transmitting the response to a user terminal when the determination means determines that the interactive AI's response does not contain hallucination.

Images