Hallucination suppression system and program

Abstract

To provide a hallucination suppression system and a program capable of suppressing hallucination of an interactive AI.SOLUTION: The system for suppressing hallucination in an interactive AI 10 includes a custom prompt acquirer 104 that acquires a custom prompt for eliciting a more accurate response, the custom prompt being generated by an interactive AI50 based on a prompt from a user, a determiner 106 that determines whether or not the response from the interactive AI50 to the custom prompt includes hallucination based on evaluation by a machine-learning model, and a transceiver 102 that transmits the response to the user when it is determined that the response from the interactive AI does not include hallucination.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a hallucination suppression system and program. [Background technology]

[0002] Non-Patent Document 1 describes how ChatGPT ("CHATGPT" is an internationally registered trademark) works and why it works so well, and describes how ChatGPT sometimes generates information that does not exist in the actual data, i.e., hallucination. [Prior art documents] [Patent documents]

[0003] [Non-Patent Document 1] "What Is ChatGPT Doing… and Why Does It Work?", [online], [Retrieved July 5, 2024], https: / / writings.stephenwolfram.com / 2023 / 02 / what-is-chatgpt-doing-and-why-does-it-work / , Internet Summary of the Invention [Problem to be solved by the invention]

[0004] The present invention aims to provide a hallucination suppression system and program that can suppress hallucination in interactive AI. [Means for solving the problem]

[0005] The invention described in claim 1 comprises a custom prompt acquisition unit that acquires a custom prompt that is generated by an interactive AI based on a user's prompt to elicit a more accurate answer; a judgment unit that determines whether the answer of the interactive AI contains hallucination based on an evaluation by a machine learning model; and a transceiver unit that sends the answer to the user when the judgment unit determines that the answer of the interactive AI does not contain hallucination.

[0006] The invention described in claim 2 is a hallucination suppression system for an interactive AI described in claim 1, in which, when the judgment unit determines that the answer of the interactive AI contains hallucination, the transmission / reception unit sends a reconsideration prompt to the interactive AI to cause it to reconsider the answer.

[0007] The invention described in claim 3 is a hallucination suppression system for an interactive AI described in claim 2, in which the transmitter / receiver unit performs a repetitive process of receiving an answer and sending the reconsideration prompt until the judgment unit determines that the answer of the interactive AI does not contain hallucination.

[0008] The invention described in claim 4 is an interactive AI hallucination suppression system described in claim 3, which outputs an error when predetermined conditions are met.

[0009] The invention described in claim 5 is a hallucination suppression system for an interactive AI described in claim 4, in which the judgment unit outputs the response of the interactive AI when the error is output as training data for an answer including hallucination for the machine learning model.

[0010] The invention described in claim 6 is an interactive AI hallucination suppression system described in claim 4, in which the predetermined condition is that the number of times the repetitive processing has been performed has exceeded a predetermined number of times or the time required for the repetitive processing has exceeded a predetermined length.

[0011] The invention described in claim 7 is a hallucination suppression system for an interactive AI described in claim 5, in which the content of the custom prompt includes an instruction to generate a new answer if the answer of the interactive AI does not meet predetermined evaluation criteria, and the content of the reconsideration prompt includes the evaluation content by the machine learning model.

[0012] The invention described in claim 8 is an interactive AI hallucination suppression system described in claim 7, in which the predetermined evaluation criteria are evaluation points based on multiple levels.

[0013] The invention described in claim 9 is an interactive AI hallucination suppression system described in any one of claims 1 to 8, in which the interactive AI that generates the custom prompt and the interactive AI that returns an answer to the custom prompt are each constructed based on different large-scale language models.

[0014] The invention described in claim 10 is a hallucination suppression system for an interactive AI, comprising a judgment unit that judges based on a machine learning model whether or not a response from an interactive AI to a prompt sent by a user contains hallucination, and a transceiver unit that sends the response to the user when the judgment unit determines that the response from the interactive AI does not contain hallucination, and when the judgment unit determines that the response from the interactive AI contains hallucination, causes the interactive AI to reconsider the response until it is determined that the response does not contain hallucination.

[0015] The invention described in claim 11 is a hallucination suppression program that causes a computer to function as a custom prompt acquisition means that acquires a custom prompt that is generated by an interactive AI based on a user's prompt and is used to elicit a more accurate answer, a determination means that determines based on a machine learning model whether the interactive AI's answer to the custom prompt contains hallucination, and a transmission / reception means that sends the answer to the user when the determination means determines that the interactive AI's answer does not contain hallucination. [Effects of the Invention]

[0016] According to the present invention, a hallucination suppression system and program can be provided that can suppress hallucination in interactive AI. [Brief explanation of the drawings]

[0017] [Figure 1] 1 is an explanatory diagram showing the configuration of a hallucination suppression system according to one embodiment of the present invention; [Figure 2] 3 is a flowchart showing the operation of the hallucination suppression system. DETAILED DESCRIPTION OF THE INVENTION

[0018] Next, embodiments of the present invention will be described with reference to the accompanying drawings to facilitate understanding of the present invention. Note that in the drawings, parts that are not relevant to the description may be omitted.

[0019] A hallucination suppression system 10 according to an embodiment of the present invention can suppress hallucination from being included in a conversational AI's response to a prompt sent by a user. The conversational AI is, for example, ChatGPT ("CHATGPT" is an internationally registered trademark) provided by OpenAI, Inc., and is configured based on large language models.

[0020] As shown in FIG. 1, the hallucination suppression system 10 includes a transmitter / receiver 102, a custom prompt acquisition unit 104, and a determination unit 106.

[0021] The transmitting / receiving unit (an example of a transmitting / receiving means) 102 is connected to the user terminal 20 and the interactive AI 50 provided as a cloud service via a network. Therefore, the transceiver 102 can receive prompts entered by the user from the user terminal 20 and transmit data such as answers to the prompts. The transceiver 102 can also transmit prompts to the interactive AI 50 and receive answers to the prompts from the interactive AI 50.

[0022] Here, the prompts sent to the conversational AI 50 include custom prompts. These custom prompts are generated by the conversational AI based on the user's prompts, and are prompts that elicit more accurate answers from the conversational AI 50.

[0023] Specifically, the custom prompt is written, for example, based on Markdown notation and includes at least the following content: (1) Command Commands specify the specific content of a question or request by clearly indicating to the conversational AI 50 the information or task being sought. An example of an instruction might be, "Analyze this dataset and report key statistical indicators."

[0024] (2) Role The role indicates the expected role of the conversational AI 50 by indicating the position and perspective from which the conversational AI 50 should respond. An example of a role would be, "You are a hydraulics expert."

[0025] (3) Output Limit Output restrictions limit the type and range of responses by specifying the length, detail, format, etc. of the responses. Specific examples of output restrictions include "Please answer concisely in 200 characters or less" and "Please explain in a way that even a high school student can understand."

[0026] (4) Evaluation The evaluation indicates that the answer of the interactive AI 50 will be evaluated in a predetermined manner, and if it does not meet the predetermined evaluation criteria, it instructs the AI ​​to generate an answer again. A specific example of evaluation is, "Answers are always evaluated on a three-point scale. If your evaluation is below two, please consider a different answer." This evaluation may be included in the above-mentioned command. The evaluation criteria are not limited to a multi-level evaluation.

[0027] (5) Reference example The reference examples show examples of desirable answers. A specific example of a reference example is "Includes an overview of the failure, investigation of the cause, and repair procedures."

[0028] (6) Execution procedure Execution procedures detail the specific steps for performing a task.

[0029] In addition, at least a portion of the contents of "(2) Role," "(3) Output Restriction," "(4) Evaluation," "(5) Reference Example," and "(6) Execution Procedure" may be described as "(1) Command." In addition, data is transmitted and received between the transmitting / receiving unit 102 and the interactive AI 50 using an API.

[0030] The custom prompt acquisition unit (an example of a custom prompt acquisition means) 104 can send a custom prompt generation prompt to the interactive AI 50 to generate a custom prompt based on the user's prompt received by the transmission / reception unit 102, and can acquire the custom prompt as an answer generated by the interactive AI 50. Data is transmitted and received between the custom prompt acquisition unit 104 and the interactive AI 50 using an API. The custom prompt acquired by the custom prompt acquisition unit 104 is transmitted from the transmission / reception unit 102 to the interactive AI 50 .

[0031] The determination unit (an example of a determination means) 106 can determine whether or not hallucination is included in the response of the interactive AI 50 to the custom prompt based on an evaluation by a machine learning model. The machine learning model has previously learned answers that include hallucination and can determine whether the generated answer contains hallucination. The machine learning model can output the degree of hallucination as a multi-level evaluation score (for example, a 5-point evaluation score). The evaluation output by the machine learning model is not limited to multiple levels of evaluation points, but may be anything that corresponds to the evaluation criteria specified in the "Evaluation" section of the custom prompt.

[0032] If the determination unit 106 determines that the answer of the interactive AI 50 does not include hallucination, it can instruct the transmission / reception unit 102 to transmit the answer to the user. On the other hand, if the judgment unit 106 determines that the answer of the interactive AI 50 contains hallucination, it can generate a reconsideration prompt to cause the interactive AI 50 to reconsider the answer and instruct the transmission / reception unit 102 to send the reconsideration prompt to the interactive AI 50.

[0033] The reconsideration prompt is the evaluation content output by the machine learning model. As mentioned above, if the "evaluation" included in the custom prompt does not meet the evaluation criteria, the conversational AI 50 is instructed to generate a new answer. Therefore, by simply sending the evaluation output by the machine learning model as a prompt, the conversational AI 50 reconsiders the answer and generates a new answer. The reconsideration prompt is not limited to the rating output by the machine learning model, but may be any rating that causes the conversational AI 50 to reconsider the answer.

[0034] At least the transmitting / receiving unit 102, the custom prompt acquisition unit 104, and the determining unit 106 are realized by a computer program, and function as a transmitting / receiving means, a custom prompt acquisition means, and a determining means, respectively. Furthermore, the present invention is not limited to a single computer having all of the transmission / reception unit 102, custom prompt acquisition unit 104, and judgment unit 106; at least some of the functions may be separated into multiple computers connected to each other via a network, or at least some of the functions may be realized by cloud computing. Furthermore, the conversational AI that generates the custom prompt and the conversational AI that returns an answer to the custom prompt do not have to be a common conversational AI, but may be conversational AIs constructed based on different large-scale language models.

[0035] Next, the operation of the hallucination suppression system 10 (hallucination suppression method) will be described with reference to FIG. Hallucination is suppressed in accordance with the following steps S1 to S10, however, the steps may be executed interchangeably or in parallel, if possible. For ease of understanding, a hypothetical example will be described in which a user obtains information about company A from the interactive AI 50.

[0036] (Step S1) The user sends a prompt from the user terminal 20. The content of the prompt is, for example, "Please tell me about Company A at the following URL: https: / / e-ryowa.com / page1.html."

[0037] (Step S2) The transceiver 102 of the hallucination suppression system 10 receives the user's prompt. The custom prompt acquisition unit 104 transmits a custom prompt generation prompt to the interactive AI 50, instructing the interactive AI 50 to generate a custom prompt that will provide a more accurate answer from the received prompt. The interactive AI 50 returns answers to the custom prompt generation prompts, i.e., the interactive AI 50 generates custom prompts based on the user's prompts. The generated custom prompt is acquired by the custom prompt acquisition unit 104 .

[0038] The custom prompt returned by the conversational AI 50 may be, for example, as follows: (1) Command "Crawl the following URL and summarize the company's content in approximately 200 characters: https: / / e-ryowa.com / page1.html" (2) Role "You're a talented prompt engineer. Your goal is to create the best prompt that meets my needs." (3) Output Limit Please explain in a way that even an elementary school student can understand. (4) Evaluation "Your answers will always be graded on a 5-point scale. If you receive a 3 or lower, please consider a different answer." (5) Reference example "Company name: {company name} Title: {title} Body:{body}” (6) Execution procedure The user will provide you with a company website URL. You will crawl the URL and extract relevant information about the company. You will then provide a concise summary of the company in approximately 200 characters.

[0039] The aforementioned "(4) Evaluation" indicates that the evaluation criteria are based on a five-point scale, and the evaluation points (1 to 5 points) correspond to the evaluation points output by the machine learning model.

[0040] (Step S3) The transmitting / receiving unit 102 transmits the custom prompt acquired by the custom prompt acquiring unit 104 to the interactive AI 50 . As a result, the conversational AI 50 outputs information about Company A as answers to the custom prompts.

[0041] (Step S4) The transceiver 102 receives the answer generated by the interactive AI 50.

[0042] (Step S5) The judgment unit 106 obtains an evaluation score representing the degree to which hallucination is included in the answer from a machine learning model that has previously learned answers that include hallucination, and determines whether or not hallucination is included in the answer generated by the interactive AI 50 based on this evaluation score.

[0043] Specifically, for example, if the evaluation score output by the machine learning model for the answer is 3 points or less, the determining unit 106 determines that the answer regarding company A includes hallucination. On the other hand, for example, if the evaluation score output by the machine learning model is 4 points or more, the determining unit 106 determines that the answer regarding company A does not include hallucination.

[0044] (Step S6) If the judgment unit 106 determines that the answer of the interactive AI 50 does not contain hallucination, step S7 is executed, and if it determines that the answer contains hallucination, step S8 is executed.

[0045] (Step S7) The transmitter / receiver 102 returns a response to the user terminal 20. That is, information about company A is returned as a response to the prompt sent by the user in step S1. The transmitting / receiving unit 102 also transmits the evaluation score obtained in the previous step S6 to the interactive AI 50.

[0046] (Step S8) If the predetermined condition is not met, step S9 is executed, and if the predetermined condition is met, step S10 is executed. Here, the predetermined condition is, for example, that the number of times Nr that step S9 is repeated exceeds a predetermined number, and this predetermined number is, for example, 10 times.

[0047] However, the judgment condition (predetermined condition) of this step S8 is not limited to whether or not the number of times Nr to repeat step S9 exceeds a predetermined number of times. Another example of the predetermined condition is, for example, whether the elapsed time since any of steps S4 to S6 was executed exceeds a predetermined length. That is, the predetermined conditions include the number of times that steps S4 (answer receiving process), S5, S6, S8, and S9 (review prompt sending process) are repeated exceeding a predetermined number of times, or the time required for the repeated process exceeding a predetermined length of time.

[0048] (Step S9) The transmitting / receiving unit 102 transmits a prompt indicating the evaluation score (an example of a reconsideration prompt) to the interactive AI 50. This prompt is, for example, the content of the evaluation by the machine learning model, such as "The evaluation score of your answer is 3 points." Based on the custom prompts initially entered, the conversational AI 50 understands the assessment points and returns the results of reconsidering the answers, i.e., generating an answer that is different from the previous answer.

[0049] (Step S10) The judgment unit 106 outputs an error and outputs the answer of the interactive AI 50 at that time as training data of the answer including hallucination. The output error is notified to the user terminal 20. The output training data is used to train the machine learning model, contributing to improving the accuracy of hallucination detection by the machine learning model.

[0050] Thus, the hallucination suppression system 10 is equipped with a custom prompt acquisition unit 104 that acquires a custom prompt generated from a user's prompt and transmits the custom prompt to the interactive AI 50, so that the interactive AI 50 can be expected to return a more accurate answer that suppresses hallucination. Furthermore, if the answer of the interactive AI 50 includes hallucination, the hallucination suppression system 10 causes the interactive AI 50 to reconsider the answer until it is determined that the answer does not include hallucination, thereby suppressing hallucination.

[0051] Although the embodiments of the present invention have been described above, the present invention is not limited to the above-described embodiments, and all changes in conditions that do not depart from the gist of the present invention are within the scope of application of the present invention. [Explanation of symbols]

[0052] 10 Hallucination Suppression System 20 user terminals 50 Conversational AI 102 Transmitter / Receiver 104 Custom prompt acquisition section 106 Judgment section

Claims

1. A custom prompt acquisition unit that acquires a custom prompt that is generated by the conversational AI based on a user's prompt and that is used to elicit a more accurate answer; A determination unit that determines whether or not hallucination is included in the answer of the interactive AI based on an evaluation by a machine learning model; A hallucination suppression system for an interactive AI, comprising: a transceiver unit that transmits the answer to a user when the judgment unit determines that the answer of the interactive AI does not contain hallucination.

2. 2. The hallucination suppression system of claim 1, A hallucination suppression system for an interactive AI, in which, when the judgment unit determines that the interactive AI's answer contains hallucination, the transceiver unit sends the interactive AI a reconsideration prompt to cause the interactive AI to reconsider the answer.

3. 3. The hallucination suppression system of claim 2, A hallucination suppression system for an interactive AI in which the transceiver unit performs a repetitive process of receiving an answer and sending the reconsideration prompt until the judgment unit determines that the answer of the interactive AI does not contain hallucination.

4. 4. The hallucination suppression system of claim 3, An interactive AI hallucination suppression system that outputs an error when predetermined conditions are met.

5. 5. The hallucination suppression system of claim 4, A hallucination suppression system for an interactive AI in which the judgment unit outputs the answer of the interactive AI when the error is output as training data of an answer including hallucination for the machine learning model.

6. 5. The hallucination suppression system of claim 4, An interactive AI hallucination suppression system in which the predetermined condition is that the number of times the repetitive processing has been performed has exceeded a predetermined number of times or the time required for the repetitive processing has exceeded a predetermined length.

7. 6. The hallucination suppression system of claim 5, The content of the custom prompt includes an instruction to request the conversational AI to generate a new answer if the answer does not satisfy a predetermined evaluation criterion; An interactive AI hallucination suppression system in which the content of the reconsideration prompt includes evaluation content by the machine learning model.

8. 8. The hallucination suppression system of claim 7, An interactive AI hallucination suppression system in which the predetermined evaluation criteria are evaluation points based on multiple levels.

9. In the interactive AI hallucination suppression system according to any one of claims 1 to 8, A hallucination suppression system in which the conversational AI that generates the custom prompt and the conversational AI that returns an answer to the custom prompt are each conversational AIs constructed based on different large-scale language models.

10. A determination unit that determines whether or not a hallucination is included in a response of the interactive AI to a prompt sent by a user based on a machine learning model; a transceiver that transmits the answer to the user when the determination unit determines that the answer of the interactive AI does not include hallucination; A hallucination suppression system for an interactive AI that, when the judgment unit determines that the answer of the interactive AI contains hallucination, causes the interactive AI to reconsider the answer until it is determined that it does not contain hallucination.

11. Computer, A custom prompt acquisition means for acquiring a custom prompt that is generated by the conversational AI based on a user's prompt and that is used to elicit a more accurate answer; a determining means for determining whether or not a hallucination is included in the response of the interactive AI to the custom prompt based on a machine learning model; A hallucination suppression program that functions as a transmission / reception means that transmits the answer to the user when the judgment means determines that the answer of the interactive AI does not contain hallucination.

Images