Hallucination Suppression System and Program

Abstract

Provided are a hallucination suppression system and a program capable of suppressing hallucinations of a dialogue-type AI. 【Solution means】A hallucination suppression system 10 for a dialogue-type AI includes a custom prompt acquisition unit 104 that acquires a custom prompt that is a prompt generated by the dialogue-type AI 50 based on a user's prompt and is for eliciting a more accurate answer, a determination unit 106 that determines whether or not a hallucination is included in the answer of the dialogue-type AI 50 to the custom prompt based on an evaluation by a machine learning model, and a transmission / reception unit 102 that transmits the answer to the user when it is determined that no hallucination is included in the answer of the dialogue-type AI.

Description

Technical Field

[0001] The present invention relates to a hallucination suppression system and a program.

Background Art

[0002] Non-Patent Document 1 describes how ChatGPT (「CHATGPT」 is an international registered trademark) operates and why it functions well, and also describes that ChatGPT sometimes generates information that does not exist in actual data, that is, hallucination.

Prior Art Documents

Patent Documents

[0003]

Non-Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] An object of the present invention is to provide a hallucination suppression system and a program capable of suppressing hallucination of a dialogue-type AI.

Means for Solving the Problems

[0005] The invention according to claim 1 is a custom prompt generated by an interactive AI based on a user's prompt, and includes a custom prompt acquisition unit that acquires a custom prompt for obtaining a more accurate answer, a determination unit that determines whether hallucination is included in the answer of the interactive AI based on an evaluation by a machine learning model, and when the determination unit determines that hallucination is not included in the answer of the interactive AI, transmits the answer to a user terminal Transmission and a unit When the determination unit determines that the response of the dialogue-type AI contains hallucination, the transmission unit repeatedly transmits a reconsideration prompt for reconsidering the response until it is determined that no hallucination is included. When the determination unit satisfies a predetermined condition, the response of the dialogue-type AI is output as teacher data of a response including hallucination for the machine learning model It is a hallucination suppression system for an interactive AI

[0006] The invention according to claim 2 is A custom prompt acquisition unit that acquires a custom prompt for obtaining a more accurate response, which is a prompt generated by the dialogue-type AI based on the user's prompt, a determination unit that determines whether or not the response of the dialogue-type AI contains hallucination based on an evaluation by a machine learning model, and a transmission unit that transmits the response to the user terminal when the determination unit determines that the response of the dialogue-type AI does not contain hallucination. When the determination unit determines that the response of the dialogue-type AI contains hallucination, the transmission unit executes a process of repeatedly transmitting a reconsideration prompt for reconsidering the response until it is determined that no hallucination is included, and outputs an error when the number of times of the process exceeds a predetermined number of times or when the time required for the process exceeds a predetermined length. It is a hallucination suppression system for a dialogue-type AI 。

[0007]

[0008]

[0009]

[0010]

[0011] Claim 3 The invention described in is, in the hallucination suppression system for an interactive AI described in claim 1 or 2 wherein the content of the custom prompt includes an instruction to request generation of a reconsideration prompt when the answer of the interactive AI does not meet a predetermined evaluation criterion, and the content of the reconsideration prompt includes an evaluation by the machine learning model is re including valued 。

[0012] Claim 4 The invention described in is, in the hallucination suppression system for an interactive AI described in claim 3 wherein the predetermined evaluation criterion is an evaluation score based on multiple levels

[0013] Claim 5 The invention described in is, in the hallucination suppression system for an interactive AI described in claim 1 or 2In the hallucination suppression system of the described interactive AI, the interactive AI that generates the custom prompt and the interactive AI that returns an answer to the custom prompt are each interactive AIs configured based on different large language models.

[0014] The invention according to claim 6 causes a computer to function as a custom prompt acquisition means for acquiring a custom prompt that is a prompt generated by an interactive AI based on a user's prompt and that can elicit a more accurate answer, a determination means for determining based on a machine learning model whether hallucination is included in the answer of the interactive AI to the custom prompt, and a transmission means for transmitting the answer to a user terminal when the determination means determines that hallucination is not included in the answer of the interactive AI. When the determination means determines that hallucination is included in the answer of the interactive AI, the transmission means repeatedly transmits a reconsideration prompt for causing the answer to be reconsidered until it is determined that hallucination is not included. When the determination means satisfies a predetermined condition, it is a hallucination suppression program that outputs the answer of the interactive AI as teacher data for an answer including hallucination for the machine learning model.

[0015] Claim 7 The invention according to claim is to cause a computer to function as a custom prompt acquisition means for acquiring a custom prompt that is a prompt generated by an interactive AI based on a user's prompt and is for eliciting a more accurate answer, a determination means for determining whether or not hallucination is included in the answer of the interactive AI to the custom prompt based on a machine learning model, and when the determination means determines that hallucination is not included in the answer of the interactive AI, transmitting the answer to a user terminal Transmission means, , when the determination means determines that hallucination is included in the answer of the interactive AI, the transmission means executes a process of repeatedly transmitting a reconsideration prompt for causing the answer to be reconsidered until it is determined that hallucination is not included, and outputs an error when the number of times of the process exceeds a predetermined number of times or when the time required for the process exceeds a predetermined length. It is a hallucination suppression program.

Effect of the Invention

[0016] According to the present invention, it is possible to provide a hallucination suppression system and program that can suppress hallucination of an interactive AI.

Brief Description of the Drawings

[0017]

Figure 1

Figure 2

Embodiments for Carrying Out the Invention

[0018] Subsequently, with reference to the attached drawings, embodiments embodying the present invention will be described to aid understanding of the present invention. In the figures, parts not relevant to the description may be omitted from the illustration.

[0019] The hallucination suppression system 10 according to an embodiment of the present invention can suppress the inclusion of hallucinations in the responses of the conversational AI to the prompts sent by the user. The conversational AI is, for example, ChatGPT provided by OpenAI (「CHATGPT」 is an international registered trademark) and is configured based on large language models.

[0020] As shown in FIG. 1, the hallucination suppression system 10 includes a transceiver 102, a custom prompt acquisition unit 104, and a determination unit 106.

[0021] The transceiver (an example of transceiver means) 102 is connected to the user terminal 20 and the conversational AI 50 provided as a cloud service via a network. Therefore, the transceiver 102 can receive the prompt input by the user from the user terminal 20 and transmit data such as the response to the prompt. Further, the transceiver 102 can transmit the prompt to the conversational AI 50 and receive the response to the prompt from the conversational AI 50.

[0022] Here, the prompt transmitted to the conversational AI 50 includes a custom prompt. This custom prompt is a prompt generated by the conversational AI based on the user's prompt and is a prompt for eliciting a more accurate response from the conversational AI 50.

[0023] Specifically, the custom prompt is described, for example, based on markdown notation and includes at least the following content. (1) Instruction The instruction indicates the specific content of the question or request by clearly showing the information or task required of the conversational AI 50. As a specific example of the instruction, there is the content of "Analyze this dataset and report the main statistical indicators".

[0024] (2) Role The role indicates the role expected of the interactive AI 50 by showing from what position and perspective the interactive AI 50 should answer. As a specific example of the role, there is the content "You are an expert in hydraulics."

[0025] (3) Output Limit The output limit restricts the format and scope of the answer by specifying the length, detail, format, etc. of the answer. As specific examples of the output limit, there are contents such as "Please answer briefly within 200 characters." and "Please explain in a way that can be understood by high school students."

[0026] (4) Evaluation Evaluation indicates that the answer of the interactive AI 50 is evaluated by a predetermined method, and if it does not meet the predetermined evaluation criteria, it instructs to generate an answer again. As a specific example of the evaluation, there is the content "The answer is always evaluated in three steps. If the evaluation is 2 or less, please consider another answer." Note that this evaluation may be included in the aforementioned instructions. Also, the evaluation criteria are not limited to multi-step evaluations.

[0027] (5) Reference Example The reference example shows an example of a desirable answer. As a specific example of the reference example, there is the content "Including an overview of the failure, cause investigation, and repair procedure."

[0028] (6) Execution Procedure The execution procedure details the specific procedure for executing the task.

[0029] Note that at least a part of the content of "(2) Role", "(3) Output Limit", "(4) Evaluation", "(5) Reference Example", and "(6) Execution Procedure" may be described as "(1) Instruction". Incidentally, the data between the transmission / reception unit 102 and the dialogue AI 50 is transmitted and received using an API.

[0030] The custom prompt acquisition unit (an example of custom prompt acquisition means) 104 transmits a custom prompt generation prompt for generating a custom prompt based on the user's prompt received by the transmission / reception unit 102 to the dialogue AI 50, and can acquire the custom prompt as an answer generated by the dialogue AI 50. The data between the custom prompt acquisition unit 104 and the dialogue AI 50 is transmitted and received using an API. The custom prompt acquired by the custom prompt acquisition unit 104 is transmitted from the transmission / reception unit 102 to the dialogue AI 50.

[0031] The determination unit (an example of determination means) 106 can determine whether or not the answer of the dialogue AI 50 to the custom prompt contains hallucination based on an evaluation by a machine learning model. The machine learning model has previously learned answers containing hallucinations and can determine whether the generated answer contains hallucinations. The machine learning model can output the degree of hallucination as evaluation points at multiple levels (for example, evaluation points by a five-level evaluation). The evaluation output by the machine learning model is not limited to evaluation points at multiple levels, and may be content corresponding to the evaluation criteria specified in the "evaluation" of the custom prompt.

[0032] When the determination unit 106 determines that the answer of the dialogue AI 50 does not contain hallucination, it can instruct the transmission / reception unit 102 to transmit the answer to the user terminal 20. On the other hand, when the determination unit 106 determines that the answer of the dialogue AI 50 contains hallucination, it can generate a reconsideration prompt for causing the dialogue AI 50 to reconsider the answer, and instruct the transmission / reception unit 102 to transmit the reconsideration prompt to the dialogue AI 50.

[0033] The reconsideration prompt is the evaluation content output by the machine learning model. As described above, in the "evaluation" included in the custom prompt, when the evaluation criteria are not met, the interactive AI 50 is instructed to generate an answer again. Therefore, by simply sending the evaluation output by the machine learning model as a prompt, the interactive AI 50 reconsiders the answer and generates a new answer. The reconsideration prompt is not limited to the evaluation output by the machine learning model, and any content that causes the interactive AI 50 to reconsider the answer is acceptable.

[0034] Note that at least the transmission / reception unit 102, the custom prompt acquisition unit 104, and the determination unit 106 are realized by a computer program and function as transmission / reception means, custom prompt acquisition means, and determination means, respectively. Also, it is not limited to one computer having all of the transmission / reception unit 102, the custom prompt acquisition unit 104, and the determination unit 106. It may be configured by being divided into a plurality of computers with at least a part of them connected to each other via a network, or at least a part of the functions may be realized by cloud computing. Furthermore, the interactive AI that generates the custom prompt and the interactive AI that returns an answer to the custom prompt do not have to be the same interactive AI, and may be interactive AIs configured based on different large language models.

[0035] Next, the operation of the hallucination suppression system 10 (hallucination suppression method) will be described with reference to FIG. 2. Hallucinations are suppressed according to the following steps S1 to S10. However, if possible, each step may be executed by being swapped or executed in parallel. For ease of understanding, a virtual example in which a user obtains information about Company A from the interactive AI 50 will be described together.

[0036] (Step S1) The user sends a prompt from the user terminal 20. The content of the prompt is, for example, "Please tell me about Company A at the following URL. https: / / e-ryowa.com / page1.html".

[0037] (Step S2) The transceiver 102 of the hallucination suppression system 10 receives the user's prompt. The custom prompt acquisition unit 104 sends a custom prompt generation prompt to the dialog AI 50, instructing it to generate a custom prompt that can obtain a more accurate answer from the received prompt. The dialog AI 50 returns an answer to the custom prompt generation prompt. That is, the dialog AI 50 generates a custom prompt based on the user's prompt. The generated custom prompt is acquired by the custom prompt acquisition unit 104.

[0038] The custom prompt returned by the dialog AI 50 has the following content, for example. (1) Instruction "Crawl the following URL and summarize the company's content in about 200 characters. https: / / e-ryowa.com / page1.html" (2) Role "You are a capable prompt engineer. Your goal is to create the best prompt that meets my needs." (3) Output limit "Please explain it in a way that even a primary school student can understand." (4) Evaluation "The answer is always evaluated on a 5-point scale. If the evaluation score is 3 or less, please consider another answer." (5) Reference example "Company name: {Company name} Title: {Title} Text: {Text} " (6) Execution procedure "The user provides the URL of the company's website. You crawl the specified URL, extract relevant information about the company, and then briefly summarize the content of the company in about 200 words."

[0039] The above-mentioned "(4) Evaluation" indicates that the evaluation criteria are evaluation points based on a five-level evaluation, and the evaluation points (1 to 5 points) correspond to the evaluation points output by the machine learning model.

[0040] (Step S3) The transceiver unit 102 transmits the custom prompt acquired by the custom prompt acquisition unit 104 to the dialog AI 50. As a result, the dialog AI 50 outputs information about Company A as an answer to the custom prompt.

[0041] (Step S4) The transceiver unit 102 receives the answer generated by the dialog AI 50.

[0042] (Step S5) The determination unit 106 obtains an evaluation point representing the degree to which hallucination is included in the answer from a machine learning model that has learned answers including hallucination in advance, and based on this evaluation point, determines whether the answer generated by the dialog AI 50 includes hallucination.

[0043] Specifically, for example, if the evaluation point output by the machine learning model for the answer is 3 points or less, the determination unit 106 determines that the answer regarding Company A includes hallucination. On the other hand, for example, if the evaluation point output by the machine learning model is 4 points or more, the determination unit 106 determines that the answer regarding Company A does not include hallucination.

[0044] (Step S6) When the determination unit 106 determines that the response of the dialogue AI 50 does not include hallucination, step S7 is executed. When it is determined that the response includes hallucination, step S8 is executed.

[0045] (Step S7) The transmission / reception unit 102 returns a response to the user terminal 20. That is, information on Company A is returned as a response to the prompt transmitted by the user in step S1. In addition, the transmission / reception unit 102 transmits the evaluation score in the previous step S6 to the dialogue AI 50.

[0046] (Step S8) If the predetermined conditions are not met, step S9 is executed. If the predetermined conditions are met, step S10 is executed. Here, the predetermined condition is, for example, that the number of repetitions Nr of step S9 exceeds a predetermined number of times, and this predetermined number of times is, for example, 10 times.

[0047] However, the determination condition (predetermined condition) of this step S8 is not limited to whether the number of repetitions Nr of step S9 exceeds a predetermined number of times. Other examples of the predetermined condition include, for example, that the elapsed time since any one of steps S4 to S6 was executed exceeds a predetermined length. That is, examples of the predetermined condition include that the number of repetitions of the repetitive processes of step S4 (response reception process), S5, S6, S8, and S9 (reconsideration prompt transmission process) exceeds a predetermined number of times, or that the time required for the repetitive process exceeds a predetermined length.

[0048] (Step S9) The transmission / reception unit 102 transmits a prompt indicating the evaluation score (an example of a reconsideration prompt) to the dialogue AI 50. This prompt is, for example, the content of the evaluation by the machine learning model such as "The evaluation score of the response is 3 points." The dialogue AI 50 understands the evaluation points based on the initially input custom prompt and returns the result of reconsidering the answer. That is, it generates an answer different from the previous answer.

[0049] (Step S10) The determination unit 106 outputs an error and outputs the answer of the dialogue AI 50 at that time as teacher data of an answer including hallucination. The output error is notified to the user terminal 20. The output teacher data is used for the learning of the machine learning model and contributes to improving the accuracy of the determination of hallucination by the machine learning model.

[0050] Thus, according to the hallucination suppression system 10, since it includes a custom prompt acquisition unit 104 that acquires a custom prompt generated from the user's prompt and transmits the custom prompt to the dialogue AI 50, it can be expected that the dialogue AI 50 returns a more accurate answer with hallucination suppressed. Further, when the answer of the dialogue AI 50 includes hallucination, the hallucination suppression system 10 causes the dialogue AI 50 to reconsider the answer until it is determined that no hallucination is included, so that hallucination is suppressed.

[0051] As described above, although the embodiments of the present invention have been described, the present invention is not limited to the above-described embodiments, and all changes and the like that do not deviate from the gist are within the scope of application of the present invention.

Explanation of Signs

[0052] 10 Hallucination suppression system 20 User terminal 50 Dialogue AI 102 Transmission / reception unit 104 Custom prompt acquisition unit 106 Determination unit

Claims

1. A custom prompt acquisition unit that acquires a custom prompt that is generated by the conversational AI based on a user's prompt and is used to elicit a more accurate answer; A determination unit that determines whether or not the answer of the conversational AI includes hallucination based on an evaluation by a machine learning model; A transmission unit that transmits the answer of the conversational AI to a user terminal when the determination unit determines that the answer does not include hallucination, When the determination unit determines that the answer of the conversational AI includes hallucination, the transmission unit repeatedly transmits a reconsideration prompt to cause the answer to be reconsidered until it is determined that the answer does not include hallucination; A hallucination suppression system for an interactive AI that outputs the answer of the interactive AI as teacher data of an answer including hallucination for the machine learning model when the judgment unit satisfies predetermined conditions.

2. A custom prompt acquisition unit that acquires a custom prompt that is generated by the conversational AI based on a user's prompt and is used to elicit a more accurate answer; A determination unit that determines whether or not the answer of the conversational AI includes hallucination based on an evaluation by a machine learning model; A transmission unit that transmits the answer of the conversational AI to a user terminal when the determination unit determines that the answer does not include hallucination, When the determination unit determines that the answer of the conversational AI includes hallucination, the transmission unit executes a process of repeatedly transmitting a reconsideration prompt to cause the answer to be reconsidered until it is determined that the answer does not include hallucination; An interactive AI hallucination suppression system that outputs an error if the number of times the processing exceeds a predetermined number or if the time required for the processing exceeds a predetermined length.

3. In the hallucination suppression system of the interactive AI according to claim 1 or 2, The content of the custom prompt includes an instruction to generate a new answer if the answer of the conversational AI does not satisfy a predetermined evaluation criterion; An interactive AI hallucination suppression system, wherein the content of the reconsideration prompt includes an evaluation by the machine learning model.

4. In the conversational AI hallucination suppression system according to claim 3, An interactive AI hallucination suppression system, wherein the predetermined evaluation criteria are evaluation points based on a plurality of stages.

5. In the hallucination suppression system of the interactive AI according to claim 1 or 2, A conversational AI hallucination suppression system, in which the conversational AI that generates the custom prompt and the conversational AI that returns an answer to the custom prompt are each conversational AIs constructed based on different large-scale language models.

6. Computer, A custom prompt acquisition means for acquiring a custom prompt that is generated by the conversational AI based on a user's prompt and is used to elicit a more accurate answer; A determination means for determining whether or not a hallucination is included in the response of the interactive AI to the custom prompt based on a machine learning model; When the determination means determines that the answer of the conversational AI does not include hallucination, the determination means functions as a transmission means for transmitting the answer to a user terminal; When the determination means determines that the answer of the conversational AI includes hallucination, the transmission means repeatedly transmits a reconsideration prompt to cause the answer to be reconsidered until it is determined that the answer does not include hallucination; A hallucination suppression program that outputs the answer of the interactive AI as teacher data of an answer including hallucination for the machine learning model when the judgment means satisfies predetermined conditions.

7. Computer, A custom prompt acquisition means for acquiring a custom prompt that is generated by the conversational AI based on a user's prompt and is used to elicit a more accurate answer; A determination means for determining whether or not a hallucination is included in the response of the interactive AI to the custom prompt based on a machine learning model; When the determination means determines that the answer of the conversational AI does not include hallucination, the determination means functions as a transmission means for transmitting the answer to a user terminal; When the determination means determines that the answer of the conversational AI includes hallucination, the transmission means executes a process of repeatedly transmitting a reconsideration prompt to cause the answer to be reconsidered until it is determined that the answer does not include hallucination; A hallucination suppression program that outputs an error when the number of times the process is performed exceeds a predetermined number of times or when the time required for the process exceeds a predetermined length of time.

Images