Question testing device, question testing method, and question testing program

Abstract

To provide a question testing device, a question testing method, and a question testing program that can detect an unethical inquiry to generative AI with high accuracy and deny an answer.SOLUTION: A question testing device 1 includes: a receiving unit 11 that receives input of an inquiry sentence to first generative AI; a determination unit 12 that makes a question to second generative AI as to whether or not the inquiry sentence leads to an unethical answer before the inquiry sentence is input to the first generative AI, and obtains an answer; and a responding unit 13 that inputs the inquiry sentence to the first generative AI only when an answer that the inquiry sentence does not lead to an unethical answer is obtained, and denies input to the first generative AI when an answer that the inquiry sentence leads to an unethical answer is obtained. A role of making an ethical answer as a cool-headed determiner of the second generative AI is fixed in advance in a prompt template.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a technique for detecting fraudulent use of generation AI. [Background technology]

[0002] In recent years, there has been rapid progress in the technology known as generative AI, which automatically generates images and other content from specified explanatory text. Furthermore, generative AI services that employ interactive natural language processing technology (see, for example, Non-Patent Document 1) have been provided, making it possible to generate advanced text such as manuscripts and summaries as replies to user inquiries based on a large number of information sources. [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] ChatGPT, Internet,<https: / / openai.com> Summary of the Invention [Problem to be solved by the invention]

[0004] Because generative AI may respond to unethical inquiries or inquiries that include personal information, it is usually equipped with security measures such as training it to detect and reject such problematic input. However, there are known techniques, such as jailbreaking, that can circumvent such measures, and further improvements in security mechanisms are desired.

[0005] The present invention aims to provide a question inspection device, a question inspection method, and a question inspection program that can accurately detect unethical inquiries made to a generation AI and refuse to answer them. [Means for solving the problem]

[0006] The question inspection device of the present invention comprises a reception unit that receives input of a query sentence to a first generation AI; a judgment unit that asks a second generation AI whether the query sentence will lead to an unethical answer before inputting the query sentence to the first generation AI and obtains an answer; and a response unit that inputs the query sentence to the first generation AI only if the judgment unit obtains an answer that the query sentence will not lead to an unethical answer, and rejects input to the first generation AI if the judgment unit obtains an answer that the query sentence will lead to an unethical answer, and the second generation AI has a role of providing ethical answers as a calm judge pre-defined in a prompt template.

[0007] The determination unit may input all past query sentences together into the second generation AI.

[0008] The determination unit may further summarize the query sentence together with summaries of past query sentences and input the summaries to the second generation AI.

[0009] The question inspection device of the present invention comprises a reception unit that receives input of a query sentence to a first generation AI, a judgment unit that asks a second generation AI whether or not the answer sentence from the first generation AI to the query sentence and obtains an answer before outputting the answer sentence, and a response unit that outputs the answer sentence only if the judgment unit obtains an answer that the answer sentence is not unethical, and does not output the answer sentence if the judgment unit obtains an answer that the answer sentence is unethical, and the second generation AI is pre-set in a prompt template to have the role of providing ethical answers as a calm judge.

[0010] The question inspection method of the present invention is performed by a computer, and includes a receiving step of receiving input of a query sentence to a first generation AI, a determination step of asking a second generation AI whether the query sentence will lead to an unethical answer before inputting the query sentence to the first generation AI and obtaining an answer, and a response step of inputting the query sentence to the first generation AI only if the determination step obtains an answer that the query sentence will not lead to an unethical answer, and refusing to input the query sentence to the first generation AI if the determination step obtains an answer that the query sentence will lead to an unethical answer, in which the second generation AI has a role of providing ethical answers as a calm judge pre-defined in a prompt template.

[0011] The question inspection method of the present invention is performed by a computer, and includes a receiving step of receiving an input of a query sentence to a first generation AI, a determination step of asking a second generation AI whether or not the answer sentence from the first generation AI to the query sentence is unethical and obtaining an answer before outputting the answer sentence, and a response step of outputting the answer sentence only if an answer is obtained in the determination step that the answer sentence is not unethical, and not outputting the answer sentence if an answer is obtained that the answer sentence is unethical, and the second generation AI is pre-set in a prompt template to play the role of providing ethical answers as a calm judge.

[0012] The inquiry checking program according to the present invention causes a computer to function as the inquiry checking device. Effect of the Invention

[0013] According to the present invention, unethical inquiries to a generation AI can be accurately detected and the answer can be refused. [Brief description of the drawings]

[0014] [Figure 1] 1 is a block diagram showing a functional configuration of a query inspection device in a first embodiment. FIG. [Diagram 2] FIG. 13 is a first diagram illustrating a question to a generation AI for judgment in the first embodiment. [Diagram 3] FIG. 2 is a second diagram illustrating a question to the generation AI for judgment in the first embodiment. [Figure 4] FIG. 3 is a third diagram illustrating a question to the generation AI for judgment in the first embodiment. [Diagram 5] FIG. 4 is a diagram showing a flow of processing by the query inspection device in the first embodiment. [Figure 6] FIG. 11 is a block diagram showing a functional configuration of a query inspection device in a second embodiment. [Figure 7] FIG. 11 is a diagram showing a flow of processing by a query inspection device in the second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0015] [First embodiment] A first embodiment of the present invention will now be described. In the question inspection method of this embodiment, a question to a generation AI is read into a generation AI with different settings, so that it is determined in advance whether the question is one that expects an unethical answer.

[0016] FIG. 1 is a block diagram showing a functional configuration of a query inspection device 1 according to the present embodiment. The inquiry inspection device 1 is an information processing device including a control unit 10, a storage unit 20, and various input / output interfaces.

[0017] The control unit 10 is a part that controls the entire query inspection device 1, and realizes each function in this embodiment by appropriately reading and executing various programs stored in the storage unit 20. The control unit 10 may be a CPU. The storage unit 20 is a storage area for various programs for causing the hardware group to function as the inquiry inspection device 1, various data, and the like, and may be a ROM, a RAM, a flash memory, a hard disk drive (HDD), or the like. Specifically, the control unit 10 executes software (question checking program) stored in the storage unit 20 to function as a receiving unit 11, a determining unit 12, and a responding unit 13.

[0018] The reception unit 11 receives an input of a query sentence from a user to a generation AI (first generation AI).

[0019] Before inputting the received query into the generation AI, the judgment unit 12 asks another generation AI (a second generation AI) that has been preset for judgment purposes whether the query will lead to an unethical answer, and obtains the answer.

[0020] Here, the role of the AI ​​for judgment is fixed in advance in the prompt template as a calm judge and to give ethical answers. For example, this is done by instructions such as "Answer the questions given by the user calmly, accurately, and ethically as an AI." As a result, the judgment unit 12 is only responsible for judging the ethics of the input query sentence in a context different from that of the original generation AI (first generation AI) without changing its role. The second generation AI may be a different model from the first generation AI, or the same model may be prepared as a different context.

[0021] FIG. 2 is a first diagram illustrating a question to be posed to the generation AI for judgment performed by the judgment unit 12 in this embodiment. When a user makes an initial inquiry, the inquiry is quoted and then checked to see whether there is a problem with it by asking a question such as, "Please tell us whether the inquiry contains any content related to criminal activity."

[0022] In addition, for the second or subsequent inquiry sentences, the inquiry sentence is similarly quoted and a question such as "Following the sentences given so far," "Please tell us whether the sentence contains any content that is related to criminal activity," is asked to check whether there is a problem with the new inquiry sentence within the context of the previous sentences.

[0023] Furthermore, when the reception unit 11 receives a plurality of inquiries in succession, the judgment unit 12 may input all of the past inquiry sentences together into the generation AI for judgment. Alternatively, the judgment unit 12 may combine the inquiry sentence with summaries of past inquiry sentences, further summarize the whole sentence, and input the summaries to the generation AI for judgment.

[0024] FIG. 3 is a second diagram illustrating a question to the generation AI for judgment performed by the judgment unit 12 in this embodiment. In this example, when all past queries are collectively queried, they are simply combined and quoted, and then the queries so far are checked to see if there are any problems.

[0025] FIG. 4 is a third diagram illustrating a question to the generation AI for judgment executed by the judgment unit 12 in this embodiment. When a user makes an initial inquiry, the inquiry is quoted and then summarized, and the summary is checked to see if there is a problem. For the second and subsequent queries, the previously checked summary and the new query are cited, and these are further summarized to check whether there are any problems in the summary.

[0026] The response unit 13 inputs the received query sentence into the original generation AI (first generation AI) and outputs the answer only when the judgment unit 12 receives a response from the judgment generation AI (second generation AI) that the query sentence will not lead to an unethical answer. On the other hand, if the response unit 13 receives a response that the query sentence will lead to an unethical answer, it rejects the input to the original generation AI (first generation AI).

[0027] FIG. 5 is a diagram showing a flow of processing by the query inspection device 1 in this embodiment. In the normal flow shown in the dashed box in the figure, the first inquiry (input to the prompt) to the generative AI is first checked for violations of the rules in moderation by formal input value checks. Next, the behavior of the generative AI is specified by the prompt template, and for example, unethical uses are rejected. After that, the query text is converted to a number by the encoder, and the number converted by the trained model is converted to an answer text by the decoder. This text is output as the answer (response) of the generative AI.

[0028] On the other hand, in this embodiment, the input to the prompt is input to a second generation AI, which is a different context prepared for judgment. In the second generation AI, as described above, the role is fixed by the prompt template, and only the ethicality of the query sentence is judged. If a query is found to be acceptable in this different context, it is transformed by the original learning model as normal, and an answer is output.

[0029] According to this embodiment, the question checking device 1 checks the ethics of a query sentence for a first generation AI by a second generation AI, and judges whether or not to output a reply sentence. Conventionally, ethics have been judged by registering a list of rejected words that are known in advance as moderation, or by having a model learn ethics. On the other hand, the method of the present embodiment does not implant ethics in the model to prevent it from answering problematic questions, but rather prepares a separate context that only judges ethics. This allows highly accurate judgments to be made for all questions, even if the learning model has not been trained to provide ethical answers. Therefore, the question inspection device 1 can accurately detect unethical questions to the generation AI and reject the answer.

[0030] Furthermore, the question checking device 1 can check ethics more accurately in context by inputting past inquiry sentences together into the second generation AI. In addition, the question inspection device 1 can save memory by summarizing past queries and storing only the summaries without storing the entire context. Furthermore, the questions to be judged are simplified, which is expected to improve the reliability of the judgment.

[0031] [Second embodiment] A second embodiment of the present invention will now be described. In the first embodiment, a query to the generation AI is read into a generation AI with different settings for judgment, whereas in the question inspection method of the second embodiment, before outputting an answer to a query to the generation AI, the answer is read into a generation AI with different settings to judge whether it is an unethical answer or not.

[0032] FIG. 6 is a block diagram showing the functional configuration of a query inspection device 1a in this embodiment. The configuration of the control unit 10a is similar to that of the first embodiment, but the functions of the determination unit 12a and the response unit 13a are different.

[0033] The receiving unit 11 receives input of a query sentence for the first generation AI, similarly to the first embodiment.

[0034] Before outputting a response sentence from a generation AI (first generation AI) to a query sentence, the judgment unit 12a asks another generation AI (second generation AI) for judgment as to whether the response sentence is unethical or not, and obtains a response.

[0035] The response unit 13a inputs the inquiry sentence into the first generation AI to obtain an answer sentence, but outputs the answer sentence only if the judgment unit 12a responds that the answer sentence is not unethical (there is no problem), and does not output the answer sentence if the judgment unit 12a responds that the answer sentence is unethical (there is a problem).

[0036] FIG. 7 is a diagram showing a process flow of the inquiry inspection device 1a in this embodiment. In the first embodiment, the query sentence to be passed from the prompt to moderation was input to the second generation AI, which is a different context. On the other hand, in the present embodiment, the answer sentence to be passed to the response is input to the second generation AI.

[0037] In the second generation AI, as in the first embodiment, the role is fixed by the prompt template, and only the ethicality of the answer sentence is judged. Answer sentences that are judged to be acceptable are output as answers by the original first generation AI as usual.

[0038] According to this embodiment, the question checking device 1a checks the ethics of the answer sentence by the first generation AI, that is, whether the inquiry is problematic, using the second generation AI, and judges whether or not to output the answer sentence. Therefore, the question checking device 1a can accurately detect unethical questions to the generating AI and refuse to answer them.

[0039] Furthermore, the question checking device 1a may also check answers to past inquiries using the second generating AI, as in the first embodiment, thereby making it possible to appropriately determine ethics within a context.

[0040] In addition, according to the above-mentioned embodiment, for example, it is possible to limit unethical use of generative AI, which will contribute to Goal 9 of the United Nations-led Sustainable Development Goals (SDGs), which is to "build resilient infrastructure, promote sustainable industrialization and foster innovation."

[0041] Although the embodiments of the present invention have been described above, the present invention is not limited to the above-described embodiments. Furthermore, the effects described in the above-described embodiments are merely a list of the most preferable effects resulting from the present invention, and the effects of the present invention are not limited to those described in the embodiments.

[0042] The question inspection method by the question inspection device 1 or 1a is realized by software. When realized by software, a program constituting this software is installed in an information processing device (computer). Moreover, these programs may be recorded on a removable medium such as a CD-ROM and distributed to users, or may be distributed by being downloaded to the user's computer via a network. Furthermore, these programs may be provided to the user's computer as a Web service via a network without being downloaded. [Explanation of symbols]

[0043] 1, 1a Questioning and Inspection Device 10, 10a Control section 11 Reception 12, 12a Judgment section 13, 13a Response section 20 Memory section

Claims

1. A reception unit that receives an input of a query sentence to the first generation AI; a determination unit that, before inputting the query sentence into the first generation AI, asks a second generation AI whether the query sentence will lead to an unethical answer and obtains an answer from the second generation AI; a response unit that inputs the query sentence to the first generation AI only when the determination unit receives a response that the query sentence will not lead to an unethical response, and that rejects input to the first generation AI when the determination unit receives a response that the query sentence will lead to an unethical response; The second generating AI is a question inspection device in which the role of providing ethical answers as a calm judge is fixed in a prompt template in advance.

2. The question inspection device according to claim 1 , wherein the determination unit inputs past query sentences together into the second generation AI.

3. The question inspection device according to claim 1 , wherein the determination unit further summarizes the query sentence together with summaries of past query sentences and inputs the summaries to the second generation AI.

4. A reception unit that receives an input of a query sentence to the first generation AI; a determination unit that, before outputting a response sentence from the first generation AI to the query sentence, asks a second generation AI whether the response sentence is unethical or not, and obtains a response; a response unit that outputs the answer sentence only when the determination unit determines that the answer sentence is not unethical, and does not output the answer sentence when the determination unit determines that the answer sentence is unethical; The second generating AI is a question inspection device in which the role of providing ethical answers as a calm judge is fixed in a prompt template in advance.

5. A receiving step of receiving an input of a query sentence to the first generation AI; a determination step of asking a second generation AI whether the query sentence will lead to an unethical answer before inputting the query sentence to the first generation AI, and obtaining an answer from the second generation AI; a response step of inputting the query sentence to the first generation AI only if a response is received in the determination step that the query sentence will not lead to an unethical response, and refusing to input the query sentence to the first generation AI if a response is received in the determination step that the query sentence will lead to an unethical response; A question inspection method in which the second generating AI is pre-assigned the role of providing ethical answers as a calm judge in a prompt template.

6. A receiving step of receiving an input of a query sentence to the first generation AI; a determination step of asking a second generation AI whether or not the reply sentence from the first generation AI to the query sentence is unethical and obtaining a reply before outputting the reply sentence from the first generation AI; a response step of outputting the answer sentence only when a response is received in the determination step that the answer sentence is not unethical, and not outputting the answer sentence when a response is received in the determination step that the answer sentence is unethical; A question inspection method in which the second generating AI is pre-assigned the role of providing ethical answers as a calm judge in a prompt template.

7. A question inspection program for causing a computer to function as the question inspection device according to any one of claims 1 to 4.

Images