Information processing device, information processing method, and program

Abstract

This invention provides an information processing device that can prevent the generation of inappropriate answers while maintaining the quality of LLM responses. [Solution] In an information processing device, a prompt generation means generates a prompt that includes first information representing a first role, second information indicating a second role, and third information indicating a statement from the first role. A response acquisition means inputs the prompt to a language model and acquires a response from the language model.

Description

Technical Field

[0001] This disclosure relates to techniques for improving the safety of large language models.

Background Art

[0002] Attacks that cause large language models (LLMs) to generate answers (inappropriate answers) containing unethical or dangerous content are known. Patent Document 1 discloses a method of preventing the generation of inappropriate answers by implementing a filter.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, even with the method of Patent Document 1, it is not always possible to prevent the generation of inappropriate answers while maintaining the answer quality of the LLM.

[0005] One object of this disclosure is to provide an information processing apparatus capable of preventing the generation of inappropriate answers while maintaining the answer quality of the LLM.

Means for Solving the Problems

[0006] In one aspect of this disclosure, the information processing apparatus prompt generation means for generating a prompt including first information representing a first role, second information indicating a second role, and third information indicating a statement from the first role; answer acquisition means for inputting the prompt to a language model and acquiring an answer from the language model; and comprises.

[0007] From another perspective of this disclosure, the information processing method is: A computer-based information processing method, A prompt is generated that includes first information representing the first role, second information indicating the second role, and third information indicating a statement from the first role. An information processing method that inputs the aforementioned prompt into a language model and obtains a response from the language model.

[0008] In yet another aspect of this disclosure, the program is A prompt is generated that includes first information representing the first role, second information indicating the second role, and third information indicating a statement from the first role. The computer is instructed to input the aforementioned prompt into the language model and to perform the process of obtaining a response from the language model. [Effects of the Invention]

[0009] According to this disclosure, it is possible to prevent the generation of inappropriate answers while maintaining the quality of LLM responses. [Brief explanation of the drawing]

[0010] [Figure 1] This figure shows the overall configuration of the information processing device related to this disclosure. [Figure 2] This is a block diagram showing the hardware configuration of the information processing device related to this disclosure. [Figure 3] This is a block diagram showing the functional configuration of the information processing device related to this disclosure. [Figure 4] This is a diagram to explain the processing of the role specification section. [Figure 5] This is a flowchart of the processing performed by the information processing device related to this disclosure. [Figure 6] Here are some other examples of templates. [Figure 7] This is a block diagram showing the functional configuration of other information processing devices related to this disclosure. [Figure 8] This is a flowchart of the processing performed by other information processing devices related to this disclosure. [Embodiments of the Invention]

[0011] Hereinafter, preferred embodiments of the present disclosure will be described with reference to the drawings.

[0012] <First Embodiment> [Overview Explanation] A safeguard is implemented in a language model (hereinafter, for convenience of explanation, the language model is referred to as "LLM") so as not to generate inappropriate answers. However, the safeguard is not omnipotent, and for example, the existence of a "jailbreak prompt" in which a malicious user deliberately breaks through the safeguard has been confirmed. As defense methods against the above attacks, there are methods such as inserting question examples and exemplary answer examples into the prompt (in-context learning) and inserting persona (personality) specifications into the prompt (self-reminder). However, with the above methods, sufficient quality may not be obtained. Therefore, in the present embodiment, an LLM service is provided that generates high-quality answers for legitimate questions while preventing the generation of inappropriate answers.

[0013] Note that a language model is a model that outputs an answer to the input text in text. The type of language of the input text and the type of language of the output text do not necessarily have to match. The language model may be a model that outputs in a format different from text, such as an image or voice. The language model is, for example, an LLM (Large Language Models).

[0014] [Overall Configuration] FIG. 1 shows the overall configuration of the information processing apparatus according to the present embodiment. First, when the information processing apparatus 10 acquires an input sentence from a user, it generates a prompt based on the input sentence. Next, the information processing apparatus 10 inputs the generated prompt to the LLM and acquires an answer from the LLM. Then, the information processing apparatus 10 outputs the acquired answer as an answer to the input sentence.

[0015] Note that the input sentence is a question sentence, an instruction sentence, etc. entered by the user. Also, a prompt is text data used to instruct the LLM to perform a specific task.

[0016] The information processing apparatus 10 of the present embodiment is characterized in that it uses a structural sentence with a specified role as a prompt. A prompt consisting of a structural sentence is hereinafter also referred to as a "structural prompt". Examples of the roles in the present embodiment include a question side and an answer side, a user side and an LLM side, etc. Also, the structural sentence in the present embodiment is a sentence that distinguishes roles. Examples of structural sentences include question-and-answer format sentences such as Q&A collections, dialogue format sentences such as meeting minutes, and sentences in a format where items and explanations are paired, such as manuals. Note that the format indicates the structure, rules, and regulations when the sentence is used.

[0017] The information processing apparatus 10 can generate a structural prompt with a specified role by using a predetermined template. The predetermined template is a template for inserting a role name (that is, a template for specifying a role and distinguishing between a question side and an answer side). An example of the template is shown below. (Example of template) User:{input} ￥n Assistant:

[0018] "User" indicates the role name of the question side. "Assistant" indicates the role name of the answer side. "￥n" indicates a line break code. "{input}" is the insertion position of the input sentence. The information processing apparatus 10 can generate a prompt that associates the question side (User) with the input sentence and associates the answer side (Assistant) with the answer from the LLM by inserting the input sentence obtained from the user into {input}.

[0019] The information processing device 10 can generate prompts with a structure in which role names and their corresponding content are paired, using the above template. In other words, the information processing device 10 can generate structured prompts in which the questioner and the answerer are clearly specified and distinguished.

[0020] As described above, when the prompt text is structured, the probabilistic association between the structured text and the data the LLM has learned from increases, and this influence is more likely to be reflected in the output. In other words, LLM uses a large amount of text data as training data, and this includes structured text. Structured text can be found, for example, in meeting minutes, and these texts are likely to be ethical texts or official documents. Therefore, when a structured prompt is input to the LLM, the probability of the LLM selecting vocabulary and phrases related to ethical texts and official documents when generating answers increases. This makes it possible to obtain highly ethical answers from the LLM.

[0021] Furthermore, since the method of this embodiment uses only the minimum necessary additions (addition of role names) as prompts, it has less impact on the quality of LLM responses compared to cases where example questions, exemplary answers, or persona specifications are inserted into the prompts.

[0022] Through the above method, the information processing device 10 can output high-quality answers while ensuring ethical integrity.

[0023] Regarding the roles described above, the questioner and user are examples of the first role, while the answerer and LLM are examples of the second role. Furthermore, the template shown above is an example of a template that includes first information representing the first role and second information representing the second role.

[0024] [Hardware configuration] Figure 2 is a block diagram showing the hardware configuration of the information processing device 10 according to the first embodiment. As shown in the figure, the information processing device 10 includes an interface (I / F) 11, a processor 12, a memory 13, a recording medium 14, and a database (DB) 15.

[0025] I / F11 performs data input and output with external devices. Specifically, I / F11 receives input sentences and outputs responses to those input sentences. I / F11 may also communicate with external LLM services via a network such as the Internet.

[0026] Processor 12 is a computer such as a CPU (Central Processing Unit) and controls the entire information processing device 10 by executing a pre-prepared program. Processor 12 may be a GPU (Graphics Processing Unit), DSP (Digital Signal Processor), MPU (Micro Processing Unit), FPU (Floating Point Number Processing Unit), PPU (Physics Processing Unit), TPU (Tensor Processing Unit), quantum processor, microcontroller, or a combination thereof. Processor 12 executes the answer generation process described later.

[0027] Memory 13 consists of ROM (Read Only Memory), RAM (Random Access Memory), and other components. Memory 13 is also used as working memory while the processor 12 is executing various processes.

[0028] The recording medium 14 is a non-volatile, non-temporary recording medium such as a disk-shaped recording medium or semiconductor memory, and is configured to be detachable from the information processing device 10. The recording medium 14 stores various programs that the processor 12 will execute. When the information processing device 10 performs various processes, the programs stored on the recording medium 14 are loaded into the memory 13 and executed by the processor 12.

[0029] DB15 stores templates and other information used by the role specification section described later.

[0030] In addition to the above, the information processing device 10 may also be equipped with a display device such as a liquid crystal display, and an input device such as a keyboard or mouse. These display devices and input devices are used, for example, by the administrator of the information processing device 10 to perform necessary management.

[0031] [Functional Configuration] Figure 3 is a block diagram showing the functional configuration of the information processing device 10 of the first embodiment. Functionally, the information processing device 10 comprises an input sentence acquisition unit 101, a role specification unit 102, a response acquisition unit 103, and a response output unit 104.

[0032] The input text acquisition unit 101 acquires input text from the user. The input text acquisition unit 101 also acquires a template number. Input text is a question or instruction to the LLM, etc. The template number is a number used to uniquely identify the template, which will be described later. The template number is set in advance by the service provider, for example. The input text acquisition unit 101 outputs the acquired input text and template number to the role specification unit 102.

[0033] The role specification unit 102 retrieves the template corresponding to the template number from DB15. Then, the role specification unit 102 inserts the input text into the corresponding part of the retrieved template and generates a prompt. The role specification unit 102 outputs the generated prompt to the response acquisition unit 103. Note that one or more templates are prepared in advance and stored in DB15.

[0034] Figure 4 is a diagram illustrating the processing of the role specification unit 102. Figure 4(A) shows an example of a template used by the role specification unit 102. In Figure 4(A), multiple templates with different role names are provided. Note that in Figure 4(A), "User," "Human," and "Client" are used as the role names for the questioner, and "Assistant," "bot," and "Doctor" are used as the role names for the responder, but the role names are not limited to those listed above.

[0035] Figure 4(B) shows an example of prompt generation by the role specification unit 102. Assume that the role specification unit 102 has input Q and template number 1. As shown in Figure 4(B), the role specification unit 102 generates prompt Q' by substituting input Q into {input} of template 1.

[0036] Returning to Figure 3, the response acquisition unit 103 inputs a prompt to the LLM and obtains a response from the LLM. For example, by inputting the prompt Q' shown in Figure 4(B) to the LLM, the response acquisition unit 103 can obtain a message from the LLM that rejects an inappropriate response, such as "I'm sorry, I cannot respond it." The response acquisition unit 103 can use, for example, ChatGPT published by OpenAI, Llama (Large Language Model Meta AI) published by Meta, or a proprietary model as the LLM. The response acquisition unit 103 outputs the response from the LLM to the response output unit 104. The response output unit 104 outputs the response from the LLM to a display device or the user's terminal device.

[0037] In the above configuration, the input sentence acquisition unit 101 and the role specification unit 102 are examples of prompt generation means, and the answer acquisition unit 103 and the answer output unit 104 are examples of answer acquisition means.

[0038] [Processing flow] Next, the response generation process by the information processing device 10 will be explained. Figure 5 is a flowchart of the response generation process by the information processing device 10. This process is realized when the processor 12 shown in Figure 2 executes a pre-prepared program and operates as each element shown in Figure 3.

[0039] First, the input statement acquisition unit 101 acquires the input statement and template number (step S101). The input statement acquisition unit 101 outputs the acquired input statement and template number to the role specification unit 102.

[0040] Next, the role specification unit 102 retrieves a template based on the template number (step S102). Then, the role specification unit 102 inserts the input text into the retrieved template and generates a prompt (step S103). The role specification unit 102 outputs the generated prompt to the response acquisition unit 103.

[0041] Next, the response acquisition unit 103 inputs a prompt to the LLM and acquires a response from the LLM (step S104). The response acquisition unit 103 outputs the response from the LLM to the response output unit 104. The response output unit 104 outputs the response from the LLM (step S105). Then the process ends.

[0042] [Differentiation] Next, a modified version of the first embodiment will be described. The following modifications can be combined as appropriate and applied to the first embodiment.

[0043] (Variation 1) The role specification section 102 may also include prompts with techniques such as in-context learning and self-reminders. Figure 6 shows an example of a modified template. Underlines indicate examples of in-context learning. Dotted underlines indicate examples of self-reminders. Double underlines indicate role names. As shown in Figure 6, example questions and examples of refusing to answer have been added to the template as in-context learning. This allows the LLM to learn examples of refusing to answer and perform the task. Also, as shown in Figure 6, persona (character) specification has been added to the template as a self-reminder. This encourages the LLM to respond responsibly.

[0044] In this way, by clearly stating the role (inserting the role name), the accuracy of the response can be improved, mitigating the degradation of response accuracy caused by methods such as in-context learning and self-reminders, thus achieving a better balance between safety and high quality. Note that in Modification 1, tuning of the persona specification method is required, so the information processing device 10 can execute Modification 1 only if it has computational resources that can be tuned.

[0045] In-context learning and self-reminders are examples of fourth information that controls the behavior of the second role.

[0046] Furthermore, in-context learning is described, for example, in the following literature. The following literature is incorporated as references in this specification. Zeming Wei, Yifei Wang, Ang Li, Yichuan Mo, Yisen Wang, "Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrations," arXiv:2310.06387.

[0047] Furthermore, self-reminders are described, for example, in the following literature. The following literature is incorporated herein as a reference. Yueqi Xie, Jingwei Yi, Jiawei Shao, Justin Curl, Lingjuan Lyu, Qifeng Chen, Xing Xie & Fangzhao Wu, "Defending ChatGPT against jailbreak attack via self-reminders," Nature Machine Intelligence volume 5, pages1486-1496 (2023).

[0048] (Modification 2) The role specification section 102 may change the template used depending on the number of response rounds if a continuous conversation (multiple response rounds) is expected between the user and the LLM. For example, the role specification section 102 may select a template that includes the role name, in-context learning, and self-reminder when generating the initial prompt (first round of conversation), and select a template that includes only the role name when generating subsequent prompts (second round of conversation and beyond). In this way, by limiting the mention of in-context learning and self-reminder to only once per query, the redundancy of response rounds can be eliminated, and the quality of the LLM's response can be prevented.

[0049] (Variation 3) In the above embodiment, the role specification unit 102 selects a template from DB15 based on the template number. Alternatively, the role specification unit 102 may dynamically select a template based on the content of the user input text, in the manner of RAG (Retrieval Augmented Generation). Specifically, the role specification unit 102 extracts keywords contained in the input text and searches DB15 for templates containing phrases highly related to the extracted keywords. For example, if the role specification unit 102 extracts keywords such as pharmaceuticals or drugs from the input text, it can select the template "Client:{input} \n Doctor:" from DB15, which contains phrases highly related to pharmaceuticals or drugs. By dynamically selecting a template based on the content of the input text in this way, the improvement effect in terms of quality and accuracy based on the role name can be enhanced.

[0050] (Modification 4) The response acquisition unit 103 may remove unethical questions and answers in the input and output of the LLM using mechanisms such as filtering or guardrails. For example, the response acquisition unit 103 can filter out unethical expressions by checking the input content to the LLM and the output content from the LLM based on predetermined rules.

[0051] [Examples of application] Next, we will describe an example of how the information processing device 10 of this embodiment can be applied.

[0052] The information processing device 10 can be applied to automated response services for local governments. By using the information processing device 10, local governments can respond accurately and securely to questions from local residents. Furthermore, the information processing device 10 can be applied to manual case search in the manufacturing industry. The information processing device 10 can respond accurately to search requests from employees without leaking confidential information.

[0053] <Second Embodiment> Figure 7 is a block diagram showing the functional configuration of the information processing device of the second embodiment. The information processing device 20 includes a prompt generation means 201 and an answer acquisition means 202.

[0054] Figure 8 is a flowchart of the processing by the information processing device of the second embodiment. The prompt generation means 201 generates a prompt that includes first information representing a first role, second information indicating a second role, and third information indicating a statement from the first role (step S201). The response acquisition means 202 inputs the prompt to a language model and acquires a response from the language model (step S202).

[0055] The prompt generation means 201 can be implemented using the input sentence acquisition unit 101 and the role specification unit 102 according to the first embodiment. The response acquisition means 202 can be implemented using the response acquisition unit 103 and the response output unit 104 according to the first embodiment.

[0056] According to the information processing device 20 of the second embodiment, it is possible to prevent the generation of inappropriate answers while maintaining the quality of the LLM answers.

[0057] Some or all of the above embodiments may also be described as follows, but are not limited to the following:

[0058] (Note 1) A prompt generation means that generates a prompt including first information representing a first role, second information indicating a second role, and third information indicating a statement from the first role, A response acquisition means that inputs the aforementioned prompt to a language model and obtains a response from the language model, An information processing device equipped with the following features.

[0059] (Note 2) The prompt generation means is an information processing device according to Appendix 1, which obtains a template including the first information and the second information, and generates the prompt including the third information representing the input sentence and the template.

[0060] (Note 3) The first role mentioned above is to indicate the questioner, The second role described above is the information processing device described in Appendix 1, which indicates the responding side.

[0061] (Note 4) The first role mentioned above represents the user side, The second role described above is the information processing device described in Appendix 1, which represents the language model side.

[0062] (Note 5) The information processing device described in Appendix 2 includes at least one of the following structural texts: a question-and-answer format text, a dialogue format text, and a format text consisting of item-and-explanation pairs.

[0063] (Note 6) The information processing apparatus described in Appendix 1 includes a fourth piece of information that controls the operation of the second role.

[0064] (Note 7) The information processing device described in Appendix 6 includes at least one of the following: an example of refusal to answer, and a persona designation.

[0065] (Note 8) The information processing device according to Appendix 6, wherein the prompt generation means generates a prompt including the fourth information when generating the first prompt, and generates a prompt that does not include the fourth information when generating the second and subsequent prompts.

[0066] (Note 9) The template acquisition means is an information processing device according to Appendix 1 that acquires one template from a plurality of templates based on the input sentence.

[0067] (Note 10) The template acquisition means is an information processing device according to Appendix 9, which extracts keywords contained in the input sentence and acquires a template containing words or phrases highly related to the keywords.

[0068] (Note 11) A computer-based information processing method, A prompt is generated that includes first information representing the first role, second information indicating the second role, and third information indicating a statement from the first role. An information processing method that inputs the aforementioned prompt into a language model and obtains a response from the language model.

[0069] (Note 12) A prompt is generated that includes first information representing the first role, second information indicating the second role, and third information indicating a statement from the first role. A program that causes a computer to input the aforementioned prompt into a language model and retrieve a response from the language model.

[0070] Furthermore, some or all of the configurations described in Appendices 2 to 10, which are subordinate to Appendice 1 above, may also be subordinate to Appendices 11 and 12 in the same way as those described in Appendices 2 to 10. Moreover, not limited to Appendices 1, 11, and 12, some or all of the configurations described as appendices may also be subordinate to various hardware, software, various recording means for recording software, or systems, without departing from the embodiments described above.

[0071] Although the present disclosure has been described above with reference to embodiments and examples, the present disclosure is not limited to the above embodiments and examples. Various modifications to the structure and details of the present disclosure can be understood by those skilled in the art within the scope of the present disclosure. [Explanation of Symbols]

[0072] 10 Information Processing Devices 101 Input text acquisition unit 102 Section specifying roles 103 Answer acquisition part 104 Answer Output Section

Claims

1. A prompt generation means that generates a prompt including first information representing a first role, second information indicating a second role, and third information indicating a statement from the first role, A response acquisition means that inputs the aforementioned prompt to a language model and obtains a response from the language model, An information processing device equipped with the following features.

2. The information processing apparatus according to claim 1, wherein the prompt generation means obtains a template including the first information and the second information, and generates the prompt including the third information representing the input sentence and the template.

3. The first role mentioned above is to indicate the questioner, The second role is the information processing device according to claim 1, which indicates the responding side.

4. The first role mentioned above represents the user side, The information processing device according to claim 1, wherein the second role is to indicate the language model side.

5. The information processing apparatus according to claim 2, wherein the structural text includes at least one of the following: a question-and-answer format text, a dialogue format text, and a format text in which items and explanations are paired.

6. The information processing apparatus according to claim 1, wherein the prompt includes fourth information that controls the operation of the second role.

7. The information processing apparatus according to claim 6, wherein the fourth information includes at least one of the following: an example of refusal to answer, and a persona designation.

8. The information processing apparatus according to claim 6, wherein the prompt generation means generates a prompt including the fourth information when generating the first prompt, and generates a prompt that does not include the fourth information when generating the second and subsequent prompts.

9. A computer-based information processing method, A prompt is generated that includes first information representing a first role, second information indicating a second role, and third information indicating a statement from the first role. An information processing method that inputs the aforementioned prompt into a language model and obtains a response from the language model.

10. A prompt is generated that includes first information representing a first role, second information indicating a second role, and third information indicating a statement from the first role. A program that causes a computer to input the aforementioned prompt into a language model and retrieve a response from the language model.

Images