Information processing device, information processing method, and program

Abstract

This information processing device comprises at least one memory and at least one processor. The at least one processor acquires an evaluation result obtained by evaluating one or more first templates using a first generation model, generates input information to be input to a second generation model on the basis of the evaluation result, and acquires a second template generated by inputting the input information to the second generation model. The input information includes at least one or more pieces of template information selected on the basis of the evaluation result and information for designating the structure of the templates.

Description

Information processing device, information processing method, and program

[0001] The present disclosure relates to an information processing device, an information processing method, and a program.

[0002] Generative models such as large language models (LLMs) are known. Generative models are machine learning models that execute a given task according to input information called prompts and output the resulting data. Techniques for optimizing prompts have been proposed to obtain good output results from generative models.

[0003] To generate prompts for input to the generative model, a template engine is used, which is a program that uses predefined templates to generate text data based on input data.

[0004] Yongchao Zhou, Andrei Ioan Muresanu, Ziwen Han, Keiran Paster, Silviu Pitis, Harris Chan, Jimmy Ba, "Large Language Models Are Human-Level Prompt Engineers", [online], [Searched on September 1, 2020], Internet<URL: https: / / arxiv.org / pdf / 2211.01910v2.pdf>

[0005] An object of the present disclosure is to provide a technology for optimizing templates for generating input information to a generative model.

[0006] An information processing device according to one aspect of the present disclosure includes at least one memory and at least one processor, wherein the at least one processor obtains evaluation results obtained by evaluating one or more first templates using a first generative model, generates input information for a second generative model based on the evaluation results, and obtains a second template generated by inputting the input information to the second generative model, wherein the input information includes at least one or more template information selected based on the evaluation results and information specifying the structure of the template.

[0007] FIG. 1 is a block diagram showing an example of the overall configuration of an information processing system. FIG. 2 is a block diagram showing an example of the functional configuration of an optimization device in the first embodiment. FIG. 3 is a diagram showing a first example of a template of a rewrite prompt. FIG. 4 is a diagram showing an example of a template of an initial generation prompt. FIG. 5 is a diagram showing an example of an initial generation prompt. FIG. 6 is a diagram showing an example of an initial template. FIG. 7 is a block diagram showing an example of the functional configuration of an execution device. FIG. 8 is a flowchart showing an example of an optimization process in the first embodiment. FIG. 9 is a flowchart showing an example of an evaluation process. FIG. 10 is a flowchart showing an example of a rewrite process. FIG. 11 is a flowchart showing an example of an execution process. FIG. 12 is a diagram showing a second example of a template of a rewrite prompt. FIG. 13 is a block diagram showing an example of the functional configuration of an optimization device in Modification 1. FIG. 14 is a diagram showing an example of a prompt template in Modification 2. FIG. 15 is a diagram showing an example of a feedback information input field. FIG. 16 is a diagram showing an example of an evaluation value input field. FIG. 17 is a block diagram showing an example of the functional configuration of an optimization device in the second embodiment. FIG. 18 is a flowchart showing an example of an additional learning process in the second embodiment. FIG. 19 is a block diagram showing an example of the hardware configuration of a computer.

[0008] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. In this specification and drawings, components having substantially the same functional configurations are designated by the same reference numerals, and redundant description will be omitted.

[0009] [First Embodiment] A first embodiment of the present disclosure is an information processing system that generates various data based on a generative model. The information processing system in this embodiment has a function of optimizing input information to be input to the generative model. In this embodiment, the generative model may be, for example, a large language model (LLM). The generative model is not limited to a large language model, and may be a model that generates other data such as audio or images, or may be a base model.

[0010] It is known that the output results of a generative model vary depending on the content of the prompt. It is also known that the prompts that produce good output results from a generative model vary depending on the type of generative model and the task that the generative model is made to execute. Techniques for optimizing prompts have been proposed to obtain good output results from generative models. This type of technique is called prompt engineering, etc.

[0011] Conventionally, prompt engineering has been performed by experts through trial and error. That is, in conventional prompt engineering, the output result when a prompt is input into a generative model is obtained, the expert rewrites the prompt based on the output result, and the output result when the rewritten prompt is input into the generative model is obtained, and this process is repeated until a satisfactory output result is obtained. Meanwhile, technologies for automatically optimizing prompts have been proposed. This type of technology automatically performs prompt rewriting according to a predetermined algorithm, instead of the trial and error that experts previously performed.

[0012] Conventional prompt engineering focuses on how to instruct a generative model to execute a desired task. In contrast, this embodiment optimizes the structure of the prompt without changing the semantic content of the instruction, thereby causing the generative model to execute the desired task.

[0013] An example of an instruction might be, "Below is a combination of an instruction that describes a task and contextual input. Write a response that satisfies the requirement appropriately." This instruction can be written in the following three formats (1) to (3), for example. The instructions shown below all have the same meaning, but the document structure (specifically, the symbols indicating headings) is different.

[0014] (1) "Instructions: Below are instructions that describe the task and input with context. Write a response that satisfies the requirement appropriately." (2) "Instructions: Below are instructions that describe the task and input with context. Write a response that satisfies the requirement appropriately." (3) "##Instructions: Below are instructions that describe the task and input with context. Write a response that satisfies the requirement appropriately."

[0015] There are various formats for writing prompts, but it is not obvious which format will produce good output results. In addition, the format that produces good output results may differ depending on the type of generative model and the content of the task.

[0016] Another example may be a prompt that includes a question and answer options and instructs a user to select the correct answer option. This prompt includes the content of the "question" and the "options," but it is not obvious in what order these contents should be written to improve the accuracy rate. For example, when considering whether the order "question" → "options" or "options" → "question" is better, even if the former seems more natural to humans, it is not obvious which order will output a better result for a generative model.

[0017] A template engine is used to generate prompts to be input into a generative model. A template engine is a program that generates text data based on input data using predefined templates. Template engines are sometimes used to generate, for example, web pages and source code. By using a template engine, users can efficiently generate complex prompts.

[0018] In this embodiment, optimization of input information to a generative model is achieved by optimizing a template for generating input information to the generative model. In this embodiment, the template is evaluated based on the generative model, and the template is rewritten based on the evaluation results, repeatedly. In the template rewriting, the structure of the template is changed so as to maintain the semantics of the input information. In one aspect, according to this embodiment, good output results can be obtained from the generative model without changing the semantics of the input information.

[0019] <Overall Configuration of Information Processing System> The overall configuration of an information processing system according to this embodiment will be described with reference to Fig. 1. Fig. 1 is a block diagram showing an example of the overall configuration of an information processing system.

[0020] 1, the information processing system 1000 includes two generating devices 10 (10-1, 10-2), an optimization device 20, an execution device 30, and a terminal device 40. The generating device 10, the optimization device 20, the execution device 30, and the terminal device 40 may be connected to each other so as to be able to communicate data with each other via a communication network such as a local area network (LAN) or the Internet.

[0021] Hereinafter, when distinguishing between the multiple generation devices 10, they will be referred to using sub-numbers such as "generation device 10-1," "generation device 10-2," etc. When simply referred to as "generation device 10," this applies to all generation devices 10.

[0022] The generating device 10 is an information processing device such as a personal computer, workstation, or server that executes a predetermined task using a generative model. One generating device 10-1 may include a target generative model 11. The other generating device 10-2 may include a template generative model 12.

[0023] The target generation model 11 is a generation model for executing a predetermined task. The template generation model 12 is a generation model for executing a task of generating a template. The target generation model 11 and the template generation model 12 may be, for example, large-scale language models.

[0024] The target generative model 11 and the template generative model 12 may be realized by a single generative model. The target generative model 11 and the template generative model 12 may be the same generative model. The target generative model 11 and the template generative model 12 may share some parameters. Furthermore, for example, the target generative model 11 or the template generative model 12 may be realized by multiple generative models working together. The target generative model 11 or the template generative model 12 may be composed of multiple generative models according to the tasks to be executed.

[0025] The optimization device 20 is an information processing device such as a personal computer, a workstation, a server, etc. that optimizes templates. The templates optimized by the optimization device 20 are templates for generating input information for the target generation model 11.

[0026] The optimization device 20 may evaluate a template based on the target generation model 11. The optimization device 20 may optimize the template by rewriting the template based on the template evaluation result. The optimization device 20 may select a template to be rewritten (hereinafter also referred to as a "parent template") based on the template evaluation result. The optimization device 20 may generate a rewritten template (hereinafter also referred to as a "child template") by rewriting the parent template based on the template generation model 12.

[0027] The optimization device 20 may repeatedly evaluate the template based on the target generation model 11 and rewrite the template based on the template generation model 12. The optimization device 20 may continue the repeated process until a predetermined condition is satisfied. The optimization device 20 may output the template at the point in time when the predetermined condition is satisfied as an optimized template.

[0028] The execution device 30 is an information processing device such as a personal computer, workstation, or server that instructs the generation device 10-1 to execute a predetermined task in response to a request from the terminal device 40. The execution device 30 stores the optimized template output by the optimization device 20, and may generate input information for the target generation model 11 based on the input data and optimized template input by the terminal device 40. The execution device 30 may obtain the output result when the generated input information is input to the target generation model 11.

[0029] The input information may include text data, image data, or audio data. The text data may be, for example, a natural language sentence called a prompt. The image data may be, for example, a still image or a video. The image data may include, for example, an image of the user. The audio data may include, for example, audio uttered by the user. The input information may include a result of recognizing the image data or audio data.

[0030] The terminal device 40 is an information processing terminal such as a personal computer, smartphone, or tablet terminal operated by a user of the information processing system 1000. The terminal device 40 may transmit an execution request for a predetermined task to the execution device 30. The execution request may include input data entered by the user. The terminal device 40 may receive an output result of the target generation model 11 from the execution device 30 and present it to the user. As an example, the terminal device 40 may display the output result on a display device of the terminal device 40. As an example, the terminal device 40 may emit a voice synthesized from the output result from a speaker of the terminal device 40.

[0031] The overall configuration of the information processing system 1000 shown in FIG. 1 is an example, and various system configuration examples are possible depending on the application and purpose. The information processing system 1000 may be configured with one or more devices. The devices included in the information processing system 1000 may be a system configured with multiple devices. Each function included in the information processing system 1000 may be realized by any device that constitutes the system. Each component included in the information processing system 1000 may be included in any device that constitutes the system.

[0032] The target generation model 11 and the template generation model 12 may be built into a single generation device 10. The target generation model 11 or the template generation model 12 may be built into the optimization device 20 or the execution device 30. The target generation model 11 or the template generation model 12 may be distributed and held in an external information processing system consisting of multiple devices. In this case, the information processing system 1000 does not need to include the generation device 10.

[0033] The information processing system 1000 may include multiple units of one or more of the generation device 10, optimization device 20, execution device 30, and terminal device 40. The generation device 10, optimization device 20, or execution device 30 may be realized by multiple computers, or may be realized as a cloud computing service. Two or more of the generation device 10, optimization device 20, and execution device 30 may be realized by standalone computers. The information processing system 1000 may be realized by a single device that combines the functions of the generation device 10, optimization device 20, and execution device 30, and the terminal device 40. The division of devices such as the generation device 10, optimization device 20, execution device 30, and terminal device 40 shown in FIG. 1 is an example.

[0034] <Functional Configuration of Optimization Apparatus> The functional configuration of the optimization apparatus 20 will be described with reference to Fig. 2. Fig. 2 is a block diagram showing an example of the functional configuration of the optimization apparatus in the first embodiment.

[0035] 2, the optimization device 20 includes an evaluation data storage unit 201, an initialization unit 202, an evaluation unit 203, an evaluation history storage unit 204, a selection unit 205, a rewriting unit 206, and a determination unit 207. The optimization device 20 functions as the evaluation data storage unit 201, the initialization unit 202, the evaluation unit 203, the evaluation history storage unit 204, the selection unit 205, the rewriting unit 206, and the determination unit 207 by executing an optimization program installed in advance.

[0036] The evaluation data storage unit 201 pre-stores evaluation data used to evaluate templates. The evaluation data may be question-and-answer data, for example. The question-and-answer data may be data in which a question sentence, answer options, and a correct answer option are associated with each other. The question-and-answer data may be stored in a database in which items and values ​​are associated with each other. As a method for associating items and values, for example, the items and values ​​may be stored as a set, information that allows one piece of information to be obtained from the other piece of information may be stored, or one piece of information and identification information for the other piece of information may be stored as a set.

[0037] As another example, the evaluation data may be natural language inference data or calculation data. The natural language inference data may be data indicating a relationship between two or more sentences (e.g., an implication relationship). The calculation data may be data indicating a calculation formula and a correct answer (e.g., 1 + 2 = 3).

[0038] If the target generation model 11 is a generation model that involves voice recognition, the evaluation data may be data in which voice data indicating a question is associated with text data indicating a correct answer. If the target generation model 11 is a generation model that generates an image, the evaluation data may be data in which image data indicating the image is associated with an evaluation value obtained by a human evaluating the image.

[0039] For example, the evaluation data may satisfy the following conditions: The first condition is that the evaluation data is described in a manner that can be interpreted by the target generation model 11 and the template generation model 12. The second condition is that the evaluation data has content or quantity that can be estimated by the target generation model 11 and the template generation model 12.

[0040] The initialization unit 202 acquires one or more initial templates. The initial templates are templates in an initial state before optimization. The initialization unit 202 may accept input of an initial template designated by a user. The initialization unit 202 may generate the initial templates based on evaluation data read from the evaluation data storage unit 201.

[0041] When generating an initial template, the initialization unit 202 may, for example, generate the initial template using the template generation model 12. Specifically, first, the initialization unit 202 generates input information instructing template generation and transmits the input information to the generation device 10-2. The generation device 10-2 inputs the received input information into the template generation model 12, and then transmits the generated template to the optimization device 20. Then, the initialization unit 202 obtains the template received from the generation device 10-2 as the initial template.

[0042] The initialization unit 202 may generate the initial template using a generation model different from the template generation model 12. The initialization unit 202 may also generate the initial template using the target generation model 11.

[0043] The evaluation unit 203 evaluates one or more templates (hereinafter also referred to as "evaluation target templates") based on the target generation model 11. The evaluation target templates may include an initial template acquired by the initialization unit 202. The evaluation target templates may also include rewritten templates obtained by rewriting the initial template one or more times.

[0044] The evaluation unit 203 may evaluate the template to be evaluated using the evaluation data read out from the evaluation data storage unit 201. Specifically, first, the evaluation unit 203 generates input information for the target generation model 11 using the template to be evaluated for each piece of evaluation data read out from the evaluation data storage unit 201, and transmits the information to the generation device 10-1. The generation device 10-1 transmits the output result when the received input information is input into the target generation model 11 to the optimization device 20. Then, the evaluation unit 203 generates an evaluation result for the template to be evaluated based on the output result received from the generation device 10-1.

[0045] The evaluation result of the template may be an evaluation value that quantifies the evaluation level. The evaluation value may be a different index depending on the task. For example, when evaluation is performed using a question-answering task, the evaluation value may be the accuracy rate. Also, for example, when evaluation is performed using a translation task, the evaluation value may be the translation accuracy.

[0046] The accuracy rate of the question-answering task may be calculated, for example, as follows: The evaluation unit 203 generates multiple pieces of input information for executing the question-answering task for each template to be evaluated. The input information includes a question sentence and answer options of the question-answering data, and includes an instruction to select the correct answer option for the question sentence. The evaluation unit 203 determines whether the output result when each piece of input information is input to the target generation model 11 is correct, and calculates the accuracy rate for each template to be evaluated.

[0047] The evaluation history storage unit 204 stores history information of the evaluation results generated by the evaluation unit 203. The evaluation result history information may be information that accumulates the evaluation results generated by the evaluation unit 203. The number of pieces of history information stored in the evaluation history storage unit 204 is not limited. The evaluation result history information may include a predetermined number of evaluation results that were generated most recently, or may include multiple evaluation results that were generated within a predetermined period of time.

[0048] The selection unit 205 selects one or more parent templates to be rewritten from the evaluation target templates based on the evaluation results generated by the evaluation unit 203. The selection unit 205 may select evaluation target templates with high evaluations as parent templates. The selection unit 205 may select one or more parent templates based on history information stored in the evaluation history storage unit 204. The selection unit 205 may extract a predetermined number of evaluation target templates with high evaluation values ​​from the evaluation result history information.

[0049] The selection unit 205 may select one or more parent templates according to a predetermined optimization algorithm, which may include, for example, a genetic algorithm, Bayesian optimization, an evolutionary strategy, or reinforcement learning.

[0050] In the genetic algorithm, as one example, a parent template may be selected using tournament selection. When tournament selection is used, a predetermined number of evaluation target templates are randomly selected from a plurality of evaluation target templates, and the evaluation target template with the highest evaluation value is selected from among them. As another example, roulette wheel selection may be used.

[0051] The rewriting unit 206 obtains child templates by rewriting one or more parent templates selected by the selection unit 205, based on the template generation model 12. Specifically, the rewriting unit 206 first generates input information for the template generation model 12. The input information includes template information on one or more parent templates and information specifying the structure of the template. Multiple pieces of template information may be included. The input information may also include the evaluation result of the evaluation target template selected as the parent template.

[0052] The template information may include at least one of the template itself and input information generated using the template. The template structure may include, for example, a placeholder indicating a position where a predetermined item is to be embedded. Other examples of the template structure may include the type of markup language used to describe the template, the type of symbols used in the template, the order of content included in the template, or the repetition of items included in the template.

[0053] For example, the symbol may be a bullet point in a list, parentheses in a heading, etc. For example, the content may be information indicating instructions, context, conditions, etc., or information that subdivides these. The information specifying the repetition of an item may be whether or not a specific item is to be repeated, or the number of times it is to be repeated.

[0054] The rewriting unit 206 transmits the generated input information to the generation device 10-2. The generation device 10-2 inputs the received input information into the template generation model 12, and transmits the generated template to the optimization device 20. The rewriting unit 206 then acquires the template received from the generation device 10-2 as a child template.

[0055] The determining unit 207 determines whether a predetermined convergence condition is satisfied. The predetermined convergence condition is a condition that must be satisfied in order to terminate template optimization. For example, the predetermined convergence condition may be that the template has been rewritten a predetermined number of times, or that the difference in the evaluation results before and after the rewriting is within a predetermined threshold (in other words, that the evaluation value no longer improves significantly due to the rewriting), etc.

[0056] <Specific Examples of Input Information> The input information used by the optimization device 20 will be specifically described with reference to Figures 3 to 6. The input information used by the optimization device 20 may include, for example, a rewrite prompt and an initial generation prompt. The rewrite prompt is a prompt that instructs the rewriting unit 206 to rewrite a parent template. The initial generation prompt is a prompt that instructs the initialization unit 202 to generate an initial template.

[0057] <<Rewriting Prompt>> Figure 3 is a diagram showing a first example of a rewriting prompt template. As shown in Figure 3, the rewriting prompt template 500 may include a template rewriting instruction 501 ("Below are example prompt templates for evaluating large-scale language models. Please output one template that paraphrases these templates without changing the meaning").

[0058] The rewrite prompt template 500 may include placeholders 502 ({{{{question}}}}, {{{{option_0}}}}, {{{{option_1}}}}, {{{{option_2}}}}, {{{{option_3}}}}, {{{{option_4}}}}). Note that question represents a question, and option_0 to option_4 represent answer choices.

[0059] The rewrite prompt template 500 may include constraints 503 ("Never delete or rename symbols, make sure the output contains identical copies"; "Output one template, do not output anything other than the template").

[0060] The rewriting prompt template 500 may include information 504 specifying the type of markup language (Jinja2 format), information 505 specifying the order of content or repetition of items ("You can rearrange the sentence structure or the order of symbols as long as the meaning remains the same"), and information 506 specifying the type of symbols ("It's okay to change symbols like #").

[0061] The rewriting prompt template 500 may include placeholders 507 for template information related to the parent template ("Template example: {parent_0}", "Template example: {parent_1}"). The rewriting prompt template 500 shown in FIG. 3 includes two placeholders for template information, but the number of placeholders for template information may be one, three, or more. The placeholders for template information may be filled with prompt examples generated using the parent template.

[0062] 3, placeholders {parent_0} and {parent_1} indicating parent templates are written in placeholder 507 of the template information. In a rewrite prompt generated using template 500, the parent templates selected by selection unit 205 are embedded in each of placeholders {parent_0} and {parent_1}. Note that the parent template may be written in a markup language (jinja2 format) specified in the rewrite prompt, and may include placeholders 502 ({{{{question}}}}, {{{{option_0}}}}, {{{{option_1}}}}, {{{{option_2}}}}, {{{{option_3}}}}, {{{{option_4}}}}).

[0063] <<Initial Generation Prompt>> Fig. 4 is a diagram showing an example of an initial generation prompt template. As shown in Fig. 4, the initial generation prompt template 510 may include a template generation instruction 511 ("Please generate a prompt template for evaluating a large-scale language model based on the following example questions and answers").

[0064] The initial generated prompt template 510 may include placeholders 512 ({{{{question}}}}, {{{{option_0}}}}, {{{{option_1}}}}, {{{{option_2}}}}, {{{{option_3}}}}, {{{{option_4}}}}).

[0065] The initial generation prompt template 510 may include constraints 513 ("But please use the following placeholders," "Never delete or rename symbols, make sure to include exact duplicates in your output," "Output one template, do not output anything other than the template").

[0066] The initial generation prompt template 510 may include information 514 specifying the type of markup language (Jinja2 format), information 515 specifying the order of content or repetition of items ("You can rearrange the sentence structure or the order of symbols as long as the meaning remains the same"), and information 516 specifying the type of symbols ("It's okay to change symbols like #").

[0067] The initially generated prompt may include an example prompt. The initially generated prompt template 510 shown in FIG. 4 includes two example prompt placeholders 517 ("Question / Answer Example: {qa_0}" and "Question / Answer Example: {qa_1}"), but the number of example prompt placeholders may be one, three, or more.

[0068] In the template 510 of the initial generation prompt shown in FIG. 4, the placeholders {qa_0} and {qa_1} indicating example questions and answers are written in the prompt example placeholder 517. In the initial generation prompt generated using the template 510, the placeholders {qa_0} and {qa_1} are filled with example questions and answers including a question sentence and answer options.

[0069] Fig. 5 is a diagram showing an example of an initial-generation prompt. The initial-generation prompt 520 shown in Fig. 5 is an initial-generation prompt generated using the initial-generation prompt template 510 shown in Fig. 4. As shown in Fig. 5, in the initial-generation prompt 520, specific question-and-answer examples are embedded in the placeholders {qa_0} and {qa_1} that were used in the initial-generation prompt template 510.

[0070] Fig. 6 is a diagram showing an example of an initial template. The initial template 530 shown in Fig. 6 is an example of an initial template generated by inputting the initial generation prompt 520 shown in Fig. 5 into the template generation model 12. As shown in Fig. 6, the initial template 530 includes all of the placeholders 531 ({{{{question}}}}, {{{{option_0}}}}, {{{{option_1}}}}, {{{{option_2}}}}, {{{{option_3}}}}, {{{{option_4}}}}).

[0071] <Functional Configuration of Execution Device> The functional configuration of the execution device 30 will be described with reference to Fig. 7. Fig. 7 is a block diagram showing an example of the functional configuration of the execution device in the first embodiment.

[0072] 7, the execution device 30 includes a template storage unit 301, a request reception unit 302, an input generation unit 303, a result acquisition unit 304, and a result output unit 305. The execution device 30 functions as the template storage unit 301, the request reception unit 302, the input generation unit 303, the result acquisition unit 304, and the result output unit 305 by executing an execution program installed in advance.

[0073] Optimized templates are stored in the template storage unit 301. The optimized templates may be templates optimized by the optimization device 20.

[0074] The request receiving unit 302 receives an input of an execution request from the terminal device 40. The execution request may include input data input by the user to the terminal device 40. The input data may include data to be embedded in a placeholder included in the template.

[0075] The input generation unit 303 generates input information for the target generation model 11 based on the input data accepted by the request acceptance unit 302 and the optimized template read from the template storage unit 301. The input generation unit 303 may generate the input information by embedding the input data in a placeholder included in the optimized template.

[0076] The result acquisition unit 304 acquires the output result when the input information generated by the input generation unit 303 is input to the target generation model 11. Specifically, first, the result acquisition unit 304 transmits the input information generated by the input generation unit 303 to the generation device 10-1. The generation device 10-1 transmits the output result when the received input information is input to the target generation model 11 to the execution device 30. Then, the result acquisition unit 304 acquires the output result received from the generation device 10-1.

[0077] The result output unit 305 transmits the output results acquired by the result acquisition unit 304 to the terminal device 40. The result output unit 305 may transmit information obtained by processing the output results to the terminal device 40. Examples of processing may include extracting desired information from the output results, embedding predetermined information in the output results, etc.

[0078] <Flow of Optimization Process> The optimization process executed by the information processing system 1000 will be described with reference to Fig. 8 to Fig. 10. Fig. 8 is a flowchart showing an example of the optimization process in the first embodiment. The optimization process is a process for optimizing a template for generating input information.

[0079] In step S1, the initialization unit 202 of the optimization device 20 acquires one or more initial templates. Here, as an example, an example in which an initial template is generated using the template generation model 12 will be described.

[0080] First, the initialization unit 202 generates input information that instructs the generation of a template, and then transmits the generated input information to the generation device 10-2.

[0081] The generation device 10-2 receives input information from the optimization device 20. Next, the generation device 10-2 inputs the received input information to the template generation model 12. The template generation model 12 generates and outputs a template in accordance with the input information. Then, the generation device 10-2 transmits the output result of the template generation model 12 to the optimization device 20.

[0082] The initialization unit 202 receives the output result of the template generation model 12 from the generation device 10-2. Next, the initialization unit 202 obtains initial templates from the received output result. Then, the initialization unit 202 sends one or more initial templates to the evaluation unit 203.

[0083] In step S2, the evaluation unit 203 of the optimization device 20 receives one or more initial templates from the initialization unit 202. Next, the evaluation unit 203 evaluates the one or more initial templates as evaluation target templates based on the target generation model 11. Then, the evaluation unit 203 stores the evaluation results of the evaluation target templates in the evaluation history storage unit 204. In addition, the evaluation unit 203 sends the evaluation results of the evaluation target templates to the selection unit 205.

[0084] <Evaluation Process> The evaluation process (step S2 in FIG. 8) in this embodiment will be described in more detail with reference to FIG. 9. FIG. 9 is a flowchart showing an example of the evaluation process.

[0085] In step S2-1, the evaluation unit 203 of the optimization device 20 reads evaluation data from the evaluation data storage unit 201. The evaluation unit 203 may read all of the evaluation data stored in the evaluation data storage unit 201. The evaluation unit 203 may also read a predetermined number of evaluation data randomly selected from the evaluation data stored in the evaluation data storage unit 201.

[0086] In step S2-2, the evaluation unit 203 of the optimization device 20 generates input information for the target generation model 11 based on the evaluation data and the evaluation target template read out in step S2-1. As an example, the evaluation unit 203 may generate the input information by embedding the evaluation data in a placeholder included in the evaluation target template. The evaluation unit 203 may generate input information for each of the multiple evaluation data read out in step S2-1.

[0087] In step S2-3, the evaluation unit 203 of the optimization device 20 transmits the input information generated in step S2-2 to the generation device 10-1. The generation device 10-1 receives the input information from the optimization device 20. Next, the generation device 10-1 inputs the received input information to the target generation model 11. The target generation model 11 executes a predetermined task in accordance with the input information and outputs data generated by executing the task. Then, the generation device 10-1 transmits the output result of the target generation model 11 to the optimization device 20.

[0088] In step S2-4, the evaluation unit 203 of the optimization device 20 receives the output result of the target generation model 11 from the generation device 10-1. Next, the evaluation unit 203 generates an evaluation result of the template to be evaluated based on the received output result. Then, the evaluation unit 203 stores the evaluation result of the template to be evaluated in the evaluation history storage unit 204. In addition, the evaluation unit 203 sends the evaluation result of the template to be evaluated to the selection unit 205.

[0089] The evaluation unit 203 executes the processes from step S2-1 to step S2-4 for each of one or more templates to be evaluated. The evaluation unit 203 may repeatedly execute the processes from step S2-1 to step S2-4 for each template to be evaluated. The evaluation unit 203 may execute the processes from step S2-1 to step S2-4 in parallel for each template to be evaluated.

[0090] Returning to FIG. 8 , in step S3, the selection unit 205 of the optimization device 20 receives the evaluation result of the evaluation target template from the evaluation unit 203. The selection unit 205 may read out history information of the evaluation result from the evaluation history storage unit 204. Next, the selection unit 205 selects one or more parent templates based on the evaluation result of the evaluation target template. The selection unit 205 may also select one or more parent templates based on the history information of the evaluation result. Then, the selection unit 205 sends the selected parent templates to the rewriting unit 206.

[0091] In step S4, the rewriting unit 206 of the optimization device 20 receives one or more parent templates from the selection unit 205. Next, the rewriting unit 206 obtains child templates obtained by rewriting one or more parent templates based on the template generation model 12.

[0092] <<Rewrite Process>> The rewrite process (step S4 in FIG. 8) in this embodiment will be described in more detail with reference to Fig. 10. Fig. 10 is a flowchart showing an example of the rewrite process.

[0093] In step S4-1, the rewriting unit 206 of the optimization device 20 generates input information for the template generation model 12 based on one or more parent templates. As an example, the rewriting unit 206 may generate the input information by embedding template information related to one or more parent templates in placeholders of a rewriting prompt.

[0094] In step S4-2, the rewriting unit 206 of the optimization device 20 transmits the input information generated in step S4-1 to the generation device 10-2. The generation device 10-2 receives the input information from the optimization device 20. Next, the generation device 10-2 inputs the received input information to the template generation model 12. The template generation model 12 generates and outputs a template according to the input information. Then, the generation device 10-2 transmits the output result of the template generation model 12 to the optimization device 20.

[0095] In step S4-3, the rewriting unit 206 of the optimization device 20 receives the output result of the template generation model 12 from the generation device 10-2. Next, the rewriting unit 206 obtains a child template from the received output result.

[0096] Returning to FIG. 8 , in step S5, the determination unit 207 of the optimization device 20 determines whether or not a predetermined convergence condition has been satisfied. If the predetermined convergence condition has been satisfied (YES), the determination unit 207 outputs the child template acquired in the immediately preceding step S4 as an optimized template, and ends the optimization process. On the other hand, if the predetermined convergence condition has not been satisfied (NO), the determination unit 207 returns the process to step S2.

[0097] In step S2 from the second time onward, the evaluation unit 203 performs the evaluation process using the child template acquired in step S4 immediately before as the evaluation target template. Thereafter, the optimization device 20 repeatedly performs the processes from step S2 to step S4 until a predetermined convergence condition is satisfied in step S5.

[0098] 8 , after step S4 in which the rewriting unit 206 rewrites one or more parent templates, step S5 in which the determination unit 207 determines whether a predetermined convergence condition is satisfied is executed, but step S5 may be executed at a different timing. For example, step S5 may be executed after step S2 in which the evaluation unit 203 evaluates the evaluation target template. Step S5 may be executed at any timing depending on the content of the convergence condition.

[0099] <Flow of Execution Processing> The execution processing executed by the information processing system 1000 will be described with reference to Fig. 11. Fig. 11 is a flowchart showing an example of the execution processing in the first embodiment. The execution processing is processing for executing a predetermined task based on an optimized template.

[0100] In step S11, the user of the information processing system 1000 performs an operation to execute a predetermined task on the terminal device 40. The operation to execute the predetermined task may be, for example, an operation to input data to be used in the predetermined task onto a screen displayed on a display device of the terminal device 40, or an operation to input data to be used in the predetermined task by voice into a microphone of the terminal device 40.

[0101] The terminal device 40 accepts an operation for executing a predetermined task. Next, the terminal device 40 transmits an execution request for the predetermined task to the execution device 30 in response to the accepted operation. The execution request may include data input by the user. The data input by the user may include data to be embedded in a placeholder of the template.

[0102] The request receiving unit 302 of the optimization device 20 receives an execution request from the terminal device 40. Next, the request receiving unit 302 sends the input data included in the execution request to the input generating unit 303.

[0103] In step S12, the input generation unit 303 of the execution device 30 receives the input data from the request reception unit 302. Next, the input generation unit 303 reads out the optimized template from the template storage unit 301.

[0104] Next, the input generation unit 303 generates input information for the target generation model 11 based on the input data and the optimized template. As an example, the input generation unit 303 may generate the input information by embedding the input data in a placeholder included in the optimized template. Then, the input generation unit 303 sends the generated input information to the result acquisition unit 304.

[0105] In step S13, the result acquisition unit 304 of the execution device 30 receives the input information from the input generation unit 303. Next, the result acquisition unit 304 transmits the input information to the generation device 10-1.

[0106] The generation device 10-1 receives input information from the execution device 30. Next, the generation device 10-1 inputs the received input information to the target generation model 11. The target generation model 11 executes a predetermined task in accordance with the input information and outputs data generated by executing the task. The generation device 10-1 then transmits the output result of the target generation model 11 to the execution device 30.

[0107] The result acquisition unit 304 receives the output result of the target generation model 11 from the generation device 10-1. Next, the result acquisition unit 304 sends the received output result to the result output unit 305.

[0108] In step S14, the result output unit 305 of the execution device 30 receives the output result of the target generation model 11 from the result acquisition unit 304. Next, the result output unit 305 transmits the received output result to the terminal device 40. The result output unit 305 may also transmit data obtained by processing the received output result to the terminal device 40.

[0109] The terminal device 40 receives the output result of the target generation model 11 from the execution device 30. Next, the terminal device 40 presents the received output result to the user. As an example, the terminal device 40 may display a screen including the output result on a display device. As an example, the terminal device 40 may emit a voice synthesized from the output result from a speaker.

[0110] [Variation 1] One conventional prompt engineering technique involves including a numerical vector called a soft prompt in the input information to a generative model. A soft prompt is a vector with a structure similar to a word vector, but is not based on a specific word. In conventional techniques, a soft prompt is included in the input information to a generative model, and the value of the soft prompt is optimized to obtain good output results from the generative model.

[0111] In conventional methods, the position of a soft prompt in input information is fixed. For example, the soft prompt is embedded at the beginning or end of the input information. By optimizing the position of the soft prompt in addition to the value of the soft prompt, better output results can be expected. In this modified example, a template including a placeholder indicating the position where the soft prompt is embedded is optimized.

[0112] 12 is a diagram illustrating a second example of a template for a rewriting prompt. As illustrated in FIG. 12, the template for a rewriting prompt 540 may include a placeholder 541 ({{{soft_prompt}}}) indicating the position of a soft prompt. The template for a rewriting prompt 540 may be similar to the template for a rewriting prompt 500 in the first embodiment, except that the template for a rewriting prompt 540 includes the placeholder 541 indicating the position of a soft prompt.

[0113] <Functional Configuration of Optimization Apparatus> The functional configuration of the optimization apparatus in this modification will be described with reference to Fig. 13. Fig. 13 is a block diagram showing an example of the functional configuration of the optimization apparatus in Modification 1.

[0114] 13, the optimization device 20 includes an evaluation data storage unit 201, an initialization unit 202, an evaluation unit 203, an evaluation history storage unit 204, a selection unit 205, a rewriting unit 206, a determination unit 207, and a soft prompt optimization unit 208. That is, the optimization device 20 in the first modification example differs from the first embodiment in that it further includes a soft prompt optimization unit 208.

[0115] The soft prompt optimization unit 208 optimizes the soft prompt. First, the soft prompt optimization unit 208 generates an initial soft prompt. The initial soft prompt may be, for example, a random numeric vector or a predetermined numeric vector.

[0116] Next, the soft prompt optimization unit 208 evaluates the soft prompt. The evaluation method for the soft prompt may be the same as the evaluation method for the template to be evaluated. That is, the soft prompt optimization unit 208 generates input information in which the soft prompt and evaluation data are embedded in the placeholders of the template to be evaluated, and transmits the input information to the generation device 10-1. Then, the soft prompt optimization unit 208 generates an evaluation result for the soft prompt based on the output result received from the generation device 10-1.

[0117] Next, the soft prompt optimization unit 208 optimizes the soft prompts based on the evaluation results of the soft prompts. The optimization algorithm may be, for example, Bayesian optimization or evolutionary computing. The soft prompt optimization unit 208 may search for optimal soft prompts based on the evaluated soft prompts and their evaluation results, taking into account a trade-off between exploration and exploitation. When the soft prompts converge, the soft prompt optimization unit 208 sends the optimized soft prompts to the evaluation unit 203.

[0118] In this modification, the evaluation unit 203 evaluates the template to be evaluated using the soft prompt optimized by the soft prompt optimization unit 208 and the evaluation data. The selection unit 205 selects one or more parent templates to be rewritten from the template to be evaluated, based on the evaluation results generated by the evaluation unit 203. The rewriting unit 206 generates child templates by rewriting the one or more parent templates selected by the selection unit 205, based on the template generation model 12. The evaluation unit 203, selection unit 205, and rewriting unit 206 repeat the process until the determination unit 207 determines that a predetermined convergence condition is satisfied.

[0119] [Variation 2] In the first embodiment, a configuration has been described in which an evaluation value indicating the evaluation result of the evaluation target template is calculated based on the output result of the target generation model 11. In Variation 2, a configuration will be described in which the output result of the target generation model 11 is presented to a user, and the evaluation result of the evaluation target template is obtained based on feedback information from the user.

[0120] The user who receives the feedback information may be the same person as the user who performs the predetermined task using the optimized template, or may be a different person. The user who receives the feedback information may be multiple people. In other words, the user who receives the feedback information may or may not include the user who operates the execution device 30.

[0121] This modification may be used when it is difficult to quantify the evaluation of a template based on the output result. In the first embodiment, templates were evaluated using a question-answering task. In this case, the accuracy rate of answers to questions can be calculated as the evaluation value. On the other hand, there are tasks for which it is difficult to quantify the evaluation of the output result. For example, in tasks such as document summarization, translation, image generation, and musical sound generation, it may not be appropriate to evaluate the output result using a uniform standard. In these types of tasks, by collecting feedback information indicating the user's evaluation, the template can be optimized to obtain an output result that the user considers satisfactory.

[0122] The feedback information may be collected using, for example, a black-box optimization tool. The black-box optimization tool may be, for example, Optuna (registered trademark). The black-box optimization tool may be pre-installed in the terminal device 40. The black-box optimization tool may be installed in the optimization device 20, and the terminal device 40 may be configured to acquire feedback information from the user.

[0123] 14 is a diagram showing an example of a prompt template in Modification Example 2. As shown in Fig. 14, a prompt template 550 in Modification Example 2 may include a document summary instruction 551 ("Please summarize"). The prompt template 550 may include a single placeholder 552 ({{summary document}}) into which the document to be summarized is embedded.

[0124] In this embodiment, when the evaluation unit 203 receives the output result of the target generation model 11 from the generation device 10-1, it presents the received output result to the user. As an example, the evaluation unit 203 may display a feedback screen for receiving feedback information from the user on the display device of the terminal device 40. Specifically, the evaluation unit 203 may transmit screen data for displaying the feedback screen to the terminal device 40, and the terminal device 40 that has received the screen data may display the feedback screen on the display device based on the screen data.

[0125] The feedback screen may include an area for displaying the output result of the target generation model 11. The feedback screen may include a feedback information input field for inputting feedback information from the user. The feedback information may include, for example, an evaluation value indicating the evaluation result by the user. The feedback information may include, for example, free-form text.

[0126] 15 is a diagram showing an example of a feedback information input field 600. As shown in FIG. 15, the feedback information input field 600 may include an evaluation value input field 601, a text input field 602, a submit button 603, and a cancel button 604.

[0127] The evaluation value input field 601 may have evaluation options. Each option may be associated with a predetermined evaluation value. The evaluation options may be, for example, "Good," "So-so," or "Bad." The evaluation options may be, for example, numbers, letters, or symbols indicating the evaluation value. The evaluation options may be, for example, on a scale of 3 to 5.

[0128] For example, the evaluation value input field 601 may accept input of an evaluation value by operating a radio button. For example, the evaluation value input field 601 may accept input of an evaluation value by operating a slider. The evaluation value input field 601 may accept input of a continuous value or input of a discrete value.

[0129] FIG. 16 is a diagram showing an example of an evaluation value input field. As shown in FIG. 16, the evaluation value input field 601 may be any of an input form 611 for selecting an evaluation value by operating a radio button, an input form 612 for adjusting an evaluation value by operating a slider, or an input form 613 for performing multi-objective optimization. The multi-objective optimization may be performed by combining a quantitative evaluation value (such as AUC (Area Under the Curve)) and a subjective evaluation value (human). Note that a user (e.g., a programmer or a general user) may select which of the input forms 611 to 613 shown in FIG. 16 to use.

[0130] The user may refer to the output results displayed on the feedback screen and input feedback information for the output results in the feedback information input field 600. The feedback information may be acquired by recognizing the user's voice. The feedback information may be acquired by having the user select one or more output results that the user feels are good or one or more output results that the user feels are not good from multiple output results generated using multiple evaluation target templates.

[0131] The user may input the evaluation value through a user interface for comparative evaluation that simultaneously displays multiple output results and allows the user to select the output result that is most highly rated by the user. In this user interface, the evaluation value of a selected output result may be higher than the evaluation value of an output result that is not selected. In this user interface, a value indicating how superior an output result is to other output results may be input using a slider or the like. In addition, in this user interface, the output results of the target generation model 11 may be processed by a program in order to perform comparative evaluation.

[0132] The user may input an evaluation value through a user interface for ranking evaluation that simultaneously displays multiple output results and allows the user to rank them. The user may also input an evaluation value by changing the evaluation value calculated by the terminal device 40 or the optimization device 20. The user may also input an evaluation value by combining the evaluation value input by the user with the evaluation value calculated by the terminal device 40 or the optimization device 20.

[0133] After the user inputs the feedback information, when the user presses the submit button 603, the terminal device 40 acquires the feedback information input in the feedback information input field 600 and transmits it to the optimization device 20. In the optimization device 20, the evaluation unit 203 receives the feedback information from the terminal device 40. The evaluation unit 203 acquires an evaluation result from the received feedback information. Then, the evaluation unit 203 sends the evaluation result of the template to be evaluated to the selection unit 205.

[0134] When the user presses the stop button 604 without inputting feedback information, the terminal device 40 transmits a signal indicating that the evaluation is to be stopped to the optimization device 20. In the optimization device 20, the evaluation unit 203 receives the signal indicating that the evaluation is to be stopped from the terminal device 40. The evaluation unit 203 may exclude the evaluation target template for which the signal indicating that the evaluation is to be stopped has been received from the evaluation targets.

[0135] Thereafter, the optimization device 20 optimizes the template based on the feedback information. Specifically, the optimization device 20 selects a parent template to be embedded in the rewrite prompt using an optimization algorithm that uses the feedback information as the evaluation result of the evaluation target template. The optimization algorithm based on the feedback information may be, for example, a black-box optimization method or a gray-box optimization method. The feedback information may be obtained from the user or may be automatically generated by the terminal device 40. Black-box optimization and gray-box optimization are examples of optimization algorithms based on feedback information.

[0136] Examples of black-box optimization include multi-objective optimization, Bayesian optimization, and evolutionary computing. Examples of gray-box optimization include multi-fidelity optimization. Specifically, black-box optimization may use methods such as TPE (Tree-Structured Parzen Estimator), GP-BO (Gaussian Process - Bayesian Optimization), CMA-ES, and NSGA-II (Non-Dominated Sorting Genetic Algorithm-II).

[0137] [Modification 3] The optimization device 20 may optimize parameters of a generative model for generating templates. In this modification, as an example, a configuration will be described in which the optimization device 20 optimizes parameters of the template generation model 12 by reinforcement learning.

[0138] In this modification, the optimization device 20 generates input information for the template generation model 12. The input information for the template generation model 12 may be a prompt instructing template generation. The input information for the template generation model 12 may or may not include a template selected according to an optimization algorithm.

[0139] As an example, the input information to the template generation model 12 may be a rewrite prompt generated using the rewrite prompt template 500 shown in FIG. 3, an initial generation prompt generated using the initial generation prompt template 510 shown in FIG. 4, or an initial generation prompt 520 shown in FIG. 5.

[0140] The optimization device 20 transmits input information for the template generation model 12 to the generation device 10-2. The generation device 10-2 inputs the received input information into the template generation model 12, and transmits the generated template to the optimization device 20.

[0141] The optimization device 20 evaluates the template received from the generation device 10-2. For example, the optimization device 20 may evaluate the template based on the target generation model 11. For example, the optimization device 20 may evaluate the template using evaluation data read from the evaluation data storage unit 201. Then, the optimization device 20 performs reinforcement learning on the parameters of the template generation model 12, using input information to the target generation model 11 as an action and an evaluation value indicating the evaluation result of the template as a reward.

[0142] Second Embodiment In the first embodiment, a configuration was described in which a template was optimized so that an output result with a high evaluation value could be obtained from a generative model, and a predetermined task was executed based on the optimized template. In the second embodiment, a configuration was described in which a template was optimized so that an output result with a low evaluation value could be obtained from a generative model, and additional training of a generative model was performed based on the optimized template.

[0143] The information processing system according to the second embodiment will be described below, focusing on the differences from the first embodiment.

[0144] <Functional Configuration of Optimization Apparatus> The functional configuration of the optimization apparatus in this embodiment will be described with reference to Fig. 17. Fig. 17 is a block diagram showing an example of the functional configuration of the optimization apparatus in the second embodiment.

[0145] 17 , the optimization device 20 includes an evaluation data storage unit 201, an initialization unit 202, an evaluation unit 203, an evaluation history storage unit 204, a selection unit 205, a rewriting unit 206, a determination unit 207, a verification unit 209, a data generation unit 210, and an additional learning unit 211. That is, the optimization device 20 in the second embodiment differs from the first embodiment in that it further includes a verification unit 209, a data generation unit 210, and an additional learning unit 211.

[0146] In this embodiment, the selection unit 205 selects an evaluation target template with a low evaluation as a parent template. As an example, when the evaluation data is question-answering data, the selection unit 205 may select one or more evaluation target templates in descending order of the accuracy rate in the question-answering task.

[0147] The verification unit 209 verifies whether the template generated by the template generation model 12 satisfies predetermined requirements. The template to be verified may include the initial template acquired by the initialization unit 202 or the child template generated by the rewriting unit 206.

[0148] The predetermined requirements are requirements that the template must meet. For example, the predetermined requirements may include that the template contains all necessary placeholders, contains at least content that indicates instructions, etc.

[0149] The data generation unit 210 generates a dataset based on the optimized template. The dataset may include input information to the target generation model 11 and correct values ​​of the output results. The data generation unit 210 may generate the dataset based on evaluation data read from the evaluation data storage unit 201. As an example, the data generation unit 210 may use the optimized template to generate input information including a question sentence and answer options included in the question-answering data, and generate a dataset including the input information and correct answer options included in the question-answering data.

[0150] The additional learning unit 211 additionally learns the target generation model 11 based on the dataset generated by the data generation unit 210. The additional learning method may be fine tuning, for example. Another example of the additional learning method may be low-rank adaptation (LoRA) or quantized low-ranking adaptation (QLoRA).

[0151] <Flow of Additional Learning Process> The additional learning process executed by the information processing system 1000 in the second embodiment will be described with reference to Fig. 18. Fig. 18 is a flowchart showing an example of the additional learning process.

[0152] The processes from step S21 to step S24 are the same as the processes from step S1 to step S4 in the first embodiment (see FIG. 8 ). However, in this embodiment, in step S23, the selection unit 205 selects an evaluation target template with a low evaluation as a parent template.

[0153] In step S25, the verification unit 209 of the optimization device 20 receives the child template from the rewriting unit 206. Next, the verification unit 209 verifies whether the received child template satisfies predetermined requirements.

[0154] If the child template satisfies the predetermined requirements (YES), the verification unit 209 proceeds to step S26. On the other hand, if the child template does not satisfy the predetermined requirements (NO), the verification unit 209 discards the child template and returns to step S24.

[0155] In step S26, the determination unit 207 of the optimization device 20 determines whether a predetermined convergence condition is satisfied. In this embodiment, the predetermined convergence condition may be, for example, that the evaluation value acquired in step S22 is equal to or less than a predetermined threshold. For example, the threshold may be determined based on the lowest evaluation value among the evaluation values ​​of existing templates.

[0156] If the predetermined convergence condition is satisfied (YES), determination unit 207 sends the child template acquired in the immediately preceding step S24 to data generation unit 210 as an optimized template, and proceeds to step S27. On the other hand, if the predetermined convergence condition is not satisfied (NO), determination unit 207 returns the process to step S22.

[0157] In step S27, the data generation unit 210 of the optimization device 20 receives the optimized template from the determination unit 207. Next, the data generation unit 210 reads the evaluation data from the evaluation data storage unit 201. Subsequently, the data generation unit 210 generates a dataset based on the evaluation data and the optimized template. As an example, the data generation unit 210 may generate the dataset by embedding the evaluation data in a placeholder included in the optimized template. Then, the data generation unit 210 sends the generated dataset to the additional learning unit 211.

[0158] In step S28, the additional learning unit 211 of the optimization device 20 receives the dataset from the data generation unit 210. Next, the additional learning unit 211 additionally learns the target generation model 11 based on the received dataset. As an example, the additional learning unit 211 may fine-tune the target generation model 11 based on the received dataset.

[0159] The additional learning process may be executed repeatedly. For example, the target generation model 11 after the additional learning process is completed may be used to evaluate the evaluation target template, and if the evaluation result does not improve, the additional learning process may be executed again.

[0160] [Application Examples] In each embodiment of the present disclosure, an example of using a large-scale language model that generates text data has been described as an example of a generative model, but the applications of the generative model are not limited to this. Examples of applications of the generative model include image generation, code review, material generation, finance, etc.

[0161] Image generation may be the generation of characters such as anime characters. Material generation may be the generation of materials from raw materials. Code review may select items such as tests, data structures, and documentation through black-box optimization. Variations of items may include items from qualitative perspectives such as consistency, maintainability, and errors. Furthermore, variations of items may include items related to context (code differences). Finance may be the creation of individual stock investment strategies.

[0162] <Summary> As is clear from the above description, the optimization device 20 according to an embodiment of the present disclosure obtains evaluation results obtained by evaluating one or more evaluation target templates using the target generation model 11, generates input information for the template generation model 12 based on the evaluation results, and obtains a generated template by inputting the input information to the template generation model 12. The input information includes at least information about one or more templates selected based on the evaluation results and information specifying the structure of the template.

[0163] The optimization device 20 may repeatedly acquire an evaluation result using the acquired template as one or more evaluation target templates, generate input information based on the evaluation result, and acquire a template until a predetermined condition is satisfied.

[0164] The information specifying the template structure may include a placeholder indicating a position where a predetermined item is to be embedded. The placeholder may include a placeholder indicating a position where a predetermined numeric vector is to be embedded. The information specifying the template structure may include at least information specifying the type of markup language, the type of symbol, the order of content, or the repetition of items.

[0165] The optimization device 20 may generate input information based on a template selected from one or more evaluation target templates by an optimization algorithm using the evaluation results. The optimization device 20 may select one or more templates having a high evaluation value indicating the evaluation result from the one or more evaluation target templates. The optimization device 20 may select one or more templates having a high evaluation value indicating the evaluation result from a plurality of templates arbitrarily selected from the one or more evaluation target templates.

[0166] The optimization device 20 may generate template information based on a trained model that has learned the relationship between input information generated using the evaluation target template and the evaluation result of evaluating the evaluation target template. The trained model may perform reinforcement learning using the input information as an action and the evaluation result as a reward.

[0167] The optimization device 20 may generate input information for the target generation model 11 using one or more evaluation target templates, obtain output results generated by inputting the input information into the target generation model 11, and obtain evaluation results based on the output results. The input information for the target generation model 11 may be input information for executing a task of answering a question, and the evaluation results may be information indicating the accuracy rate of answers included in the output results. The optimization device 20 may present the output results to a user and obtain evaluation results based on feedback information obtained from the user.

[0168] The optimization device 20 may generate the evaluation target template based on a generation model different from the target generation model 11 and the template generation model 12 .

[0169] The optimization device 20 may generate template information based on one or more templates having low evaluation values ​​indicating evaluation results among the one or more evaluation target templates. The optimization device 20 may use the acquired templates to generate a dataset including input information to the target generation model 11, and the dataset may be a dataset used for additional learning of the target generation model 11.

[0170] The target generation model 11 and the template generation model 12 may be the same generation model.

[0171] As a result, according to one embodiment of the present disclosure, a technology for optimizing templates for generating input information for a generative model can be provided. In one aspect, according to one embodiment, a good output result can be obtained from a generative model without changing the semantic content of the input information. Also, according to one aspect, according to one embodiment, a generative model can be additionally trained so as to obtain a good output result from input information that is found not to obtain a good output result from the generative model.

[0172] [Hardware Configuration of Information Processing Device] Some or all of the devices (generation device 10, optimization device 20, execution device 30, and terminal device 40) in the above-described embodiments may be configured with hardware, or may be configured with software (program) information processing executed by a CPU (Central Processing Unit), GPU (Graphics Processing Unit), or the like. In the case of software information processing, software that realizes at least some of the functions of each device in the above-described embodiments may be stored on a non-transitory storage medium (non-transitory computer-readable medium) such as a CD-ROM (Compact Disc-Read Only Memory) or USB (Universal Serial Bus) memory, and the software information processing may be executed by loading the software into a computer. The software may also be downloaded via a communication network. Furthermore, all or part of the software processing may be implemented in a circuit such as an ASIC (Application Specific Integrated Circuit) or FPGA (Field Programmable Gate Array), thereby executing the software information processing by hardware.

[0173] The storage medium that stores the software may be a removable medium such as an optical disk, or a fixed medium such as a hard disk, memory, etc. The storage medium may be provided inside the computer (such as a main storage device or auxiliary storage device) or outside the computer.

[0174] 19 is a block diagram showing an example of the hardware configuration of each device (the generation device 10, the optimization device 20, the execution device 30, and the terminal device 40) in the above-described embodiment. Each device may be realized as a computer 7 including, for example, a processor 71, a main storage device 72 (memory), an auxiliary storage device 73 (memory), a network interface 74, and a device interface 75, which are connected via a bus 76.

[0175] Although the computer 7 in FIG. 19 includes one of each component, it may also include multiple of the same component. Also, while FIG. 19 shows one computer 7, the software may be installed on multiple computers, with each of the multiple computers executing the same or different parts of the software. In this case, a distributed computing configuration may be used in which each computer communicates via a network interface 74 or the like to execute processing. That is, each device in the above-described embodiment (the generation device 10, the optimization device 20, the execution device 30, and the terminal device 40) may be configured as a system in which one or more computers execute instructions stored in one or more storage devices to realize functions. Furthermore, information transmitted from a terminal may be processed by one or more computers on a cloud, and the processing results may be transmitted to the terminal.

[0176] The various calculations of each device (the generation device 10, the optimization device 20, the execution device 30, and the terminal device 40) in the above-described embodiments may be executed in parallel using one or more processors, or using multiple computers via a network. Furthermore, the various calculations may be distributed to multiple processing cores within a processor and executed in parallel. Furthermore, some or all of the processes, means, etc. disclosed herein may be realized by at least one of a processor and a storage device provided on a cloud that can communicate with the computer 7 via a network. Thus, each device in the above-described embodiments may be implemented in the form of parallel computing using one or more computers.

[0177] The processor 71 may be an electronic circuit (processing circuit, processing circuitry, CPU, GPU, FPGA, ASIC, etc.) that at least controls or performs calculations on a computer. The processor 71 may be a general-purpose processor, a dedicated processing circuit designed to perform a specific calculation, or a semiconductor device that includes both a general-purpose processor and a dedicated processing circuit. The processor 71 may also include an optical circuit or a calculation function based on quantum computing.

[0178] The processor 71 may perform arithmetic processing based on data or software input from each device or the like configured internally of the computer 7, and may output arithmetic results or control signals to each device or the like. The processor 71 may control each component constituting the computer 7 by executing the OS (Operating System) of the computer 7, applications, etc.

[0179] Each device in the above-described embodiment (the generation device 10, the optimization device 20, the execution device 30, and the terminal device 40) may be realized by one or more processors 71. Here, the processor 71 may refer to one or more electronic circuits arranged on one chip, or may refer to one or more electronic circuits arranged on two or more chips or two or more devices. When multiple electronic circuits are used, the electronic circuits may communicate with each other via wire or wirelessly.

[0180] The main memory device 72 may store instructions executed by the processor 71, various data, etc., and information stored in the main memory device 72 may be read by the processor 71. The auxiliary memory device 73 is a memory device other than the main memory device 72. Note that these memory devices refer to any electronic component capable of storing electronic information and may be semiconductor memory. The semiconductor memory may be either volatile memory or non-volatile memory. The memory devices for saving various data, etc. in each device (the generation device 10, the optimization device 20, the execution device 30, and the terminal device 40) in the above-described embodiments may be realized by the main memory device 72 or the auxiliary memory device 73, or may be realized by an internal memory built into the processor 71. For example, each memory unit in the above-described embodiments may be realized by the main memory device 72 or the auxiliary memory device 73.

[0181] When each device (the generation device 10, the optimization device 20, the execution device 30, and the terminal device 40) in the above-described embodiment is configured with at least one storage device (memory) and at least one processor connected (coupled) to this at least one storage device, at least one processor may be connected to one storage device. Also, at least one storage device may be connected to one processor. Also, a configuration in which at least one processor among multiple processors is connected to at least one storage device among multiple storage devices may be included. Also, this configuration may be realized by storage devices and processors included in multiple computers. Furthermore, a configuration in which a storage device is integrated with a processor (for example, a cache memory including an L1 cache and an L2 cache) may be included.

[0182] The network interface 74 is an interface for connecting to the communication network 8 wirelessly or via a wire. The network interface 74 may be an appropriate interface, such as one that conforms to an existing communication standard. The network interface 74 may exchange information with an external device 9A connected via the communication network 8. The communication network 8 may be any one of a wide area network (WAN), a local area network (LAN), a personal area network (PAN), etc., or a combination thereof, as long as information is exchanged between the computer 7 and the external device 9A. An example of a WAN is the Internet, an example of a LAN is IEEE 802.11 or Ethernet (registered trademark), and an example of a PAN is Bluetooth (registered trademark) or NFC (Near Field Communication), etc.

[0183] The device interface 75 is an interface such as a USB that directly connects to the external device 9B.

[0184] The external device 9A is a device connected to the computer 7 via a network, and the external device 9B is a device connected directly to the computer 7.

[0185] The external device 9A or the external device 9B may be, for example, an input device. The input device may be, for example, a camera, a microphone, a motion capture device, various sensors, a keyboard, a mouse, a touch panel, or the like, and provides acquired information to the computer 7. Alternatively, the external device 9A or the external device 9B may be a device equipped with an input unit, a memory, and a processor, such as a personal computer, a tablet terminal, or a smartphone.

[0186] Furthermore, the external device 9A or the external device 9B may be, for example, an output device. The output device may be, for example, a display device such as an LCD (Liquid Crystal Display) or an organic EL (Electro Luminescence) panel, or a speaker that outputs sound or the like. Alternatively, the external device 9A or the external device 9B may be a device including an output unit, a memory, and a processor, such as a personal computer, a tablet terminal, or a smartphone.

[0187] The external device 9A or the external device 9B may be a storage device (memory). For example, the external device 9A may be a network storage or the like, and the external device 9B may be a storage device such as an HDD.

[0188] Furthermore, the external device 9A or the external device 9B may be a device having some of the functions of the components of each device (the generation device 10, the optimization device 20, the execution device 30, and the terminal device 40) in the above-described embodiments. In other words, the computer 7 may transmit some or all of the processing results to the external device 9A or the external device 9B, or may receive some or all of the processing results from the external device 9A or the external device 9B.

[0189] In this specification (including the claims), when the expression "at least one of a, b, and c" or "at least one of a, b, or c" (including similar expressions) is used, it includes any of a, b, c, ab, ac, bc, or abc. It may also include multiple instances of any element, such as aa, abb, aabbcc, etc. Furthermore, it also includes the addition of elements other than the enumerated elements (a, b, and c), such as having d, as in abcd.

[0190] In this specification (including claims), when expressions such as "using / using data as input / based on / according to / in response to data" (including similar expressions) are used, unless otherwise specified, this includes cases where the data itself is used, or where data that has been processed in some way (e.g., data with noise added, normalized data, features extracted from data, intermediate representations of data, etc.) is used. Furthermore, when a statement is made that a result is obtained "using data as input / based on / according to / in response to data" (including similar expressions), this includes cases where the result is obtained based solely on the data, or where the result is influenced by other data, factors, conditions, and / or states other than the data. Furthermore, when a statement is made that "data is output" (including similar expressions), this includes cases where the data itself is used as output, or where data that has been processed in some way (e.g., data with noise added, normalized data, features extracted from data, intermediate representations of various data, etc.) is used as output, unless otherwise specified.

[0191] When the terms "connected" and "coupled" are used in this specification (including the claims), they are intended as open-ended terms that include any of direct connection / coupling, indirect connection / coupling, electrically connection / coupling, communicatively connection / coupling, functionally connection / coupling, and physically connection / coupling. These terms should be interpreted appropriately according to the context in which they are used, but any connection / coupling form that is not intentionally or naturally excluded should be interpreted as being included in these terms without limitation.

[0192] In this specification (including the claims), the expression "A configured to B" may include the physical structure of element A having a configuration capable of performing operation B, and the permanent or temporary setting / configuration of element A being configured / set to actually perform operation B. For example, if element A is a general-purpose processor, it is sufficient that the processor has a hardware configuration capable of performing operation B, and is configured to actually perform operation B by setting a permanent or temporary program (instruction). Also, if element A is a dedicated processor, dedicated arithmetic circuit, etc., it is sufficient that the circuit structure of the processor is implemented to actually perform operation B, regardless of whether control instructions and data are actually attached.

[0193] Whenever words implying containing or possessing (e.g., "comprising / including," "having," etc.) are used in this specification (including the claims), they are intended to be open-ended terms that include containing or possessing things other than the object designated by the object of the term. When the object of such words implying containing or possessing does not specify a quantity or suggests a singular number (e.g., expressions using the articles "a" or "an"), the expression should be construed as not being limited to a specific number.

[0194] In this specification (including the claims), although expressions such as "one or more" and "at least one" are used in some places and expressions that do not specify a quantity or that imply a singular number (expressions using the articles "a" or "an") are used in other places, the latter expressions are not intended to mean "one." In general, expressions that do not specify a quantity or that imply a singular number (expressions using the articles "a" or "an") should be interpreted as not necessarily being limited to a specific number.

[0195] In this specification, when a particular advantage / result is described as being obtained with respect to a particular configuration of an embodiment, it should be understood that the same advantage / result can also be obtained with one or more other embodiments having the same configuration, unless otherwise stated. However, it should be understood that the presence or absence of the effect generally depends on various factors, conditions, and / or situations, and that the effect is not necessarily obtained with the configuration. The effect is merely obtained by the configuration described in the embodiment when various factors, conditions, and / or situations are satisfied, and the effect does not necessarily occur in a claimed invention that defines the same configuration or a similar configuration.

[0196] In this specification (including claims), when multiple pieces of hardware perform a predetermined process, the pieces of hardware may cooperate to perform the predetermined process, or some of the hardware may perform all of the predetermined process. Furthermore, some of the hardware may perform part of the predetermined process, and other hardware may perform the rest of the predetermined process. In this specification (including claims), when an expression such as "one or more pieces of hardware perform a first process, and the one or more pieces of hardware perform a second process" (including similar expressions) is used, the hardware performing the first process and the hardware performing the second process may be the same or different. In other words, it is sufficient that the hardware performing the first process and the hardware performing the second process are included in the one or more pieces of hardware. Note that hardware may include an electronic circuit, a device including an electronic circuit, etc.

[0197] In this specification (including the claims), when multiple storage devices (memories) store data, each of the multiple storage devices may store only a portion of the data, or may store the entire data. Also, a configuration in which only some of the multiple storage devices store data may be included.

[0198] In this specification (including the claims), terms such as "first," "second," etc. are used merely as a way of distinguishing between two or more elements, and are not necessarily intended to impose technical meanings such as temporal aspect, spatial aspect, sequence, quantity, etc. on the subject. Thus, for example, a reference to a first element and a second element does not necessarily mean that only two elements may be employed therein, that the first element must precede the second element, that the first element must be present in order for the second element to be present, etc.

[0199] Although the embodiments of the present disclosure have been described in detail above, the present disclosure is not limited to the individual embodiments described above. Various additions, modifications, substitutions, partial deletions, etc. are possible within the scope of the conceptual idea and spirit of the present invention, which is derived from the content defined in the claims and their equivalents. For example, when numerical values ​​or formulas are used in the above-described embodiments, they are shown for illustrative purposes and do not limit the scope of the present disclosure. Furthermore, the order of each operation shown in the embodiments is also illustrative and does not limit the scope of the present disclosure.

[0200] This application claims priority from Japanese Patent Application No. 2023-172466, filed on October 4, 2023, with the Japan Patent Office, the entire contents of which are incorporated herein by reference.

[0201] REFERENCE SIGNS LIST 10 Generation device 11 Target generation model 12 Template generation model 20 Optimization device 30 Execution device 40 Terminal device 201 Evaluation data storage unit 202 Initialization unit 203 Evaluation unit 204 Evaluation history storage unit 205 Selection unit 206 Rewriting unit 207 Determination unit 208 Soft prompt optimization unit 209 Verification unit 210 Data generation unit 211 Additional learning unit 301 Template storage unit 302 Request reception unit 303 Input generation unit 304 Result acquisition unit 305 Result output unit 1000 Information processing system

Claims

1. An information processing device comprising at least one memory and at least one processor, wherein the at least one processor obtains an evaluation result obtained by evaluating one or more first templates using a first generative model, generates input information for a second generative model based on the evaluation result, and obtains a second template generated by inputting the input information to the second generative model, the input information including at least one or more template information selected based on the evaluation result and information specifying a structure of the template.

2. The information processing device according to claim 1, wherein the at least one processor repeatedly executes the steps of: obtaining an evaluation result in which the second template is used as the one or more first templates; generating the input information based on the evaluation result; and obtaining the second template until a predetermined condition is satisfied.

3. The information processing device according to claim 1 or 2, wherein the information specifying the structure of the template includes a placeholder indicating a position at which a specified item is to be embedded.

4. The information processing device according to claim 3, wherein the placeholders include a placeholder indicating a position at which a predetermined numeric vector is to be embedded.

5. An information processing device according to any one of claims 1 to 4, wherein the information specifying the structure of the template includes at least information specifying the type of markup language, the type of symbol, the order of contents, or the repetition of items.

6. An information processing device according to any one of claims 1 to 5, wherein the at least one processor generates the input information based on a template selected from the one or more first templates by an optimization algorithm using the evaluation results.

7. The information processing apparatus according to claim 6, wherein the at least one processor selects one or more templates having a high evaluation value indicating the evaluation result from among the one or more first templates.

8. The information processing device according to claim 6 or 7, wherein the at least one processor selects one or more templates having a high evaluation value indicating the evaluation result from among a plurality of templates arbitrarily selected from the one or more first templates.

9. An information processing device according to any one of claims 6 to 8, wherein the at least one processor generates the template information based on a trained model that has learned the relationship between input information generated using the first template and an evaluation result of evaluating the first template.

10. The information processing device according to claim 9, wherein the trained model is subjected to reinforcement learning using the input information as an action and the evaluation result as a reward.

11. An information processing device described in any one of claims 1 to 10, wherein the at least one processor generates second input information using the one or more first templates, obtains an output result generated by inputting the second input information into the first generative model, and obtains the evaluation result based on the output result.

12. The information processing device according to claim 11, wherein the second input information is input information for executing a task of answering a question, and the evaluation result is information indicating the accuracy rate of the answers included in the output result.

13. The information processing device according to claim 11 or 12, wherein the at least one processor presents the output result to a user, and obtains the evaluation result based on feedback information obtained from the user.

14. The information processing device according to claim 1, wherein the at least one processor generates the first template based on a third generative model.

15. An information processing device according to any one of claims 6 to 14, wherein the at least one processor generates the template information based on one or more templates having a low evaluation value indicating the evaluation result among the one or more first templates.

16. The information processing device according to claim 15, wherein the at least one processor uses the second template to generate a dataset including input information for the first generative model, the dataset being a dataset used for additional training of the first generative model.

17. An information processing device according to any one of claims 1 to 16, wherein the first generative model and the second generative model are the same generative model.

18. An information processing method, in which at least one processor obtains an evaluation result obtained by evaluating one or more first templates using a first generative model, generates input information for a second generative model based on the evaluation result, and obtains a second template generated by inputting the input information to the second generative model, the input information including at least one or more template information selected based on the evaluation result and information specifying a structure of the template.

19. A program that causes at least one processor to execute the steps of: obtaining an evaluation result of evaluating one or more first templates using a first generative model; generating input information for a second generative model based on the evaluation result; and obtaining a second template generated by inputting the input information to the second generative model, wherein the input information includes at least one or more template information selected based on the evaluation result and information specifying the structure of the template.

Images