Information processing device, information processing system, information processing method, and program

Abstract

An information processing device comprising at least one memory and at least one processor, wherein the at least one processor: generates a first prompt; acquires a first output result generated by inputting the first prompt into a language model; acquires first feedback information on the first output result; generates a second prompt using information acquired by running an optimization algorithm based on the first feedback information; and acquires a second output result generated by inputting the second prompt into the language model.

Description

Information processing device, information processing system, information processing method and program

[0001] The present disclosure relates to an information processing device, an information processing system, an information processing method, and a program.

[0002] A language model recognizes text, extracts and classifies information from data containing large amounts of text, such as documents, and directly generates text. Language models are used in various text-based fields, such as search engines, document creation, free dialogue, data parsing, providing expert knowledge, programming, and text style conversion. For example, when a command statement called a prompt is input, the language model generates target information according to the conditions specified by the prompt (see, for example, Patent Document 1).

[0003] JP 2023-43129 A

[0004] An object of the present disclosure is to provide an information processing device that assists in generating prompts to be input into a language model.

[0005] An information processing device according to one aspect of the present disclosure includes at least one memory and at least one processor, wherein the at least one processor generates a first prompt, obtains a first output result generated by inputting the first prompt into a language model, obtains first feedback information for the first output result, generates a second prompt using information obtained by executing an optimization algorithm based on the first feedback information, and obtains a second output result generated by inputting the second prompt into the language model.

[0006] FIG. 1 is a configuration diagram of an example of an information processing system according to the present embodiment. FIG. 2 is an explanatory diagram of an example of a prompt according to the present embodiment. FIG. 3 is an image diagram of an example of an app screen according to the present embodiment. FIG. 4 is a sequence diagram of an example of a processing procedure of the information processing system according to the present embodiment. FIG. 5 is a sequence diagram of an example of a processing procedure of the information processing system according to the present embodiment. FIG. 6 is an explanatory diagram of an example of a prompt according to the present embodiment. FIG. 7 is a code example for measuring the distance between vector expressions of the text of items and using the measured distance to select the next item. FIG. 8 is an explanatory diagram of an example of an evaluation value input form. FIG. 9 is an explanatory diagram of an example of an evaluation value input form. FIG. 10 is a block diagram showing an example of the hardware configuration of each device in the above-described embodiment. FIG. 11 is an explanatory diagram of an example of a prompt template according to the present embodiment.

[0007] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. In this specification and drawings, components having substantially the same functional configurations are designated by the same reference numerals, and redundant description will be omitted.

[0008] [System Configuration] Fig. 1 is a configuration diagram of an example of an information processing system 10 according to this embodiment. The information processing system 10 in Fig. 1 may include a client terminal 12 and a server device 14. The client terminal 12 and the server device 14 may be connected to each other so as to be able to communicate data with each other via a communication network 20 such as a local area network (LAN) or the Internet.

[0009] The client terminal 12 may be an information processing device operated by a user, such as a personal computer (PC), a tablet terminal, or a smartphone. The client terminal 12 may execute an application program (hereinafter referred to as an app) according to this embodiment and display the app screen. The client terminal 12 may also be equipped with a program that executes black-box optimization, which is an example of an optimization algorithm. Optuna (registered trademark) is an example of a black-box optimization tool.

[0010] The black-box optimizer may iteratively select one or more promising items based on the history of completed trials and perform the next trial with a prompt containing the selected one or more items to select (decide) one or more items to try in the next trial from among the multiple items available in the prompt as options. A prompt is an example of a statement to a language model.

[0011] The server device 14 may be equipped with a language model such as a large language model. The server device 14 may input a prompt received from the client terminal 12 into the language model, output an output result according to the conditions specified by the prompt, and transmit the output result to the client terminal 12. The server device 14 may be an information processing device such as a PC or a workstation.

[0012] The information processing system 10 in FIG. 1 is an example. There are various system configuration examples for the information processing system 10 depending on the application and purpose. For example, the information processing system 10 may be composed of one or more information processing devices. A single information processing device may be a system composed of multiple devices. The information processing system 10 may cause the black-box optimization tool executed on the client terminal 12 to be executed on the server device 14 instead of the client terminal 12. The information processing system 10 may have a server device other than the server device 14 that executes the black-box optimization tool instead of the client terminal 12. The server device 14 of the information processing system 10 may be realized as a cloud computing service.

[0013] [Example of Prompt] Figure 2 is an explanatory diagram of an example of a prompt 1000 according to this embodiment. The prompt 1000 in Figure 2 shows a hypothetical example in which a language model is made to generate a cooking recipe.

[0014] Portion 1002 of prompt 1000 specifies one or more conditions determined by the black-box optimization tool. Portion 1002 of prompt 1000 specifies one or more conditions of the genre, cooking method, taste, and texture of the recipe to be generated next. For example, portion 1002 of prompt 1000 specifies the condition "ethnic" as the genre of the recipe to be generated, and the condition "fried" as the cooking method of the recipe to be generated.

[0015] Part 1004 of prompt 1000 is a constraint condition specified based on user feedback information for recipes previously generated by the language model. Part 1004 of prompt 1000 is added each time a user inputs feedback information written freely in natural language for a recipe previously generated by the language model.

[0016] Portion 1006 of prompt 1000 contains the recipe name "Sweet and spicy Korean-style stewed rice," which is part of the information of a recipe previously generated by a language model. Portion 1006 of prompt 1000 may be input by the user by copying and pasting the recipe name "Sweet and spicy Korean-style stewed rice" from a previously generated recipe. Portion 1008 of prompt 1000 contains conditionally specified feedback information in the user's natural language regarding the recipe with the recipe name in portion 1006 of prompt 1000.

[0017] For example, the text "Please use two or more types of vegetables" freely entered by the user in portion 1008 of prompt 1000 becomes a constraint for the next recipe to be generated. The portions of prompt 1000 other than portions 1002, 1006, and 1008 may be fixed.

[0018] 2, a markup language such as "Markdown," "JSON," "YAML," or "XML" may be specified as the output format in prompt 1000. Prompt 1000 that specifies a markup language as the output format can convert the output results generated by the language model into a format that is easy for the user to view by combining it with a viewer that supports the corresponding markup language, such as the Markdown viewer.

[0019] Furthermore, prompt 1000, which specifies a markup language as the output format, can extract some information, such as the name of a dish. Therefore, the app according to this embodiment can easily output a list of dish names, such as a list 1102 of multiple recipes, which will be described later.

[0020] 2 , a recipe previously generated by a language model can be specified in prompt 1000 by extracting summary information such as the "name of the dish" from the previously generated recipe and including it in prompt 1000, rather than entering the entire recipe in prompt 1000, thereby preventing the number of characters in prompt 1000 from becoming too large. For example, methods for extracting summary information include requesting a summary of the previously generated recipe from server device 14 and extracting it, or writing prompt 1000 so that the name of the dish in the recipe is entered in a predetermined position, such as the first line, of the recipe generated by the language model and extracting it mechanically.

[0021] Furthermore, in this embodiment, in order to specify one or more items from a plurality of items as options using a black-box optimization tool, candidate items 1202 may be listed in a prompt template 1200, as shown in FIG. 10 , for example. The prompt template 1200 is used to create the prompt 1000. FIG. 10 is an explanatory diagram of an example of a prompt template according to this embodiment. Portion 1204 of the prompt template 1200 is a location where a recipe name, which is part of the recipe information generated by the language model, is entered. Portion 1206 of the prompt template 1200 is a location where the user's feedback information in natural language regarding the recipe with the recipe name in portion 1204 is entered.

[0022] For example, the prompt template used to generate prompt 1000 may list candidate items as follows: In the following example, "Japanese cuisine," "Chinese cuisine," "Korean cuisine," "French cuisine," "Italian cuisine," "ethnic cuisine," and "not specified" are listed as candidate items for "genre," "grilled cuisine," "steamed cuisine," "boiled cuisine," "simmered cuisine," "fried cuisine," and "not specified" are listed as candidate items for "cooking method," and "chewy cuisine," "crispy cuisine," and "not specified" are listed as candidate items for "texture." "The following specifications are also available when creating a recipe: * Genre: {trial.suggest_categorical("Genre", ["Japanese", "Chinese", "Korean", "French", "Italian", "Ethnic", "No Specification"])} * Cooking method: {trial.suggest_categorical("Cooking method", ["Grilled", "Steamed", "Boiled", "Simmered", "Deep-fried", "No Specification"])} * Taste: {trial.suggest_categorical("Taste", ["Spicy", "Sweet", "Sour", "Salty", "No Specification"])} * Texture: {trial.suggest_categorical("Texture", ["Chewy", "Crunchy", "No Specification"])}" Furthermore, as shown in portion 1012 of prompt 1000 in Figure 2, prompt 1000 may specify not only the output format but also the units of ingredients, thereby unifying the format of the output results generated by the language model.

[0023] For example, prompt 1000 in FIG. 2 specifies the quantities of ingredients in a cooking recipe generated by a language model as "cups," "tablespoons," "teaspoons," "grams," and "milliliters."

[0024] 2, the format of the output results generated by the language model may be unified by specifying a unit of time in the prompt 1000. The prompt 1000 in FIG. 2 is an example and may include information other than text (images, audio, etc.).

[0025] [Example of Recipe] A recipe, which is a hypothetical example of an output result generated by a language model in response to input of prompt 1000 in Figure 2, may be displayed on an app screen 1100 shown in Figure 3. Figure 3 is an image diagram of an example of the app screen 1100 according to this embodiment.

[0026] The app screen 1100 of Figure 3 may display a list 1102 of multiple recipes previously generated by the language model. The app screen 1100 of Figure 3 displays a recipe name 1104 and recipe details 1106 of a recipe selected by a user from the list 1102. The recipe name 1104 and recipe details 1106 of the app screen 1100 of Figure 3 are an example of an output result generated by inputting the prompt 1000 into the language model.

[0027] The application screen 1100 of FIG. 3 displays a recipe name 1104 and a feedback information input field 1108 for receiving input of feedback information from the user regarding recipe details 1106 .

[0028] A user can input information indicating a rating value, which is an example of feedback information for a recipe displayed on the app screen 1100, and free-form text, which is an example of natural language information, in a feedback information input field 1108 on the app screen 1100. The information indicating the rating value may be, for example, "Good," "So-so," or "Bad," which is associated with the rating value, as shown in the feedback information input field 1108 of FIG. 3 . The feedback information may be obtained by recognizing the user's voice. Alternatively, the feedback information may be obtained by having the user select one or more desirable recipes from a list 1102 of multiple recipes previously generated by a language model, or by having the user select one or more undesirable recipes from the list 1102 of multiple recipes. Details of the feedback information input field 1108 on the app screen 1100 will be described later.

[0029] [Processing Procedure] The information processing system 10 according to this embodiment assists in generating a prompt to be input to a language model, for example, according to the procedures shown in Figures 4 and 5. Figures 4 and 5 are sequence diagrams illustrating an example of the processing procedure of the information processing system 10 according to this embodiment.

[0030] In step S10, the client terminal 12 generates a first prompt based on a prompt template, which is an example of a predetermined template. Note that the first prompt may be an initial prompt. For example, in the case of prompt 1000 in FIG. 2, the client terminal 12 may randomly select an item in portion 1002 of prompt 1000.

[0031] The client terminal 12 may select the items in portion 1002 of prompt 1000 according to a predetermined setting rather than randomly. Whether the items are selected randomly or according to a predetermined setting may be customizable by the user. Note that in the case of the first prompt 1000, portion 1004 may be blank. The client terminal 12 generates a first prompt based on the selection of the items in portion 1002 of prompt 1000 and the prompt template.

[0032] In step S12, the client terminal 12 transmits the first prompt generated in step S10 to the server device 14. In step S14, the server device 14 receives the first prompt. The server device 14 inputs the received first prompt into a language model to generate a first output result. For example, the server device 14 may input the first prompt into ChatGPT (registered trademark), an example of a language model, to generate the first output result. The first output result is, for example, a cooking recipe generated by the language model in accordance with the conditions specified by the first prompt. In step S16, the server device 14 transmits the first output result generated in step S14 to the client terminal 12.

[0033] In step S18, the client terminal 12 displays an app screen including the first output result received from the server device 14. For example, the client terminal 12 displays the app screen 1100 of FIG. 3 including a recipe, which is an example of the first output result received from the server device 14.

[0034] In step S20, the user evaluates the first output result displayed on the screen of the application and inputs first feedback information on the first output result to the client terminal 12.

[0035] For example, the user evaluates the recipe displayed on the screen 1100 of the application in FIG. 3 and inputs the evaluation of the recipe as the user's feedback information in the feedback information input field 1108 .

[0036] The client terminal 12 acquires, as first feedback information, the user's evaluation of the first output result displayed on the screen of the app. For example, in the case of the app screen 1100 of Fig. 3, the client terminal 12 can acquire, as feedback information for the recipe, information indicating the user's evaluation value for the recipe entered in the feedback information input field 1108 and free-form text.

[0037] The user's designation of evaluation information for the first output result displayed on the screen of the app may be received by operating a radio button associated with an evaluation value or by operating a slider associated with an evaluation value. The user's designation of evaluation information may also be performed using a UI (User Interface) for comparative evaluation, which simultaneously displays multiple first output results and allows the user to select the first output result that has received the highest evaluation. In the UI for comparative evaluation, the evaluation value of the selected first output result is higher than the evaluation values ​​of the first output results that have not been selected. In the UI for comparative evaluation, a value indicating how superior a certain output result is to other output results may be input using a slider or the like. In the UI for comparative evaluation, the output results of the language models may be processed by a program to perform the comparative evaluation.

[0038] The user's evaluation information may be specified by a UI for ranking evaluation that simultaneously displays multiple first output results and allows the user to rank them. The evaluation information may be specified by an evaluation value that can be calculated by a computer, instead of by the user's evaluation information. In the case of a recipe, the evaluation value that can be calculated by a computer may be the calories of ingredients, etc. Furthermore, the evaluation information may be specified by combining the evaluation value specified by the user and the evaluation value calculated by the computer.

[0039] In step S22, the client terminal 12 executes black-box optimization using the first feedback information as an evaluation value to obtain information to be used for generating a second prompt. The client terminal 12 determines an item to be specified as a condition in the next prompt based on a history that associates one or more items determined by past executions of black-box optimization with evaluation values ​​for a first output result generated by inputting a prompt in which the item is specified as a condition into a language model.

[0040] The optimization algorithm based on feedback information may be, for example, a black-box optimization method or a gray-box optimization method. The feedback information may be obtained from a user or may be automatically generated by the client terminal 12. Black-box optimization and gray-box optimization are examples of optimization algorithms based on feedback information. Examples of black-box optimization include multi-objective optimization, Bayesian optimization, and evolutionary computing. Examples of gray-box optimization include multi-fidelity optimization. Specifically, black-box optimization may use methods such as TPE (Tree-Structured Parzen Estimator), GP-B0, CMA-ES, and NSGA-II. Black-box optimization may balance and select items with high evaluation values ​​and items that have not been tested. Black-box optimization may also be used to adjust the value of temperature, which is an example of a parameter used when executing a language model. Temperature is a parameter that adjusts the randomness of the output results of the language model.

[0041] The client terminal 12 may use a part of the history that associates items determined by past execution of black-box optimization with evaluation values ​​for the first output results generated by inputting prompts with conditions specified for the items into the language model. For example, the history to be used may be filtered according to a user's specification.

[0042] The client terminal 12 may use a portion of the recipe history by filtering, such as the history of recipes with two or more vegetables or the history of Italian recipes. Filtering allows the client terminal 12 to respond to changes in the black box of the black box optimization. For example, when considering recipes with two or more vegetables, recipes that do not use any vegetables and their evaluation values ​​may become noise in the black box optimization algorithm. In such cases, limiting (filtering) recipes to those with two or more vegetables eliminates the noise, allowing the black box optimization algorithm to be applied as expected.

[0043] Furthermore, when filtering the history, the client terminal 12 may request the user to reassign the evaluation values. For example, the client terminal 12 may display a UI for reassigning the evaluation values ​​on the screen of the application. Filtering of the history may be performed by a user's specification.

[0044] Furthermore, the filtering of the history may be specified by a language model triggered by feedback information in the user's natural language. For example, when the first output result is a recipe, the client terminal 12 inputs a prompt question, "Does this recipe use two or more types of vegetables?" for each past recipe into a language model such as ChatGPT, and obtains a Yes or No answer.

[0045] In step S24, the client terminal 12 generates a second prompt using the information obtained by executing the black-box optimization in step S22. If the feedback information in step S20 includes text freely written by the user in natural language, the client terminal 12 further uses the free-form text to create a second prompt.

[0046] For example, in the case of prompt 1000 in Fig. 2, the client terminal 12 specifies the items determined by executing black-box optimization in step S22 as conditions in part 1002 of prompt 1000. In addition, in the case of app screen 1100 in Fig. 3, the client terminal 12 specifies free-form text, which is an example of natural language information entered in feedback information input field 1108, as a constraint in part 1008 of prompt 1000 in Fig. 2.

[0047] In step S26, the client terminal 12 transmits the second prompt generated in step S24 to the server device 14. In step S28, the server device 14 receives the second prompt. The server device 14 inputs the received second prompt into a language model to generate a second output result. For example, the server device 14 may input the second prompt into ChatGPT (registered trademark), an example of a language model, to generate the second output result. The second output result is, for example, a cooking recipe generated by the language model in accordance with the conditions specified by the second prompt. In step S30, the server device 14 transmits the second output result generated in step S28 to the client terminal 12.

[0048] In step S32, the client terminal 12 displays a screen of the app including the second output result received from the server device 14. For example, the client terminal 12 displays a screen of the app including a recipe, which is an example of the second output result received from the server device 14.

[0049] In step S34, the user evaluates the second output result displayed on the screen of the app and inputs second feedback information regarding the second output result to the client terminal 12. For example, the user evaluates the recipe displayed in step S32 and inputs the evaluation of the recipe as the user's feedback information in a feedback information input field.

[0050] The client terminal 12 acquires, as second feedback information, the user's evaluation of the second output result displayed in step S32. For example, in the case of the app screen 1100 of FIG. 3 , the client terminal 12 can acquire, as feedback information for the recipe, information indicating the user's evaluation value for the recipe entered in the feedback information input field 1108 and free-form text.

[0051] In step S36, the client terminal 12 performs black-box optimization using the second feedback information as an evaluation value. In step S38, the client terminal 12 generates the next second prompt using the information obtained by performing the black-box optimization in step S36.

[0052] Furthermore, if the feedback information in step S34 includes text freely written by the user in natural language, the client terminal 12 further uses the freely written text to create the next second prompt.

[0053] In step S40, the client terminal 12 transmits the next second prompt generated in step S38 to the server device 14. In step S42, the server device 14 receives the next second prompt. The server device 14 inputs the received next second prompt into a language model to generate a next second output result.

[0054] For example, the server device 14 may input the next second prompt into ChatGPT (registered trademark), which is an example of a language model, and generate a next second output result. The next second output result may be, for example, a cooking recipe generated by the language model in accordance with the conditions specified by the next second prompt. In step S44, the server device 14 transmits the next second output result generated in step S42 to the client terminal 12.

[0055] In step S46, the client terminal 12 displays a screen of the app including the next second output result received from the server device 14. For example, the client terminal 12 displays a screen of the app including a recipe, which is an example of the next second output result received from the server device 14.

[0056] In step S48, the client terminal 12 determines whether or not a predetermined criterion for stopping the repeated execution of the processes of steps S34 to S48 is met. If the predetermined criterion is not met, the client terminal 12 repeats the processes of steps S34 to S48. If the predetermined criterion is met, the client terminal 12 stops the processes of steps S34 to S48 until the predetermined criterion is no longer met. For example, the client terminal 12 may determine whether or not the predetermined criterion is met each time it receives an output result from the server device 14.

[0057] For example, in the example of the app screen 1100 in FIG. 3, the processing of steps S34 to S48 may be repeated until a predetermined criterion is met that the list 1102 contains five output results for which the user has not provided feedback on their evaluation.

[0058] The client terminal 12 that has stopped the processing of steps S34 to S48 may determine that the predetermined criterion is no longer satisfied by the user evaluating the output results included in the list 1102 and providing feedback, and may resume the processing of steps S34 to S48. The predetermined criterion may also be reaching an upper limit on the number of output results. The predetermined criterion may also be convergence of the output results to the user's preferences. The client terminal 12 may also check the load status of the server device 14 and stop the processing of steps S34 to S48 if the server device 14 is congested and there is no room for calculation.

[0059] The black-box optimization of the sequence diagrams shown in Figures 4 and 5 may be performed by trying all combinations of items. Alternatively, the black-box optimization of the sequence diagrams shown in Figures 4 and 5 may be performed by randomly selecting items for the first predetermined number of times (e.g., five times), and then selecting promising items based on past history for the subsequent predetermined number of times.

[0060] 4 and 5 may be omitted by copying the items specified in the previous prompt if the feedback information includes text freely written by the user in natural language. In this case, the next prompt will have the text freely written by the user in natural language specified as a condition in, for example, part 1008 of prompt 1000 in FIG. 2, and the condition specification of the items in part 1002 of prompt 1000 will remain unchanged.

[0061] [Black-box Optimization] The search space of items determined by performing black-box optimization changes depending on whether or not the user has freely written text in natural language in the feedback information, or on the content of the user's description in the feedback information. Therefore, in the black-box optimization according to this embodiment, if the previous search results change, or if the content of the user's description in the feedback information has an overall impact, the item search may be redone.

[0062] An example of a case where the content written by the user in the feedback information has an overall effect is when the condition "two or more types of vegetables" is specified without specifying the name of a dish. When the condition "two or more types of vegetables" is specified without specifying the name of a dish, search results that use two or more types of vegetables are extracted from the past history, and the search for the item is redone based on the past history.

[0063] Furthermore, the black-box optimization according to the present embodiment may be performed by continuously searching for items, which allows the information processing system 10 according to the present embodiment to generate multiple prompts and input each of the multiple prompts into a language model to obtain multiple output results (e.g., recipes).

[0064] Generally, the process of generating a prompt and inputting the generated prompt into a language model to obtain an output result takes time, so if a user repeatedly generates a prompt, inputs the generated prompt into a language model to obtain an output result, and then evaluates the output result, the user will have to wait a long time.

[0065] Therefore, in this embodiment, multiple prompts are generated in advance, and the generated multiple prompts are input into a language model to obtain multiple output results, allowing the user to evaluate the multiple output results consecutively at once, thereby improving UX (user experience).

[0066] Furthermore, in black-box optimization, conceptual closeness may be used to determine the items. For example, the conceptual closeness may be determined by using the similarity of vector representations (embeddings) of the text of each candidate item in a large-scale language model (LLM). The similarity of vector representations of the text of each candidate item is an existing technique and can be obtained as a vector.

[0067] FIG. 6 is an explanatory diagram of an example of a prompt 1000 according to this embodiment. Note that the prompt in FIG. 6 is similar to the prompt 1000 in FIG. 2 except for portion 1002. In portion 1002 of prompt 1000, the item of the basic seasoning of the recipe to be generated next is additionally specified as a condition to portion 1002 in FIG. 2. For example, in portion 1002 of prompt 1000 in FIG. 6, the item "ethnic" is specified as the genre of the recipe to be generated, the item "frying" is specified as the cooking method of the recipe to be generated, and the item "salt" is specified as the basic seasoning of the recipe to be generated.

[0068] The candidate items for basic seasonings may be listed as soy sauce, salt, miso, shio koji, chili bean paste, consommé, medium-thick sauce, ketchup, sugar, etc. In addition to the candidate items for basic seasonings, the list may also be used for candidate items for vegetables or proteins.

[0069] Potential vegetable items may be listed such as cabbage, onion, potato, eggplant, bell pepper, cucumber, tomato, etc. Potential protein items may be listed such as beef, pork, chicken, salmon, mackerel, tuna, shrimp, crab, etc.

[0070] For example, in the case of the prompt in Figure 6, if the evaluation value of the recipe generated by the prompt when the basic seasoning item "salt" is selected is high, the item "soy sauce" whose text vector representation is close to the item "salt" may be selected preferentially. Also, among items with a small number of attempts, an item similar to an item with a high evaluation value in the past history may be selected as the next item. The distance between the text vector representations, etc., can be used to determine the similarity of items.

[0071] For example, in the case of a prompt to generate a recipe, it is expected that the number of ingredients and seasonings for the dish is finite, and that there is meaning in terms of the distance between vector representations of the text.

[0072] For example, when selecting one of the "basic seasoning" items in portion 1002 of prompt 1000 in Figure 6, there are likely to be many recipes that use "salt" instead of "soy sauce." On the other hand, there are likely to be few recipes that use "sugar" instead of "soy sauce."

[0073] In this way, by utilizing the known relationships that "soy sauce" and "salt" are close and "soy sauce" and "sugar" are far from each other, the black-box optimization of this embodiment may select "salt" rather than "sugar" as the next item after "soy sauce."

[0074] For example, the following numeric vectors may be assigned to candidate items. Whether items are similar or dissimilar can be determined from cosine similarity. By using the internal representation of the language model as a numeric vector, it is possible to determine whether items are similar or dissimilar. - Soy sauce: [0.8, 0.8] - Salt: [0.7, 0.8] - Sugar: [-0.5, -0.5] Figure 7 shows an example of code that implements a process for measuring the distance between the vector representations of the text of items and using it to select the next item. The code in Figure 7 measures the distance between base condiment items, and the distance between the base condiment items can be used when selecting the next item in black-box optimization.

[0075] Furthermore, in the information processing system 10 according to this embodiment, due to the influence of the seed, the information acquired by performing black-box optimization may change even if the parameters are the same. In this case, the client terminal 12 may sample multiple parameters and generate multiple items based on the respective parameters. For example, the client terminal 12 may sample n parameters and generate items based on the respective parameters.

[0076] The user generates multiple prompts using the generated items and evaluates the output results generated from each of the multiple prompts, for example, by voting. The voting may be a ranking or a rating (including negative evaluation). The client terminal 12 may determine the evaluation value of the parameter based on the user's voting results (e.g., the number of votes cast).

[0077] [Ensuring consistency of user evaluation information] The user can specify an evaluation value for the output result displayed on the screen of the app using various evaluation value input forms, for example, as shown in Figures 8A to 8C. Figures 8A to 8C are explanatory diagrams of examples of evaluation value input forms.

[0078] For example, FIG. 8A is an example of an evaluation value input form that accepts input of an evaluation value from a user by operating a radio button associated with the evaluation value. FIG. 8B is an example of an evaluation value input form that accepts input of an evaluation value (e.g., 1 to 5) from a user by operating a slider associated with the evaluation value. FIG. 8C is an example of an evaluation value input form that accepts input of an evaluation value from a user for multi-objective optimization that combines a quantitative evaluation value (such as AUC (Area Under the Curve)) and a subjective evaluation value (human). Note that a user (e.g., a programmer or a general user) may select which of the evaluation value input forms shown in FIGS. 8A to 8C to use.

[0079] Furthermore, the user's evaluation value for the output result displayed on the screen of the app may be specified by a UI for comparative evaluation that simultaneously displays two output results and allows the user to select the output result that has received the highest evaluation.The user's evaluation value for the output result displayed on the screen of the app may be specified by a UI for comparative evaluation that simultaneously displays a past output result with the highest evaluation value and the current output result and allows the user to select the output result that has received the highest evaluation.The user's evaluation value for the output result displayed on the screen of the app may be specified by a UI for comparative evaluation that simultaneously displays a previous output result and the current output result and allows the user to select the output result that has received the highest evaluation.

[0080] In addition, the user's evaluation value for the output results displayed on the app screen may be specified using a UI for comparative evaluation that displays a list of output results evaluated by the user and allows the user to select the output result that has been highly rated by the user.

[0081] [Application Examples] In this embodiment, a cooking recipe has been described as an example of an output result generated by inputting prompts and a language model, but the information processing system 10 according to this embodiment may be used for other applications such as image generation, code review, substance generation, and finance. Image generation may be the generation of a character such as an animated character. Substance generation may be the generation of a substance from ingredients.

[0082] Code review may select items such as testing the target area with black-box optimization, data structure, and documentation. Variations of the items may also include items from qualitative perspectives such as consistency, maintainability, and errors. Furthermore, variations of the items may include items on context (code differences). Finance may involve creating an individual stock investment strategy.

[0083] In addition, while this embodiment illustrates a cooking recipe as an example of a recipe, it is also possible to apply it to diet recipes, health promotion recipes, recipes that satisfy certain conditions, and the like. Furthermore, the recipes generated by the information processing system 10 of this embodiment may be used as input to other recommendation systems that recommend recipes. For example, even in recommendation systems that require strict input such as the genre of cuisine or nutritional requirements, users can easily generate information to input to the recommendation system by inputting feedback information for the recipes generated by the information processing system 10 of this embodiment. Therefore, the recommendation system can improve the UX of users without specialized knowledge.

[0084] [Hardware Configuration] Some or all of the devices (client terminal 12 and server device 14) in the above-described embodiments may be configured as hardware, or may be configured as software (program) information processing executed by a CPU (Central Processing Unit), GPU (Graphics Processing Unit), or the like. When configured as software information processing, software that realizes at least some of the functions of each device in the above-described embodiments may be stored on a non-transitory storage medium (non-transitory computer-readable medium) such as a CD-ROM (Compact Disc-Read Only Memory) or USB (Universal Serial Bus) memory, and the software information processing may be executed by loading the software into a computer. The software may also be downloaded via a communications network. Furthermore, all or part of the software processing may be implemented in a circuit such as an ASIC (Application Specific Integrated Circuit) or FPGA (Field Programmable Gate Array), thereby allowing the software information processing to be executed by hardware.

[0085] The storage medium that stores the software may be a removable medium such as an optical disk, or a fixed medium such as a hard disk, memory, etc. The storage medium may be provided inside the computer (such as a main storage device or auxiliary storage device) or outside the computer.

[0086] 9 is a block diagram showing an example of the hardware configuration of each device (client terminal 12 and server device 14) in the above-described embodiment. Each device may be realized as a computer 7 including, for example, a processor 101, a main storage device 102 (memory), an auxiliary storage device 103 (memory), a network interface 104, and a device interface 105, which are connected via a bus 106.

[0087] Although the computer 7 in FIG. 9 includes one of each component, it may also include multiple of the same component. Also, while FIG. 9 shows one computer 7, the software may be installed on multiple computers, with each of the multiple computers executing the same or different parts of the software. In this case, a distributed computing configuration may be used in which each computer communicates via a network interface 104 or the like to execute the processing. In other words, each device (client terminal 12 and server device 14) in the above-described embodiment may be configured as a system in which one or more computers execute instructions stored in one or more storage devices to realize functions. Furthermore, information sent from a terminal may be processed by one or more computers located on the cloud, and the processing results may be sent to the terminal.

[0088] The various calculations of each device (client terminal 12 and server device 14) in the above-described embodiments may be executed in parallel using one or more processors, or using multiple computers connected via a network. Furthermore, the various calculations may be distributed to multiple processor cores within a processor and executed in parallel. Furthermore, some or all of the processes, means, etc. disclosed herein may be implemented by at least one processor and storage device provided on a cloud that can communicate with computer 7 via a network. Thus, each device in the above-described embodiments may be implemented in the form of parallel computing using one or more computers.

[0089] The processor 101 may be an electronic circuit (processing circuit, processing circuitry, CPU, GPU, FPGA, ASIC, etc.) that at least controls or performs calculations on a computer. The processor 101 may be a general-purpose processor, a dedicated processing circuit designed to perform a specific calculation, or a semiconductor device that includes both a general-purpose processor and a dedicated processing circuit. The processor 101 may also include an optical circuit or a calculation function based on quantum computing.

[0090] The processor 101 may perform arithmetic processing based on data or software input from each device or the like configured internally of the computer 7, and may output arithmetic results or control signals to each device or the like. The processor 101 may control each component constituting the computer 7 by executing the OS (Operating System) of the computer 7, applications, etc.

[0091] Each device (client terminal 12 and server device 14) in the above-described embodiment may be realized by one or more processors 101. Here, the processor 101 may refer to one or more electronic circuits arranged on one chip, or may refer to one or more electronic circuits arranged on two or more chips or two or more devices. When multiple electronic circuits are used, the electronic circuits may communicate with each other via wire or wirelessly.

[0092] The main memory device 102 may store instructions executed by the processor 101, various data, etc., and information stored in the main memory device 102 may be read by the processor 101. The auxiliary memory device 103 is a memory device other than the main memory device 102. Note that these memory devices refer to any electronic component capable of storing electronic information and may be semiconductor memory. The semiconductor memory may be either volatile memory or non-volatile memory. The memory devices for saving various data, etc. in each device (client terminal 12 and server device 14) in the above-described embodiments may be realized by the main memory device 102 or the auxiliary memory device 103, or may be realized by internal memory built into the processor 101. For example, each memory unit in the above-described embodiments may be realized by the main memory device 102 or the auxiliary memory device 103.

[0093] In the above-described embodiment, when each device (client terminal 12 and server device 14) is configured with at least one storage device (memory) and at least one processor connected (coupled) to the at least one storage device, at least one processor may be connected to one storage device. At least one storage device may be connected to one processor. The present invention may also include a configuration in which at least one processor among multiple processors is connected to at least one storage device among multiple storage devices. This configuration may also be realized by storage devices and processors included in multiple computers. Furthermore, the present invention may also include a configuration in which a storage device is integrated with a processor (for example, a cache memory including an L1 cache and an L2 cache).

[0094] The network interface 104 is an interface for connecting to the communication network 8 wirelessly or via a wire. The network interface 104 may be an appropriate interface, such as one that conforms to an existing communication standard. The network interface 104 may exchange information with an external device 9A connected via the communication network 8. The communication network 8 may be any one of a wide area network (WAN), a local area network (LAN), a personal area network (PAN), etc., or a combination thereof, as long as information is exchanged between the computer 7 and the external device 9A. An example of a WAN is the Internet, an example of a LAN is IEEE 802.11 or Ethernet (registered trademark), and an example of a PAN is Bluetooth (registered trademark) or NFC (Near Field Communication), etc.

[0095] The device interface 105 is an interface such as a USB that directly connects to the external device 9B.

[0096] The external device 9A is a device connected to the computer 7 via a network, and the external device 9B is a device connected directly to the computer 7.

[0097] The external device 9A or the external device 9B may be, for example, an input device. The input device is, for example, a camera, a microphone, a motion capture device, various sensors, a keyboard, a mouse, a touch panel, or the like, and provides acquired information to the computer 7. Alternatively, the external device 9A or the external device 9B may be a device including an input unit, a memory, and a processor, such as a personal computer, a tablet terminal, or a smartphone.

[0098] Furthermore, the external device 9A or the external device B may be, for example, an output device. The output device may be, for example, a display device such as an LCD (Liquid Crystal Display) or an organic EL (Electro Luminescence) panel, or a speaker that outputs sound or the like. Alternatively, the output device may be a device including an output unit, a memory, and a processor, such as a personal computer, a tablet terminal, or a smartphone.

[0099] The external device 9A or the external device 9B may be a storage device (memory). For example, the external device 9A may be a network storage or the like, and the external device 9B may be a storage device such as an HDD.

[0100] Furthermore, the external device 9A or the external device 9B may be a device having some of the functions of the components of each device (the client terminal 12 and the server device 14) in the above-described embodiment. That is, the computer 7 may transmit some or all of the processing results to the external device 9A or the external device 9B, or may receive some or all of the processing results from the external device 9A or the external device 9B.

[0101] In this specification (including the claims), when the expression "at least one of a, b, and c" or "at least one of a, b, or c" (including similar expressions) is used, it includes any of a, b, c, ab, ac, bc, or abc. It may also include multiple instances of any element, such as aa, abb, aabbcc, etc. Furthermore, it also includes the addition of elements other than the enumerated elements (a, b, and c), such as having d, as in abcd.

[0102] In this specification (including claims), when expressions such as "using / using data as input / based on / according to / in response to data" (including similar expressions) are used, unless otherwise specified, this includes cases where the data itself is used, or where data that has been processed in some way (e.g., data with noise added, normalized data, features extracted from data, intermediate representations of data, etc.) is used. Furthermore, when a statement is made that a result is obtained "using data as input / based on / according to / in response to data" (including similar expressions), this includes cases where the result is obtained based solely on the data, or where the result is influenced by other data, factors, conditions, and / or states other than the data. Furthermore, when a statement is made that "data is output" (including similar expressions), this includes cases where the data itself is used as output, or where data that has been processed in some way (e.g., data with noise added, normalized data, features extracted from data, intermediate representations of various data, etc.) is used as output, unless otherwise specified.

[0103] When the terms "connected" and "coupled" are used in this specification (including the claims), they are intended as open-ended terms that include any of direct connection / coupling, indirect connection / coupling, electrically connection / coupling, communicatively connection / coupling, functionally connection / coupling, and physically connection / coupling. These terms should be interpreted appropriately according to the context in which they are used, but any connection / coupling form that is not intentionally or naturally excluded should be interpreted as being included in these terms without limitation.

[0104] In this specification (including the claims), the expression "A configured to B" may include the physical structure of element A having a configuration capable of performing operation B, and the permanent or temporary setting / configuration of element A being configured / set to actually perform operation B. For example, if element A is a general-purpose processor, it is sufficient that the processor has a hardware configuration capable of performing operation B, and is configured to actually perform operation B by setting a permanent or temporary program (instruction). Also, if element A is a dedicated processor, dedicated arithmetic circuit, etc., it is sufficient that the circuit structure of the processor is implemented to actually perform operation B, regardless of whether control instructions and data are actually attached.

[0105] Whenever words implying containing or possessing (e.g., "comprising / including," "having," etc.) are used in this specification (including the claims), they are intended to be open-ended terms that include containing or possessing things other than the object designated by the object of the term. When the object of such words implying containing or possessing does not specify a quantity or suggests a singular number (e.g., expressions using the articles "a" or "an"), the expression should be construed as not being limited to a specific number.

[0106] In this specification (including the claims), although expressions such as "one or more" and "at least one" are used in some places and expressions that do not specify a quantity or that imply a singular number (expressions using the articles "a" or "an") are used in other places, the latter expressions are not intended to mean "one." In general, expressions that do not specify a quantity or that imply a singular number (expressions using the articles "a" or "an") should be interpreted as not necessarily being limited to a specific number.

[0107] In this specification, when a particular advantage / result is described as being obtained with respect to a particular configuration of an embodiment, it should be understood that the same advantage / result can also be obtained with one or more other embodiments having the same configuration, unless otherwise stated. However, it should be understood that the presence or absence of the effect generally depends on various factors, conditions, and / or situations, and that the effect is not necessarily obtained with the configuration. The effect is merely obtained by the configuration described in the embodiment when various factors, conditions, and / or situations are satisfied, and the effect does not necessarily occur in a claimed invention that defines the same configuration or a similar configuration.

[0108] When terms such as "maximize" / "maximization" are used in this specification (including the claims), they include finding a global maximum, finding an approximation of a global maximum, finding a local maximum, and finding an approximation of a local maximum, and should be interpreted appropriately according to the context in which the terms are used. They also include finding approximations of these maxima probabilistically or heuristically. Similarly, when terms such as "minimize" / "minimization" are used, they include finding a global minimum, finding an approximation of a global minimum, finding a local minimum, and finding an approximation of a local minimum, and should be interpreted appropriately according to the context in which the terms are used. They also include finding approximations of these minima probabilistically or heuristically. Similarly, when terms such as "optimize" / "optimization" are used, they include finding a global optimum, finding an approximation of a global optimum, finding a local optimum, and finding an approximation of a local optimum, and should be interpreted appropriately according to the context in which the terms are used. It also includes finding approximations of these optimum values ​​probabilistically or heuristically.

[0109] In this specification (including claims), when multiple pieces of hardware perform a predetermined process, the pieces of hardware may cooperate to perform the predetermined process, or some of the hardware may perform all of the predetermined process. Furthermore, some of the hardware may perform part of the predetermined process, and other hardware may perform the rest of the predetermined process. In this specification (including claims), when an expression such as "one or more pieces of hardware perform a first process, and the one or more pieces of hardware perform a second process" (including similar expressions) is used, the hardware performing the first process and the hardware performing the second process may be the same or different. In other words, it is sufficient that the hardware performing the first process and the hardware performing the second process are included in the one or more pieces of hardware. Note that hardware may include an electronic circuit, a device including an electronic circuit, etc.

[0110] In this specification (including the claims), when multiple storage devices (memories) store data, each of the multiple storage devices may store only a portion of the data, or may store the entire data. Also, a configuration in which only some of the multiple storage devices store data may be included.

[0111] In this specification (including the claims), terms such as "first," "second," etc. are used merely as a way of distinguishing between two or more elements, and are not necessarily intended to impose technical meanings such as temporal aspect, spatial aspect, sequence, quantity, etc. on the subject. Thus, for example, a reference to a first element and a second element does not necessarily mean that only two elements may be employed therein, that the first element must precede the second element, that the first element must be present in order for the second element to be present, etc.

[0112] Although the embodiments of the present disclosure have been described in detail above, the present disclosure is not limited to the individual embodiments described above. Various additions, modifications, substitutions, partial deletions, etc. are possible within the scope of the conceptual idea and spirit of the present invention, which is derived from the content defined in the claims and their equivalents. For example, when numerical values ​​or formulas are used in the above-described embodiments, they are shown for illustrative purposes and do not limit the scope of the present disclosure. Furthermore, the order of each operation shown in the embodiments is also illustrative and does not limit the scope of the present disclosure.

[0113] [Supplementary Notes] <1> An information processing device comprising at least one memory and at least one processor, wherein the at least one processor generates a first prompt, obtains a first output result generated by inputting the first prompt into a language model, obtains first feedback information for the first output result, generates a second prompt using information obtained by executing an optimization algorithm based on the first feedback information, and obtains a second output result generated by inputting the second prompt into the language model. <2> The information processing device according to <1>, wherein the obtained information is one or more items selected from a plurality of items by executing the optimization algorithm. <3> The information processing device according to <2>, wherein the at least one processor executes the optimization algorithm based on a history of information on one or more items selected in the past and information on evaluation values ​​obtained in the past. <4> The information processing device according to <3>, wherein the at least one processor selects the history to be used in the optimization algorithm. <5> The information processing device according to <4>, wherein the at least one processor selects the history based on either an instruction from a user or output from the language model. <6> The information processing device according to <4>, wherein the at least one processor updates the evaluation value of the history used in the optimization algorithm. <7> The information processing device according to <2>, wherein the at least one processor executes the optimization algorithm using a vector representation of text for each item. <8> The information processing device according to <7>, wherein the at least one processor executes the optimization algorithm using a similarity between vector representations of text between the plurality of items. <9> The information processing device according to <1>, wherein the at least one processor generates the second output result by inputting the second prompt to the language model. <10> The information processing device according to any one of <1> to <9>, wherein the first feedback information includes information indicating an evaluation value and information in natural language acquired from the user.<11> The information processing device according to <10>, wherein the second prompt includes information in a natural language acquired from the user. <12> The information processing device according to any one of <1> to <9>, wherein the second prompt includes information on a portion of the first output result. <13> The information processing device according to any one of <1> to <9>, wherein the at least one processor: acquires second feedback information on the second output result; generates the second prompt using information acquired by executing an optimization algorithm based on the second feedback information; and acquires the second output result generated by inputting the second prompt into the language model, repeating this process until a predetermined criterion is satisfied. <14> The information processing device according to <13>, wherein the second prompt includes information in a plurality of natural languages ​​acquired from a plurality of pieces of second feedback information. <15> The information processing device according to any one of <1> to <9>, wherein the second prompt includes the acquired information and information specifying an output format of the second output result. <16> The information processing device according to <15>, wherein the second prompt specifies a markup language as the output format of the second output result. <17> The information processing device according to any one of <1> to <9>, wherein the first feedback information includes evaluation information for the first output result. <18> The information processing device according to <17>, wherein the evaluation information for the first output result is accepted by a user's operation of a radio button or a slider. <19> The information processing device according to any one of <1> to <9>, wherein the at least one processor generates the first feedback information by analyzing the first output result. <20> The information processing device according to any one of <1> to <9>, wherein the at least one processor generates a plurality of first prompts, obtains a plurality of first output results generated by inputting the plurality of first prompts into a language model, and obtains the first feedback information from the user for the first output result selected by the user from the plurality of first output results.<21> An information processing system including at least the information processing device described in any one of <1> to <9>. <22> An information processing method, in which at least one processor included in the information processing device generates a first prompt, acquires a first output result generated by inputting the first prompt into a language model, acquires first feedback information for the first output result, generates a second prompt using information acquired by executing an optimization algorithm based on the first feedback information, and acquires a second output result generated by inputting the second prompt into the language model. <23> A program that causes at least one processor included in the information processing device to execute the following steps: generate a first prompt, acquire a first output result generated by inputting the first prompt into a language model, acquire first feedback information for the first output result, generate a second prompt using information acquired by executing an optimization algorithm based on the first feedback information, and acquire a second output result generated by inputting the second prompt into the language model.

[0114] The "second prompt" in <13> may be different from the "second prompt" in <1>, but for convenience of expression, they are both referred to as the "second prompt." For example, the "second prompt" in <13> may be a prompt subsequent to the "third prompt" generated following the "second prompt" in <1>.

[0115] Similarly, the "second output result" in <13> may be the "second output result" in <1>, or may be an output result generated by inputting prompts from the "third prompt" generated following the "second prompt" into the language model.

[0116] Similarly, the "second feedback information" in <13> may be feedback information after the "third feedback information" repeatedly acquired following the "first feedback information" in <1>.

[0117] While the present invention has been described above based on the examples, the present invention is not limited to the above examples and various modifications are possible within the scope of the claims. This application claims priority from basic application No. 2023-127594, filed with the Japan Patent Office on August 4, 2023, the entire contents of which are incorporated herein by reference.

[0118] REFERENCE SIGNS LIST 10 Information processing system 12 Client terminal 14 Server device 20 Communication network 101 Processor 1000 Prompt 1100 App screen

Claims

1. An information processing device comprising at least one memory and at least one processor, wherein the at least one processor generates a first prompt, obtains a first output result generated by inputting the first prompt into a language model, obtains first feedback information for the first output result, generates a second prompt using information obtained by executing an optimization algorithm based on the first feedback information, and obtains a second output result generated by inputting the second prompt into the language model.

2. The information processing apparatus according to claim 1, wherein the acquired information is one or more items selected from a plurality of items by execution of the optimization algorithm.

3. The information processing device according to claim 2, wherein the at least one processor executes the optimization algorithm based on a history of information on one or more items selected in the past and information on evaluation values ​​obtained in the past.

4. The information processing apparatus according to claim 3, wherein said at least one processor selects said history to be used in said optimization algorithm.

5. The information processing device according to claim 4, wherein said at least one processor selects said history based on either an instruction from a user or an output from said language model.

6. The information processing device according to claim 4, wherein said at least one processor updates said evaluation value of said history used in said optimization algorithm.

7. The information processing apparatus of claim 2, wherein said at least one processor executes said optimization algorithm using a vector representation of the text of each item.

8. The information processing apparatus according to claim 7, wherein the at least one processor executes the optimization algorithm using similarities of vector representations of text between the plurality of items.

9. The information processing apparatus according to claim 1, wherein the at least one processor generates the second output result by inputting the second prompt into the language model.

10. An information processing device according to any one of claims 1 to 9, wherein the first feedback information includes information indicating an evaluation value and information in a natural language acquired from a user.

11. The information processing apparatus according to claim 10, wherein the second prompt includes information in a natural language obtained from the user.

12. An information processing device according to any one of claims 1 to 9, wherein the second prompt includes information of a part of the first output result.

13. An information processing device according to any one of claims 1 to 9, wherein the at least one processor: acquires second feedback information for the second output result; generates the second prompt using information acquired by executing an optimization algorithm based on the second feedback information; and acquires the second output result generated by inputting the second prompt into the language model. The information processing device according to any one of claims 1 to 9, wherein the at least one processor repeats the process until a predetermined criterion is satisfied.

14. The information processing apparatus according to claim 13, wherein the second prompt includes information in a plurality of natural languages ​​obtained from a plurality of the second feedback information.

15. An information processing device according to any one of claims 1 to 9, wherein the second prompt includes the acquired information and information specifying an output format of the second output result.

16. An information processing apparatus according to claim 15, wherein the second prompt specifies a markup language as an output format of the second output result.

17. An information processing device according to any one of claims 1 to 9, wherein the first feedback information includes evaluation information for the first output result.

18. The information processing device according to claim 17, further comprising: a radio button or slider for receiving a user's specification of evaluation information for the first output result.

19. An information processing device according to any one of claims 1 to 9, wherein the at least one processor generates the first feedback information by analyzing the first output result.

20. An information processing device as described in any one of claims 1 to 9, wherein the at least one processor generates a plurality of the first prompts, obtains a plurality of the first output results generated by inputting the plurality of the first prompts into a language model, and obtains the first feedback information of the user regarding the first output result selected by the user from the plurality of the first output results.

21. An information processing system including at least an information processing device according to any one of claims 1 to 9.

22. An information processing method, comprising: at least one processor included in an information processing device: generating a first prompt; obtaining a first output result generated by inputting the first prompt into a language model; obtaining first feedback information for the first output result; generating a second prompt using information obtained by executing an optimization algorithm based on the first feedback information; and obtaining a second output result generated by inputting the second prompt into the language model.

23. A program for causing at least one processor provided in an information processing device to execute the following steps: generating a first prompt; obtaining a first output result generated by inputting the first prompt into a language model; obtaining first feedback information for the first output result; generating a second prompt using information obtained by executing an optimization algorithm based on the first feedback information; and obtaining a second output result generated by inputting the second prompt into the language model.

Images