Information processing device, information processing method, and information processing program

Abstract

To more accurately generate a prompt for obtaining more appropriate completion from a natural language model.SOLUTION: An information processing device includes a processing device which executes a prompt engineering process for generating a prompt corresponding to input regarding a business from a user by using business information on the basis of a prompt engineering plan being a rule for generating a prompt; and a prompt engineering optimization process for acquiring output data by inputting the generated prompt to a natural language model which uses the business information as an input value and a proposal regarding the business as an output value, generating a prompt engineering database in which the acquired output data and business achievement information are associated with each other, determining similarity between the output data and the achievement information by referring to the generated prompt engineering database, and updating the prompt engineering plan on the basis of the determination result of the similarity.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present invention relates to an information processing apparatus, an information processing method, and an information processing program.

Background Art

[0002] Patent Document 1 discloses a technique for generating a sentence to be input as a prompt to a large language model (LLM) by adding reference information to an input question sentence.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, the technique described in Patent Document 1 has a problem that if the tuning of the reference information is not properly performed, appropriate results are not output from the large language model.

[0005] The present invention has been made in view of such circumstances, and an object thereof is to provide an information processing system, an information processing method, and an information processing program capable of generating a prompt for obtaining a more appropriate completion (answer sentence) from a natural language model with higher accuracy.

Means for Solving the Problems

[0006] One aspect of the present invention for solving the above problems is a storage device that stores a business database storing a plurality of pieces of business information, and a prompt engineering process that generates a prompt corresponding to an input from a user related to a business using the business information based on a prompt engineering plan that is a rule for generating a prompt, and obtaining output data by inputting the generated prompt into a natural language model that uses business information as an input value and outputs a proposal related to the business as an output value, generating a prompt engineering database associating the obtained output data, the generated prompt, and the performance information of the business, determining the similarity between the output data and the performance information with reference to the generated prompt engineering database, and a processing device that executes a prompt engineering optimization process for updating the prompt engineering plan based on the determination result of the similarity.

Advantages of the Invention

[0007] According to the present invention, a prompt for obtaining a more appropriate completion from a natural language model can be generated with higher accuracy. Configurations, effects, etc. other than those described above will be clarified by the description of the following embodiments.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Mode for Carrying Out the Invention

[0009] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. The following description and drawings are examples for explaining the present invention, and for the sake of clarity of explanation, appropriate omissions and simplifications have been made. The present invention can be implemented in various other forms. Unless otherwise particularly limited, each component may be singular or plural. The positions, sizes, shapes, ranges, etc. of the respective components shown in the drawings may not represent the actual positions, sizes, shapes, ranges, etc. in order to facilitate understanding of the invention. For this reason, the present invention is not necessarily limited to the positions, sizes, shapes, ranges, etc. disclosed in the drawings. In the following description, various information may be described using expressions such as "table", "list", "queue", etc., but the various information may be represented by data structures other than these. In order to indicate that it does not depend on the data structure, "XX table", "XX list", etc. may be referred to as "XX information". When explaining identification information, expressions such as "identification information", "identifier", "name", "ID", "number", etc. are used, and these can be replaced with each other. When there are a plurality of components having the same or similar functions, they may be described by attaching different subscripts to the same reference numeral. However, when it is not necessary to distinguish these plurality of components, the subscripts may be omitted in the description. Also, in the following description, the processing performed by executing a program may be described. However, the program is executed by a processor (for example, a CPU, a GPU), and the defined processing is performed while appropriately using a storage resource (for example, a memory) and / or an interface device (for example, a communication port), etc., so the main body of the processing may be the processor. Similarly, the main body of the processing performed by executing the program may be a controller, a device, a system, a computer, or a node having a processor. The main body of the processing performed by executing the program only needs to be an arithmetic unit, and may include a dedicated circuit (for example, an FPGA or an ASIC) that performs a specific process. The program may be installed from a program source into a device such as a computer. The program source may be, for example, a program distribution server or a storage medium readable by a computer. When the program source is a program distribution server, the program distribution server includes a processor and a storage resource for storing the program to be distributed, and the processor of the program distribution server may distribute the program to be distributed to other computers. Also, in the following description, two or more programs may be realized as one program, or one program may be realized as two or more programs.

[0010] <Device Configuration> FIG. 1 is a diagram showing a configuration example of the information processing system 1 in the present embodiment. The information processing system 1 includes an information processing device 10, a user terminal 20, and an administrator terminal 30. The information processing device 10 is an information processing device that automatically generates an appropriate prompt (input sentence) to be input to a natural language model that makes a proposal regarding a business. The natural language model that makes a proposal regarding a business is, for example, a large language model (Generative Artificial Intelligence) that takes business information as an input value and outputs a proposal regarding a business (for example, business measures). The user terminal 20 is an information processing device related to a user (for example, a person in charge of inquiring about a proposal regarding a business), such as a personal computer, a tablet terminal, or a smartphone. The administrator terminal 30 is an information processing device related to an administrator of the information processing device 10, for example.

[0011] In the present embodiment, a large language model that supports sales activities for customers will be described as an example. Sales results often vary from person to person. Therefore, there is a desire to improve the contract rate or cancellation rate, etc., without relying on personal experience or intuition through the digitization of sales knowledge. Although the use of daily sales reports, etc., as sales knowledge can be considered, daily sales reports, etc., are often written in unstructured natural language, and there are also items that are not described by some salespersons, making it difficult to use for data analysis. In addition, in data utilization that only uses past sales performance, there is a risk of obsolescence or convergence, and it is difficult to generate new ideas. Therefore, it is conceivable to utilize a large language model to provide creative (reliable and flexible) proposals considering customer characteristics or the progress of sales, etc.

[0012] The strength of a large language model is that it can use natural language for both input and output. However, the output results of a large language model may vary significantly even with a slight difference in the input (prompt). Therefore, prompt engineering for generating an appropriate prompt to reduce hallucination (false information) by a large language model is an important technique for bringing out the performance of the large language model.

[0013] Generally, large language models are costly to use and have limitations such as character count limits for prompts. Therefore, constraints such as character count limits, cost-effectiveness, or cost reduction are imposed on the use of large language models. Under such constraints, it takes time and effort to manually analyze the appropriate amount of reference information (e.g., number of characters, number of reference information, etc.) to insert into the prompt, or the items to be emphasized (e.g., whether it is better to have more success cases or more failure cases, etc.). Also, it is difficult to accurately extract the appropriate reference information to insert into the prompt from business information. Therefore, the information processing apparatus 10 in the present embodiment solves such problems and automatically generates an appropriate prompt.

[0014] As shown in FIG. 1, as an example, the information processing apparatus 10 includes a processing apparatus 11 such as a CPU (Central Processing Unit), a memory 12 such as a RAM (Random Access Memory) and a ROM (Read Only Memory), a storage device 13 such as an HDD (Hard Disk Drive) and an SSD (Solid State Drive), an input device 14 such as a keyboard, a mouse, or a touch panel, an output device 15 such as a display or a printer, and a communication device 16 composed of a NIC (Network Interface Card), a wireless communication module, a USB (Universal Serial Interface) module, or a serial communication module, etc. The processing apparatus 11, the memory 12, the storage device 13, the input device 14, the output device 15, and the communication device 16 are interconnected via a bus.

[0015] The storage device 13 stores a program 100 for realizing each function required for the information processing apparatus 10, a business database 200, a vector database 300, and a prompt engineering database 400.

[0016] The business database 200 is a database in which business information (e.g., daily business reports) related to business (e.g., sales activities), customer information related to each customer, and history information representing the history of sales activities for a specific customer are accumulated. Customer IDs for identifying each customer are pre-assigned to the business information, customer information, and history information stored in the business database 200.

[0017] The vector database 300 is a database in which a plurality of vector processing results obtained by vector-converting each of a plurality of business information are stored. The prompt engineering database 400 is a database that stores, in association with each other, the completions obtained by inputting the generated prompts into a large language model and the performance information of the business.

[0018] The program 100 includes programs that implement each of the functions of the execution plan management unit 101, the vector processing management unit 102, the vector processing optimization unit 103, the prompt engineering management unit 104, the template management unit 105, the prompt engineering optimization unit 106, and the display control unit 107.

[0019] The execution plan management unit 101 manages a vector processing plan, which is a rule for summarizing and vector-converting business information, a vector processing optimization plan, which is a rule for optimizing the vector processing plan, a prompt engineering plan, which is a rule for generating a prompt, and a prompt engineering optimization plan, which is a rule for optimizing the prompt engineering plan. The execution plan management unit 101 may store the vector processing plan, the vector processing optimization plan, the prompt engineering plan, or the prompt engineering optimization plan, or may read from or write to a predetermined database.

[0020] The vector processing management unit 102 reads business information from the business database 200, generates a vector processing result obtained by vector-converting the read business information based on the vector processing plan, and writes the generated vector processing result into the vector database 300.

[0021] The vector processing optimization unit 103 operates (creates, updates, or deletes) the vector processing plan based on the vector database 300 and the vector optimization plan.

[0022] The prompt engineering management unit 104 receives an input of an input sentence related to business (for example, an inquiry about the optimal sales method (NBA: Next Best Action) for customers) from the user via a GUI (Graphical User Interface), and generates a prompt to be input to the large language model based on the input sentence, a prompt template described later, and the prompt engineering plan. The large language model may be connected so as to be able to access (read) the business database 200, or may be a pre-trained model that has been pre-trained using various data stored in the business database 200 as learning data. The prompt engineering management unit 104 inputs the prompt to the large language model and obtains a completion (for example, NBA) that is the output result thereof. Then, the prompt engineering management unit 104 associates the generated prompt with the obtained completion and writes it into the prompt engineering database 400 as the prompt engineering result. Further, the obtained completion is transmitted to the user terminal 20 to notify the user.

[0023] The template management unit 105 manages a prompt template described later. The template management unit 105 may store the prompt template, or may read from or write to a predetermined database.

[0024] Based on the prompt engineering database 400 and the prompt engineering optimization plan, the prompt engineering optimization unit 106 operates (creates, updates, or deletes) the prompt engineering plan.

[0025] The display control unit 107 (output unit) controls the GUI to be displayed on the user terminal 20 or the administrator terminal 30 via the communication device 16.

[0026] The information processing device 10 realizes each function of the execution plan management unit 101, the vector processing management unit 102, the vector processing optimization unit 103, the prompt engineering management unit 104, the template management unit 105, the prompt engineering optimization unit 106, and the display control unit 107 by the processing device 11 reading out and executing the program 100 stored in the storage device 13 into the memory 12. The program 100 that realizes each function of the execution plan management unit 101, the vector processing management unit 102, the vector processing optimization unit 103, the prompt engineering management unit 104, the template management unit 105, the prompt engineering optimization unit 106, and the display control unit 107 can be recorded and distributed on, for example, a portable or fixed recording medium.

[0027] In addition to the configuration example in which each function constituting the information processing device 10 is arranged in a single information processing device, the functions may be distributed and arranged in a plurality of information processing devices connected via a network or computing resources on the cloud. Next, the processing performed by the information processing device 10 will be described.

[0028] <Overview of Processing> FIG. 2 is a diagram for explaining the overview of the processing performed by the information processing device 10. The information processing device 10 repeatedly executes a first process (S1 to S5) for constructing a vector database 300 used for prompt engineering and a second process (S11 to S25) for executing prompt engineering.

[0029] First, the first process shown in S1 to S5 will be described. The first process is executed, for example, when a new vector processing plan is input to the execution plan management unit 101, or when the vector processing plan managed by the execution plan management unit 101 is updated.

[0030] In the first process, first, the execution plan management unit 101 outputs the input or updated vector processing plan to the vector processing management unit 102 (S1).

[0031] Subsequently, the vector processing management unit 102 reads and acquires the business information specified by the vector processing plan from the business database 200 (S2).

[0032] Subsequently, the vector processing management unit 102 executes vector processing to convert the acquired business information into a vector based on the vector processing plan. Details of the processing executed by the vector processing management unit 102 will be described later. Then, the vector processing management unit 102 writes and stores the vector processing result and vector processing summary (specific examples will be described later), which are the output information of the vector processing, in the vector database 300 (S3).

[0033] On the other hand, the vector processing optimization unit 103 reads the vector processing result and vector processing summary from the vector database 300 (S4). Then, the vector processing optimization unit 103 optimizes (operates) the vector processing plan based on the read vector processing result and vector processing summary and the vector processing optimization plan. Details of the processing of the vector processing optimization unit 103 will be described later.

[0034] Subsequently, the second process shown in S11 to S25 will be described. First, the user U (for example, a salesperson) inquires about "the optimal sales method for customers (NBA: Next Best Action)" via the GUI displayed on the user terminal 20 (S11). The user terminal 20 transmits the inquiry from the user U input via the GUI to the prompt engineering management unit 104 (S12).

[0035] The Prompt Engineering Management Department 104 receives an inquiry from the user U, and obtains a prompt engineering plan from the Execution Plan Management Department 101 based on the received inquiry (S13). Subsequently, the Prompt Engineering Management Department 104 obtains a template from the Template Management Department 105 based on the obtained prompt engineering plan (S14). Also, the Prompt Engineering Management Department 104 receives an input related to the business (e.g., customer ID) from the user based on the obtained prompt engineering plan, and obtains business information (information based on the input from the user related to the business) corresponding to the input (customer ID) from the business database 200 (S15). Also, the Prompt Engineering Management Department 104 obtains a vector processing result from the vector database 300 based on the obtained prompt engineering plan (S16). Then, the Prompt Engineering Management Department 104 generates a prompt based on the obtained prompt engineering plan, template, business information, and vector processing result. Details of the processing of the Prompt Engineering Management Department 104 will be described later.

[0036] The Prompt Engineering Management Department 104 inputs the generated prompt into the large language model to obtain its completion (NBA result). Then, the Prompt Engineering Management Department 104 writes and stores the prompt engineering processing result, which is the processing result of the prompt engineering processing, in the Prompt Engineering Database 400 (S17). The prompt engineering processing result includes the generated prompt and the completion (NBA result). Also, the Prompt Engineering Management Department 104 assigns a unique ID unique to the obtained NBA result, and transmits the NBA result with the unique ID assigned to the user terminal 20 (S18). The user terminal 20 receives the NBA result and outputs the received NBA result to the user U via the GUI.

[0037] User U conducts sales activities for customer C with reference to the NBA results (S19). Then, user U inputs sales activity information (business performance information) representing the results of the actual sales activities into user terminal 20. The unique ID of the NBS results is assigned to the sales activity information. User terminal 20 receives the input of the sales activity information from user U via the GUI, and transmits the sales activity information input from user U to information processing device 10 (S20). Display control unit 107 receives the sales activity information from user terminal 20, and writes and stores the received sales activity information in business database 200 (S21).

[0038] On the other hand, prompt engineering optimization unit 106 acquires the prompt engineering optimization plan managed by execution plan management unit 101 (S22). Subsequently, prompt engineering optimization unit 106 acquires the sales activity information from business database 200 based on the acquired prompt engineering optimization plan (S23). Subsequently, prompt engineering optimization unit 106 associates the acquired sales activity information with the NBA results based on the unique ID and stores it in prompt engineering database 400 (S24). Subsequently, prompt engineering optimization unit 106 evaluates the prompt engineering processing plan based on the similarity between the sales activity information and the NBA results (i.e., the contribution or usefulness of the NBA results). Then, prompt engineering optimization unit 106 optimizes the prompt engineering plan based on the evaluation result. Furthermore, prompt engineering optimization unit 106 instructs vector processing optimization unit 103 to optimize the vector processing plan based on the evaluation result (S25). The details of the processing of prompt engineering optimization unit 106 will be described later.

[0039] Subsequently, the details of the processing executed by each functional unit of information processing device 10 in the above-described first processing and second processing will be described.

[0040] <Processing of Vector Processing Management Unit 102> FIG. 3 is a flowchart for explaining the details of the processing executed by the vector processing management unit 102. The processing shown in this figure is executed, for example, when a new vector processing plan is input to the execution plan management unit 101, or when the vector processing plan managed by the execution plan management unit 101 is updated.

[0041] First, the vector processing management unit 102 receives the input or update of the vector processing plan (S101).

[0042] (Vector processing plan) FIG. 4 is a diagram showing an example of the vector processing plan 110. In the vector processing plan 110, for example, identification information of the vector processing plan (in this example, "vector_01"), execution conditions 111 of the vector processing (for example, execution trigger, conditions of cost or time required for processing (for example, upper limit value or lower limit value), conditions of evaluation results of similarity (for example, upper limit value or lower limit value), etc.), input information 112 of the vector processing (for example, business information acquired from the business database 200), summary method 113 of the input information (for example, model, parameters, number of summaries, granularity, number of characters, etc.), vector conversion method 114 (for example, model or parameters), evaluation method 115 of vector similarity (model, parameters, or threshold value (for example, lower limit value, etc.)), output destination 116 of the processing result (for example, storage destination in the vector database 300), and set values 117 of various parameters, etc. are set. In the illustrated example, in the vector processing plan 110, parameters $1 and $2 related to summarization and parameter $3 related to evaluation of similarity are set. For example, parameter $1 represents the granularity of the summary. Also, parameter $2 represents the number of characters after summarization. Also, parameter $3 represents the model for evaluating similarity. In addition to these, as parameters, a model for vector conversion, etc. may be set in the vector processing plan 110.

[0043] Subsequently, the vector processing management unit 102 performs the construction, update, or deletion of vector processing (S102). Specifically, the vector processing management unit 102 executes the following processes S1021 to S1027 for each piece of business information specified in the acquired vector processing plan 110.

[0044] First, the vector processing management unit 102 extracts the business information specified in the acquired vector processing plan from the business database 200 based on the execution trigger specified in the acquired vector processing plan (S1021). The execution trigger is, for example, on demand or when the business database 200 is updated.

[0045] Subsequently, the vector processing management unit 102 summarizes the text related to the acquired business information using the method (such as a large language model or parameters) specified in the vector processing plan (S1022). Hereinafter, the information of the text obtained by summarizing the text related to the business information may be referred to as "summarized business information".

[0046] Subsequently, the vector processing management unit 102 vector-converts the business information before summarization and the summarized business information respectively using the method (such as a large language model or parameters) specified in the vector processing plan. Then, the vector processing management unit 102 evaluates the converted vectors using the method (such as cosine similarity) specified in the vector processing plan (S1023).

[0047] Subsequently, the vector processing management unit 102 determines whether the evaluation result of S1023 satisfies the specified conditions (such as a lower limit value) specified in the vector processing plan (S1024). If the evaluation result satisfies the specified conditions (S1024: Yes), the vector processing management unit 102 proceeds to the process of S1027.

[0048] On the other hand, when the evaluation result does not meet the specified conditions (S1024: No), the vector processing management unit 102 determines whether the execution of the vector processing is within the range of the execution conditions (for example, the upper limit value of time or cost, etc.) specified in the vector processing plan (S1025). When the execution of the vector processing is within the range of the execution conditions (S1025: Yes), the vector processing management unit 102 returns to the process of S1022.

[0049] On the other hand, when the execution of the vector processing is not within the range of the execution conditions (S1025: No), the vector processing management unit 102 notifies the administrator terminal 30 of the failure of the vector processing (S1026). For example, the vector processing management unit 102 notifies the failure of the vector processing and transmits data (a notification prompting the update of the vector processing plan) to the administrator terminal 30 that proposes to administrator A to review the vector processing plan. Thereby, administrator A can consider reviewing the vector processing plan. When administrator A updates the vector processing plan via the administrator terminal 30, the vector processing management unit 102 re-executes the processes in order from the process of S101.

[0050] Subsequently, the vector processing management unit 102 stores the vector processing result and the vector processing summary, which are the output information of the vector processing, in the vector database 300 specified in the vector processing plan (S1027), and then ends the process.

[0051] (Vector processing result) FIG. 5 is a diagram showing an example of the vector processing result 310. The vector processing result 310 is data in, for example, a table format having each data item of No 311, Vector Processing Plan 312, Original Data ID 313, Parameters 314, Summary Data 315, Vector Data 316, Score 317, Dollar 318, Second 319, and Status 320. In No 311, identification information of the generated vector is set. In Vector Processing Plan 312, identification information of the vector processing plan is set. In Original Data ID 313, identification information of the original (before summarization) business information (hereinafter referred to as "original information") is stored. In Parameters 314, the values of the parameters set in the vector processing plan 110 are set. In Summary Data 315, summarized business information is set. In Vector Data 316, a vector is set. In Score 317, an evaluation result of similarity is set. In Dollar 318, the cost (expenses) required for vector processing is set. In Second 319, the processing time required for vector processing is set. For Status 320, "Original" is set for the original information, "Accept" is set for the summarized business information with the highest value of the evaluation result (Score 317), and "Reject" is set for other summarized business information. Here, by including the Original Data ID 313 in the vector processing result, the original information of the business information and the summarized business information can be specified, and the operation function of data lineage is provided.

[0052] (Vector Processing Summary) FIG. 6 is a diagram showing an example of the vector processing summary 330. The vector processing summary 330 is data in, for example, a table format that represents the statistical result (aggregation result) of the vector processing result 310. The vector processing summary 330 has data items of Vector Processing Plan 331, Total 332, and Top Score 333. Identification information of the vector processing plan is set in the Vector Processing Plan 331.

[0053] Statistical information of all summary service information (all summary service information where Status 320 is "Accept" or "Reject") is set in Total 332. Total 332 has data items of n, which is the number of summary service information to be generated for one piece of service information, counts, which is the total number of generated summary service information, avg, which is the average value of the evaluation result (Score 317), sigma, which is the standard deviation of the evaluation result (Score 317), dollar / n, which is the cost per one piece of summary service information (for example, the average value of Dollar 318), and sec / n, which is the processing time per one piece of summary service information (for example, the average value of Second 319).

[0054] Statistical information of the summary service information with the highest evaluation result (only the summary service information where Status 320 is "Accept") among the summary service information generated from one piece of service information is set in Top Score 333. Top Score 333 has data items of counts, which is the total number of generated summary service information, avg, which is the average value of the evaluation result (Score 317), and sigma, which is the standard deviation of the evaluation result (Score 317).

[0055] Note that the display control unit 107 may refer to the vector database 300 and display a GUI representing the content of the vector processing summary 330 on the administrator terminal 30.

[0056] (Vector Processing Summary Display Screen) FIG. 7 is a diagram showing an example of a vector processing summary display screen 800. The vertical axis of the graph shown in this figure indicates the probability density of the evaluation result, and the horizontal axis indicates the probability variable of the evaluation result. In this figure, the Total and Top Score of the vector processing plan "vector_02", and the Total of the vector processing plan "vector_02" are drawn in the graph.

[0057] <Processing of the vector processing optimization unit 103> FIG. 8 is a process flow diagram for explaining the details of the process (vector processing optimization process) executed by the vector processing optimization unit 103. The process shown in this figure is executed, for example, at a predetermined execution timing (for example, every month, etc.), or when an input for instructing optimization of the vector processing plan is received from the administrator A or the prompt engineering optimization unit 106.

[0058] First, the vector processing optimization unit 103 acquires a vector processing optimization plan from the execution plan management unit 101.

[0059] (Vector processing optimization plan) FIG. 9 is a diagram showing an example of a vector processing optimization plan 120. In the vector processing optimization plan 120, for example, a target specified vector processing plan 121, an execution timing 122, an ideal state 123, and an action 124 (operations such as creation, update, or deletion) for the vector processing plan are set.

[0060] Then, the vector processing optimization unit 103 starts optimizing the specified vector processing plan at the execution timing specified by the acquired vector processing optimization plan 120 (S201). In the example shown in FIG. 9, the vector processing optimization unit 103 executes the optimization of the specified vector processing plan every month (for example, on a specified day once a month).

[0061] Subsequently, the vector processing optimization unit 103 acquires the vector processing result and vector processing summary of the specified vector processing plan from the vector database 300, and evaluates whether the specified vector processing plan satisfies the ideal state (e.g., criteria such as score, cost, etc.) specified in the vector processing optimization plan (S202). In the example shown in FIG. 9, as the ideal state, the lower limit value of the avg of Top Score is set to "0.90", the upper limit value of dollar / n is set to "1.6", and the upper limit value of sec / n is set to "40.0". The ideal state specified in the vector processing optimization plan corresponds to the constraints of vector processing. If the specified vector processing plan is in the ideal state, the vector processing optimization 103 ends the process.

[0062] On the other hand, if the specified vector processing plan is not in the ideal state, the vector processing optimization unit 103 operates on the specified vector processing plan based on the actions (creation, update, or deletion) described in the vector processing optimization plan (S203). In the example shown in FIG. 9, when the avg of Top Score in the vector processing summary is 0.80 or less, the vector processing optimization unit 103 deletes the specified vector processing plan. Also, for example, when the vector processing optimization unit 103 updates the specified vector processing plan, it may change the values of each parameter set in the specified vector processing plan (e.g., summary granularity, number of characters after summarization, summarization method (model or parameters, etc.), or vector conversion method (model or parameters, etc.)).

[0063] Subsequently, the vector processing optimization unit 103 stores the operated (created or updated) specified vector processing plan in the execution plan management unit 101 (S204) and ends the process.

[0064] <Processing of the Prompt Engineering Management Unit 104> FIG. 10 is a processing flowchart for explaining the details of the processing executed by the prompt engineering management unit 104. The processing shown in this figure is executed, for example, when an inquiry regarding NBA (business) is received from the user U via the GUI displayed on the user terminal 20.

[0065] First, the Prompt Engineering Management Unit 104 receives an input or update of a prompt engineering plan in the Execution Plan Management Unit 101 (S301).

[0066] (Prompt Engineering Plan) FIG. 11 is a diagram showing an example of a prompt engineering plan 130. In the prompt engineering plan 130, for example, identification information of the prompt engineering plan, execution conditions 131 of vector processing (e.g., execution timing, cost or time conditions for processing (e.g., upper limit value or lower limit value), etc.), requested requirement information 132 (e.g., a prompt template, or data in a specified format for receiving input from a user), information 133 to be obtained from the business database 200 (e.g., business information, customer information, or history information, etc.), a specified vector processing plan 134 to be used in vector search, a method 135 for substituting reference information into each item of the prompt template, a generation AI 136 for inputting the generated prompt, an output destination 137 of the processing result (e.g., a storage destination in the prompt engineering database 400), parameters 138, etc. are set.

[0067] In the illustrated example, "Best_Plan" (for example, a vector processing plan with the highest statistical value of evaluation results, etc.) is set in the designated vector processing plan 134, but it is not limited to this, and any vector processing plan (for example, "vector_01", etc.) can also be set. Also, in this figure, the substitution method 135 of reference information into the item "SUCCESSFUL" of the prompt is illustrated, but the substitution method 135 may be specified in the prompt engineering plan for each item of the prompt. In the substitution method 135, for example, the type or quantity (for example, the number of characters, or the number of reference information, etc.) of the reference information (business information, summary business information, or history information, etc.) to be substituted (inserted) into each item of the prompt is parameter - set. Also, in the illustrated example, as the parameter 138, the number of characters $1 of the reference information to be substituted into the item "SUCCESSFUL" of the prompt is set. Note that in the prompt engineering plan 130, the number of characters of the reference information to be substituted into each item of the prompt may be set as parameters respectively. That is, the number of characters of each item of the prompt may be set to different values.

[0068] Subsequently, the prompt engineering management unit 104 performs the construction, update, or deletion of the prompt engineering process (S302).

[0069] Specifically, first, the prompt engineering management unit 104 acquires the prompt template specified in the acquired prompt engineering plan from the template management unit 105 and accepts data input in the specified format from the user U (S3021). In the example shown in FIG. 11, the prompt engineering management unit 104 acquires the prompt template "prompt_template_001". Also, the prompt engineering management unit 104 accepts the input of the customer_id (customer ID) in string format from the user U by displaying a GUI on the user terminal 20.

[0070] (Prompt template) FIG. 12 is a diagram showing an example of the prompt template 150. As shown, the prompt template 150 has one or more items. In the illustrated example, the prompt template 150 has items "SALES (e.g., characteristics of salespersons)" 151, "SUCCESSFUL (e.g., successful cases)" 152, and "FAILED (e.g., failed cases)" 153.

[0071] Subsequently, the prompt engineering management unit 104 acquires the information specified in the acquired prompt engineering plan from the business database 200 or the user U (salesperson) (S3022). For example, the prompt engineering management unit 104 acquires business information or history information of the customer ID for which an input has been received from the user U from the business database 200. The business information or history information of the customer ID for which an input has been received from the user U corresponds to "information based on the input from the user U regarding the business".

[0072] Subsequently, the prompt engineering management unit 104 summarizes and vector-converts the information acquired in S3022 using the vector processing plan specified in the prompt engineering plan. Then, the prompt engineering management unit 104 performs a vector search on the vector database 300 using the converted vector and extracts the vector processing result related to the information (S3023). For example, the prompt engineering management unit 104 searches for a vector processing result similar to the converted vector. At this time, the prompt engineering management unit 104 searches for the vector processing result of the summary business information (Status 320 is "Accept") with the highest similarity to the vector processing result of the original information.

[0073] Subsequently, the prompt engineering management unit 104 extracts the summary business information from the extracted vector processing result, specifies the original information or history information using the Original Data ID of the vector processing result, and acquires the specified original information or history information from the business database 200 (S3024).

[0074] Subsequently, the Prompt Engineering Management Department 104 substitutes (inserts) the specified reference information (such as summary business information, original information, or history information) into each item of the prompt template according to the substitution method specified in the obtained Prompt Engineering Plan (S3025).

[0075] Subsequently, the Prompt Engineering Management Department 104 removes (deletes) unused template items (items of the prompt template for which the substitution of reference information is not specified in the Prompt Engineering Plan) from the prompt (S3026).

[0076] (Prompt) FIG. 13 is a diagram showing an example of the prompt 160. As shown, in the prompt 160, reference information is substituted into each item of the prompt template. In the illustrated example, a successful case is described in the item "SUCCESSFUL", and for example, a failed case is described in the item "FAILED". Also, the Prompt Engineering Management Department 104 has deleted the unused item "SALES" from the prompt template.

[0077] Subsequently, the Prompt Engineering Management Department 104 inputs the generated prompt into the generation AI specified in the Prompt Engineering Plan to obtain a completion (result) from the generation AI (S3027).

[0078] Subsequently, the Prompt Engineering Management Department 104 assigns a unique ID to the obtained completion and responds to the user U with the obtained completion and the assigned unique ID (S3028). Specifically, the Prompt Engineering Management Department 104 causes a GUI that displays the obtained completion and the assigned unique ID to be displayed on the user terminal 20.

[0079] Subsequently, the Prompt Engineering Management Department 104 stores the prompt engineering processing result, which is the result of the prompt engineering process, in the Prompt Engineering Database 400 (S3029) and ends the process.

[0080] (Prompt engineering processing result) FIG. 14 is a diagram showing an example of the prompt engineering processing result 410. The prompt engineering processing result 410 is data in, for example, a table format having each data item of No 411, Prompt Engineering Plan 412, Unique ID 413, Input 414, Vector Processing Plan 415, Process 416, Original Data ID 417, Similar Score 418, Prompt 419, Completion 420, Actual Data 421, and Final Score 422.

[0081] In No 411, the identification information of the generated prompt is set. In Prompt Engineering Plan 412, the identification information of the prompt engineering plan is set. In Unique ID 413, the unique ID assigned to the prompt is set. In Input 414, the information input from user U in S3021 or S3022 (in the illustrated example, customer ID [C001]) is set. In Vector Processing Plan 415, the identification information of the vector processing plan used for vector conversion is set. In Process 416, the items of the template are set. In Original Data ID 417, the Original Data ID of the reference information substituted into the items of the template is stored. In Similar Score 418, the similarity between the customer set in Input 414 and the customer of Original Data ID 417 is set. In Prompt 419, the generated prompt is set. In Completion 420, the completion of the generation AI for Prompt 419 is set. The details of the data set in Actual Data 421 and Final Score 422 will be described later. Note that by including Original Data ID 417 in the prompt engineering processing result, the original information substituted as reference information in the prompt can be specified, and the operation function of data lineage is provided.

[0082] <Processing of Prompt Engineering Optimization Unit 106> FIG. 15 is a processing flow diagram for explaining the details of the processing (prompt engineering optimization processing) executed by the prompt engineering optimization unit 106. The processing shown in this figure is executed, for example, at a predetermined execution timing (e.g., every month, etc.), or when an input for instructing the optimization of the prompt engineering plan is received from administrator A.

[0083] First, the prompt engineering optimization unit 106 acquires the prompt engineering optimization plan 140 from the execution plan management unit 101.

[0084] (Prompt Engineering Optimization Plan) FIG. 16 is a diagram showing an example of a prompt engineering optimization plan 140. In the prompt engineering optimization plan 140, for example, a target specified prompt engineering plan 141, a trigger 142, an evaluation method 143 of the prompt engineering plan (for example, an evaluation method or a model, etc.), an output destination 144, an ideal state 145, and an action 146 (operations such as creation, update, or deletion) for the prompt engineering plan or the vector processing plan are set.

[0085] Then, the prompt engineering optimization unit 106 starts optimizing the specified prompt engineering plan at the trigger specified in the prompt engineering optimization plan 140 (S401). In the example shown in FIG. 16, the prompt engineering optimization unit 106 executes the optimization of the specified prompt engineering plan every month (for example, on a specified day once a month).

[0086] Subsequently, the prompt engineering optimization unit 106 extracts the unique ID and the performance value (the actual business content) of the business activity information newly added by the user U from the business database 200. Then, the prompt engineering optimization unit 106 registers the extracted performance value in the Actual Data 421 of the prompt engineering processing result corresponding to the extracted unique ID (S402).

[0087] Subsequently, the Prompt Engineering Optimization Unit 106 calculates a score (Final Score), such as the similarity or contribution degree between the proposal (Completion) and the actual value (Actual Data), based on the evaluation method specified in the Prompt Engineering Optimization Plan, and registers the calculated score in the Final Score 422 of the Prompt Engineering processing result. Then, the Prompt Engineering Optimization Unit 106 generates a Prompt Engineering summary by aggregating the Final Score 422 of the Prompt Engineering processing result, and writes and registers the generated Prompt Engineering summary in the Prompt Engineering database 400 (S403).

[0088] (Prompt Engineering summary) FIG. 17 is a diagram showing an example of the Prompt Engineering summary 430. The Prompt Engineering summary 430 is data in, for example, a table format representing the statistical result (aggregation result) of the Prompt Engineering processing result 410. The statistical result of the Prompt Engineering processing result 410 corresponds to the determination result of the similarity between the completion and the actual information. The Prompt Engineering summary 430 has data items of a Prompt Engineering Plan 431 and a Final Score 432. The Prompt Engineering Plan 431 is set with the identification information of the Prompt Engineering plan. The Final Score 432 is set with the statistical information of the Final Score 422 of the Prompt Engineering processing result. The Final Score 432 has data items of counts where the total number of generated prompts is set, avg where the average value of the score (Final Score 422) is set, and sigma where the standard deviation of the score (Final Score 422) is set.

[0089] Subsequently, the Prompt Engineering Optimization Unit 106 evaluates whether the aggregated result (Prompt Engineering Summary) meets the ideal state (e.g., criteria such as score, cost, etc.) specified in the Prompt Engineering Optimization Plan (S404). In the example shown in FIG. 16, as the ideal state, the lower limit value of the avg of Final Score, "0.90", is set. The ideal state specified in the Prompt Engineering Optimization Plan corresponds to the constraints of the Prompt Engineering process. If the specified Prompt Engineering Plan is in the ideal state, the Prompt Engineering Optimization Unit 106 ends the process.

[0090] On the other hand, if the specified Prompt Engineering Plan is not in the ideal state, the Prompt Engineering Optimization Unit 106 operates the specified Prompt Engineering Plan based on the actions (creation, update, or deletion) described in the Prompt Engineering Optimization Plan (S405). In the example shown in FIG. 16, the Vector Processing Optimization Unit 103 readjusts the values of the parameters. For example, the Vector Processing Optimization Unit 103 may adjust the number of characters of the reference information substituted for each item of the prompt, such as increasing the number of characters of the item "SUCCESSFUL" in the prompt and decreasing the number of characters of the item "FAILED". Also, the Vector Processing Optimization Unit 103 may change the type of reference information (e.g., business information, history information, or summary business information) substituted for each item.

[0091] Subsequently, the Prompt Engineering Optimization Unit 106 stores the operated specified Prompt Engineering Plan in the Execution Plan Management Unit 101 (S406).

[0092] Subsequently, the Prompt Engineering Optimization Unit 106 extracts the vector processing plan (the vector processing plan specified in the specified Prompt Engineering Plan) involved in the operated specified Prompt Engineering Plan, instructs the Vector Processing Optimization Unit 103 to optimize the vector processing plan (S407), and ends the process.

[0093] Note that the display control unit 107 may cause the GUI representing the content of the generated prompt engineering summary 430 to be displayed on the administrator terminal 30.

[0094] (Prompt Engineering Summary Display Screen) FIG. 18 is a diagram showing an example of a prompt engineering summary display screen 900. The vertical axis of the graph shown in this figure indicates the probability density of the Final Score, and the horizontal axis indicates the random variable of the Final Score. In this figure, the statistical results of the prompt engineering plans "prompt_001", "prompt_002", and "prompt_003" are drawn in the graph.

[0095] As described above, the information processing apparatus 10 according to the present embodiment stores the business database 200 in which a plurality of business information is stored, and based on a prompt engineering plan that is a rule for generating a prompt, a prompt corresponding to an input from a user related to the business is generated using the business information. In the prompt engineering process, the generated prompt is input to a natural language model that uses the business information as an input value and outputs a proposal related to the business as an output value to obtain output data (completion), and a prompt engineering database 400 in which the obtained output data, the generated prompt, and the performance information of the business are associated is generated. The prompt engineering optimization process of referring to the generated prompt engineering database 400 to determine the similarity between the output data and the performance information and updating the prompt engineering plan based on the determination result of the similarity is executed.

[0096] That is, the information processing apparatus 10 of the present embodiment regards the similarity between completion and performance information as the degree of contribution (usefulness of completion) of the completion to the user's work, and for example, can optimize the prompt engineering plan so that the degree of contribution of the completion is increased. For example, when generating a prompt having a plurality of items to which reference information is added, the amount of reference information (for example, business information, etc.) to be substituted for each item of the prompt (for example, the number of characters, or the number of pieces of reference information, etc.) can be adjusted to optimize the prompt engineering plan. That is, the amount of reference information to be substituted for each item can be automatically adjusted. Also, adjustments such as increasing the amount of reference information to be substituted for the item to be emphasized and decreasing the amount of reference information to be substituted for other items are possible. Therefore, for example, it is not necessary for administrator A to manually adjust by considering the appropriate amount of reference information or which item should be emphasized (whether it is better to have more successful cases or more failed cases, etc.). As a result, a prompt for obtaining more appropriate completion from the large language model can be generated with higher accuracy.

[0097] Also, the information processing apparatus 10 of the present embodiment stores a vector database 300 in which a plurality of vector processing results obtained by vector-converting each of a plurality of pieces of business information are stored. In the prompt engineering process, information based on an input from a user regarding the work is vector-converted, a vector processing result related to the information based on the vector-converted input is extracted from the vector database 300, and a prompt to be input to the natural language model is generated based on the business information of the extracted vector processing result.

[0098] In this way, by extracting from the vector database 300 the vector processing result related to the vector-converted information regarding the work, a prompt can be generated using business information relevant to the user's work.

[0099] In addition, the vector database 300 stores the vector processing results of each piece of summary business information, which is the business information and the information of the text summarized from the text related to the business information. In the prompt engineering process, the information processing device 10 extracts from the vector database the vector processing results related to the information based on the vector-converted input, and generates a prompt to be input into the natural language model based on the business information or the summary business information of the extracted vector processing results.

[0100] That is, the information processing device 10 of the present embodiment can more accurately extract the business information related to the user's business by vector-converting not only the business information but also the summary business information obtained by summarizing the business information. As a result, a prompt for obtaining more appropriate completion from the large language model can be generated with higher accuracy.

[0101] Also, the information processing device 10 of the present embodiment generates the vector processing results of the business information and the summary business information respectively based on the vector processing plan, which is the rule of vector conversion and summarization, and stores the generated vector processing results and the similarity (Score) between the vector processing result of the business information and the vector processing result of the summary business information in the vector database 300. Then, it refers to the vector database 300 to determine the similarity between the vector processing result of the business information and the vector processing result of the summary business information, and executes vector processing optimization processing to update the vector processing plan based on the determination result of the similarity (vector processing summary).

[0102] That is, the information processing device 10 of the present embodiment can optimize the vector processing so that, for example, the similarity (Score) between the vector processing result of the business information and the vector processing result of the summary business information becomes high. As a result, summary business information more similar to the original business information can be generated. Therefore, the reference information to be substituted into the prompt can be extracted with higher accuracy.

[0103] In addition, in vector processing, the information processing apparatus 10 of the present embodiment generates one or more summary business information from one piece of business information based on a vector processing plan, stores the vector processing results of the business information and the summary business information respectively, and the similarity (Score) between the vector processing result of the business information and the vector processing result of each summary business information in the vector database 300. In prompt engineering processing, based on the information related to the input after vector conversion, the vector processing result is extracted from the vector processing result of the summary business information with the highest similarity (Score), and a prompt for inputting into the natural language model is generated based on the business information or summary business information of the extracted vector processing result.

[0104] That is, the information processing apparatus 10 of the present embodiment can generate a prompt based on the summary business information (the best summary) with the highest similarity (Score) to the original business information. That is, the reference information to be substituted into the prompt can be extracted with higher accuracy.

[0105] In addition, the information processing apparatus 10 of the present embodiment executes an output process of referring to the vector database 300 and outputting a statistical result of the similarity between the vector processing result of the business information and the vector processing result of the summary business information.

[0106] Thereby, for example, the administrator A of the information processing apparatus 10 can confirm the accuracy of the vector processing.

[0107] In addition, in vector processing, when the similarity between the vector processing result of the business information and the vector processing result of the summary business information does not meet a predetermined standard, the information processing apparatus 10 of the present embodiment transmits a notification prompting an update of the vector processing plan to the information processing apparatus (administrator terminal 30) related to the administrator A.

[0108] Thereby, for example, the administrator A can review the vector processing plan in response to the notification.

[0109] Also, the prompt engineering plan is information on rules that specify a vector processing plan. In prompt engineering processing, the information processing apparatus 10 summarizes information based on an input, performs vector conversion based on the vector processing plan specified in the beprompt engineering plan, extracts a vector processing result related to the information based on the vector-converted input from the vector database 300, generates a prompt to be input to the natural language model based on the business information of the extracted vector processing result, and in prompt engineering optimization processing, executes vector processing optimization processing on the vector processing plan specified in the updated prompt engineering plan.

[0110] That is, when updating the prompt engineering plan, the information processing apparatus 10 of the present embodiment also updates the vector processing plan used in the prompt engineering processing. Thereby, for example, the amount of reference information to be substituted for each item of the prompt can be appropriately automatically adjusted, and the extraction accuracy of the reference information to be substituted for each item of the prompt can be improved.

[0111] The business database 200 stores business information and history information related to the business. The prompt engineering plan is information on rules that specify a prompt template including prompt items. In prompt engineering processing, the business information or history information of the extracted vector processing result is inserted into each item of the prompt template specified in the prompt engineering plan by the method specified in the prompt engineering plan, and items not specified in the prompt engineering plan are deleted from the prompt template to generate a prompt.

[0112] Thereby, a prompt can be efficiently generated using a prompt template with items set in advance.

[0113] Also, the prompt engineering plan is information on rules that specify the amount of information for each item of the prompt. The information processing apparatus 10 updates the amount of information for each item of the prompt based on the determination result of the similarity between the output data (completion) and the performance information in the prompt engineering optimization process.

[0114] Thereby, it is possible to optimize the prompt engineering plan by adjusting the amount of reference information (for example, the number of characters or the number of reference information) to be substituted for each item of the prompt. That is, the amount of reference information to be substituted for each item can be automatically adjusted.

[0115] In addition, the information processing apparatus 10 of the present embodiment executes an output process of referring to the prompt engineering database 400 and outputting the statistical result of the similarity (Final Score) between the output data (completion) and the performance information.

[0116] Thereby, for example, the administrator A of the information processing apparatus 10 or the like can confirm the accuracy of the prompt engineering process.

[0117] The present invention is not limited to the above-described embodiments, and can be implemented using any components without departing from the gist thereof. The embodiments and modifications described above are merely examples, and the present invention is not limited to these contents as long as the features of the invention are not impaired. Also, although various embodiments and modifications have been described above, the present invention is not limited to these contents. Other aspects conceivable within the scope of the technical idea of the present invention are also included in the scope of the present invention.

[0118] For example, a part of the hardware provided in each device of the present embodiment may be provided in another device.

[0119] Also, each program of the information processing apparatus 10 may be provided in another device, a certain program may be composed of a plurality of programs, or a plurality of programs may be integrated into one program.

Description of Symbols

[0120] 1 Information processing system, 10 Information processing device, 11 Processing device, 12 Memory, 13 Storage device, 14 Input device, 15 Output device, 16 Communication device, 20 User terminal, 30 Administrator terminal, 101 Execution plan management unit, 102 Vector processing management unit, 103 Vector processing optimization unit, 104 Prompt engineering management unit, 105 Template management unit, 106 Prompt engineering optimization unit, 107 Display control unit, 200 Business database, 300 Vector database, 400 Prompt engineering database

Claims

1. A storage device that stores a business database storing a plurality of pieces of business information, and prompt engineering processing for generating a prompt corresponding to an input from a user related to a business using the business information, based on a prompt engineering plan that is a rule for generating a prompt, obtaining output data by inputting the generated prompt into a natural language model that uses business information as an input value and outputs a proposal related to the business as an output value, and generating a prompt engineering database associating the obtained output data, the generated prompt, and the performance information of the business, a processing device that executes prompt engineering optimization processing for determining the similarity between the output data and the performance information by referring to the generated prompt engineering database and updating the prompt engineering plan based on the determination result of the similarity, An information processing apparatus comprising the same.

2. The storage device stores a vector database storing a plurality of vector processing results obtained by vector-converting each of a plurality of pieces of business information, The processing device In the prompt engineering processing, information based on an input from a user related to a business is vector-converted, a vector processing result related to the vector-converted information based on the input is extracted from the vector database, and a prompt to be input to the natural language model is generated based on the business information of the extracted vector processing result. The information processing apparatus according to claim 1.

3. The vector database stores the vector processing results of each of the business information and the summary business information, which is information of text obtained by summarizing the text related to the business information, The processing device In the prompt engineering processing, a vector processing result related to the information based on the vector-converted input is extracted from the vector database, and a prompt to be input to the natural language model is generated based on the business information or the summary business information of the extracted vector processing result. The information processing apparatus according to claim 2.

4. The processing device Based on the vector processing plan which is the rule of vector conversion and summarization, generate the vector processing results of the business information and the summary business information respectively, and store the generated vector processing results and the similarity between the vector processing result of the business information and the vector processing result of the summary business information in the vector database. This is the vector processing. Execute the vector processing optimization process of determining the similarity between the vector processing result of the business information and the vector processing result of the summary business information by referring to the vector database, and update the vector processing plan based on the determination result of the similarity. The information processing apparatus according to claim 3.

5. The processing apparatus is In the vector processing, based on the vector processing plan, generate one or more summary business information from one piece of business information, store the vector processing results of the business information and the summary business information respectively, and the similarity between the vector processing result of the business information and each of the vector processing results of the summary business information in the vector database. In the prompt engineering process, extract the vector processing result related to the information based on the vector-converted input from the vector processing result of the summary business information with the highest similarity, and generate a prompt for inputting to the natural language model based on the business information or the summary business information of the extracted vector processing result. The information processing apparatus according to claim 4.

6. The processing apparatus is Execute an output process of outputting a statistical result of the similarity between the vector processing result of the business information and the vector processing result of the summary business information by referring to the vector database. The information processing apparatus according to claim 4.

7. The processing apparatus is In the vector processing, when the similarity between the vector processing result of the business information and the vector processing result of the summary business information does not meet a predetermined standard, send a notification prompting the update of the vector processing plan to the information processing apparatus related to the administrator. The information processing apparatus according to claim 4.

8. The prompt engineering plan is information on the rule that designates the vector processing plan. The processing apparatus is In the prompt engineering process, based on the vector processing plan specified in the prompt engineering plan, summarize the information based on the input, perform vector conversion, extract the vector processing results related to the information based on the vector-converted input from the vector database, and generate a prompt to be input into the natural language model based on the business information of the extracted vector processing results. In the prompt engineering optimization process, perform the vector processing optimization process on the vector processing plan specified in the updated prompt engineering plan. The information processing apparatus according to claim 4.

9. The business database stores business information and history information related to the business. The prompt engineering plan is rule information that specifies a prompt template including prompt items. The processing apparatus In the prompt engineering process, insert the business information or history information of the extracted vector processing results into each item of the prompt template specified in the prompt engineering plan by the method specified in the prompt engineering plan, and delete the items not specified in the prompt engineering plan from the prompt template to generate the prompt. The information processing apparatus according to claim 2.

10. The prompt engineering plan is rule information that specifies the amount of information for each item of the prompt. The processing apparatus In the prompt engineering optimization process, update the amount of information based on the determination result of the similarity between the output data and the performance information. The information processing apparatus according to claim 1.

11. The processing apparatus Execute an output process that refers to the prompt engineering database and outputs a statistical result of the similarity between the output data and the performance information. The information processing apparatus according to claim 1.

12. An information processing apparatus A prompt engineering process that generates a prompt corresponding to an input from a user related to a business using business information based on a prompt engineering plan that is a rule for generating a prompt. The generated prompt is input into a natural language model that takes business information as an input value and outputs a proposal related to the business as an output value to obtain output data, and a prompt engineering database is generated by associating the obtained output data, the generated prompt, and the performance information of the business. A prompt engineering optimization process is executed to determine the similarity between the output data and the performance information with reference to the generated prompt engineering database, and to update the prompt engineering plan based on the determination result of the similarity. An information processing method.

13. In an information processing apparatus, prompt engineering processing for generating a prompt corresponding to an input from a user related to a business using business information based on a prompt engineering plan that is a rule for generating a prompt, and the generated prompt is input into a natural language model that takes business information as an input value and outputs a proposal related to the business as an output value to obtain output data, and a prompt engineering database is generated by associating the obtained output data, the generated prompt, and the performance information of the business. A prompt engineering optimization process is executed to determine the similarity between the output data and the performance information with reference to the generated prompt engineering database, and to update the prompt engineering plan based on the determination result of the similarity. An information processing program.

Images