A large language model automatic prompt optimization method based on example feedback

Abstract

The application discloses a large language model automatic prompt optimization method based on sample feedback and belongs to the technical field of large language model prompt optimization. The application solves the problem that the traditional prompt optimization method is difficult to realize automatic optimization in the prior art. The application constructs a prompt optimization module based on a large language model, inputs preprocessed input data, performs prompt optimization based on sample feedback on the input data, and obtains modified prompts. A prompt simplification module based on the large language model is constructed to simplify and rewrite the super-long prompts in the modified prompts, obtain updated prompts, and transmit the updated prompts to the prompt optimization module for iteration to obtain optimized prompts. A local search module based on the large language model is constructed to perform local search and optimization on the optimized prompts to obtain optimal prompts. The application effectively improves the performance of the large language model for prompt optimization and can be applied to automatic prompt optimization using the large language model.

Description

TECHNICAL FIELD

[0001] The present application relates to a large language model automatic prompt optimization method, in particular to a large language model automatic prompt optimization method based on example feedback, and belongs to the technical field of large language model prompt optimization. BACKGROUND

[0002] Large language models (LLMs) are one of the current research hotspots in the field of artificial intelligence. With the development of deep neural network structures and computing power, neural networks have been widely used in various industries. The good parallel operation efficiency brought by the sequential modeling scheme and model structure based on transformer has given rise to large language models. The large parameter quantity and training corpus of large language models make large language models have good generalization, and they can achieve good results in the face of complex downstream tasks that have not been trained.

[0003] Due to the large parameter quantity of large language models, training large language models using traditional fine-tuning methods will cause great computing power consumption and may also cause pre-training knowledge forgetting problems, reducing the overall performance of the model. Traditional prompt optimization methods rely heavily on human and experience input, and cannot achieve a quick and automatic optimization process when facing complex problems.

[0004] In view of the above, a large language model automatic prompt optimization method based on example feedback is needed. SUMMARY

[0005] In the following, a brief summary of the present application is given in order to provide a basic understanding of some aspects of the present application. It should be understood that this summary is not an exhaustive overview of the present application. It is not intended to identify key or important parts of the present application nor is it intended to limit the scope of the present application. Its purpose is merely to present some concepts in a simplified form as a prelude to the more detailed description to be discussed later.

[0006] In view of this, in order to solve the problem that the traditional prompt optimization method in the prior art cannot achieve automatic optimization, the present application provides a large language model automatic prompt optimization method based on example feedback.

[0007] The technical scheme is as follows: a large language model automatic prompt optimization method based on example feedback, comprising the following steps:

[0008] S1. Constructing a prompt optimization module based on a large language model, inputting pre-processed input data, and performing prompt optimization based on example feedback to obtain modified prompts;

[0009] S2. Constructing a prompt refinement module based on a large language model, refining and rewriting the super-long prompts in the modified prompts to obtain updated prompts, transmitting the updated prompts to the prompt optimization module for iteration to obtain optimized prompts;

[0010] S3. Constructing a local search module based on a large language model, performing local search and optimization on the optimized prompts to obtain optimal prompts.

[0011] Further, in S1, the following steps are specifically included:

[0012] S11. Preprocessing the input data, and inputting the preprocessed input data into the prompt optimization module based on a large language model;

[0013] S12. Forward propagation through the prompt optimization module to obtain a prediction error set

[0014] S13. According to the prediction error set , gradient calculation through the prompt optimization module to obtain a gradient set G i ;

[0015] S14. According to the gradient set G i , backward propagation through the prompt optimization module to output the modified prompts;

[0016] In S11, the input data includes an initial prompt and training data, the initial prompt is prompt_0, which is expressed as a sentence of text, used to clearly describe the target task, and the training data is supervised data;

[0017] In S12, based on a large language model, using prompt i to infer data S i in batch i, i = 1, 2,..., n, to obtain model inference result ModelOut s , based on the supervised label of data S i in batch i and the pre-set threshold, using a large language model to evaluate the inference output, and integrating the data below the pre-set threshold or the inference error into the prediction error set

[0018] The prediction error set is expressed as:

[0019]

[0020] Where s is the input of the model, and answer s is the label corresponding to the model input;

[0021] In S13, using a large language model to evaluate the prediction error set Evaluation is performed, input features of training data are analyzed, and model inference result ModelOut is obtained s The gap between the supervision label and the current prompt_i is analyzed to find the reason for the inference error using a large language model, and the output is in a fixed format text, and the gradient set G is calculated i ;

[0022] The gradient set G i is represented as:

[0023]

[0024] In the S14, the back propagation process is represented as: using a large language model to expand the current prompt_i, adding the features extracted in the gradient set G i to the prompt i to obtain a new prompt i+1, and based on the new prompt i+1 and batch i+1, the next round of iteration is performed until the maximum number of iterations is reached, and the current new prompt_i+1 is output as the modified prompt.

[0025] Further, in the S2, the following steps are specifically included:

[0026] S21. Clustering and merging the current new prompt_i+1 by a prompt simplification module based on a large language model to obtain a classification result after class merging;

[0027] S22. Re-combining the updated prompt according to the classification result after class merging and the original prompt to obtain an optimized prompt for the prompt optimization module iteration;

[0028] In the S21, for the rule description text in the current new prompt_i+1, the classification is performed according to the semantics and the classification result is output, and for each classification result, the redundant part in the same classification result is merged and deleted based on a large language model to obtain the classification result after class merging.

[0029] Further, in the S3, the following steps are specifically included:

[0030] S31. Selecting two prompts in the optimized prompt and randomly rewriting them by a local search module to obtain n rewritten prompts;

[0031] S32. According to the n rewritten prompts obtained, the local search module is used to evaluate them to obtain the optimal prompt on the validation set;

[0032] In the S31, for the k rules of the rule description text in the selected two prompts, some rules are randomly extracted, randomly rewritten using a large language model, and repeated n times to obtain n rewritten prompts, which are regarded as n local searches around the current optimal prompt.

[0033] In S32, according to the obtained n rewriting prompts, evaluation is performed on the evaluation set based on a predetermined rule and a large language model, the two prompts that perform best are selected to repeat the iteration of the local search module, until the preset maximum iteration step is reached, and the optimal prompt on the verification set is obtained.

[0034] The beneficial effects of the present application are as follows: the present application proposes a self-feedback mechanism for building a large language model based on a small number of supervised examples, and an interactive logic for building a large language model based on the method of training a model based on backpropagation, using artificial labeled examples to automatically optimize input prompts; the present application can automatically optimize and update the prompts of the large language model, freeing human labor from tedious prompt modification tasks, improving task efficiency, and at the same time, compared with artificially optimized prompts, the results of the present application can be closer to the global optimal solution of the prompts concerned in the target task, achieving better performance in the target task. BRIEF DESCRIPTION OF DRAWINGS

[0035] The accompanying drawings, which are included to provide a further understanding of the application, form a part of the application and, along with the description, serve to explain the application. The illustrative embodiments of the present application and their description serve to explain the application without limiting the application. In the drawings:

[0036] Figure 1 It is a flowchart of a large language model automatic prompt optimization method based on example feedback;

[0037] Figure 2 It is a flowchart of an embodiment of a large language model automatic prompt optimization method based on example feedback;

[0038] Figure 3 It is a flowchart of an embodiment of using a prompt simplification module for simplification and rewriting;

[0039] Figure 4 It is a flowchart of an embodiment of using a local search module for search and optimization. DETAILED DESCRIPTION

[0040] In order to make the technical solutions and advantages of the embodiments of the present application clearer, the following will further describe the exemplary embodiments of the present application with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, not all the embodiments. It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict.

[0041] REFERENCE Figures 1-4 In detail, a large language model automatic prompt optimization method based on example feedback, specifically comprising the following steps:

[0042] S1. Constructing a prompt optimization module based on a large language model, inputting pre-processed input data, performing prompt optimization based on example feedback, and obtaining modified prompts;

[0043] S2. Constructing a prompt optimization module based on a large language model, performing prompt optimization based on example feedback, and obtaining modified prompts;

[0044] S3. Constructing a prompt optimization module based on a large language model, performing prompt optimization based on example feedback, and obtaining modified prompts;

[0045] Further, in S1, the following steps are included:

[0046] S11. Preprocessing the input data and inputting the preprocessed input data into the prompt optimization module based on the large language model;

[0047] S12. Forward propagation through the prompt optimization module to obtain a prediction error set

[0048] S13. According to the prediction error set Gradient calculation through the prompt optimization module to obtain a gradient set G i ;

[0049] S14. According to the gradient set G i , backward propagation through the prompt optimization module to output the modified prompts;

[0050] In S11, the input data includes initial prompts and training data, and the initial prompts are prompt_0, which is a brief text for clearly describing the target task, such as "You are a Chinese-English translator, you need to translate the input English into Chinese and output it", and the training data is generally hundreds of supervised data, such as several hundred parallel corpora in the translation task;

[0051] In S12, based on the large language model, the data S i in batch i is inferred using prompt i, i = 1, 2,..., n, to obtain model inference result ModelOut s , based on the supervised label of data S i in batch i and the pre-set threshold (if it does not exist, it does not need to be considered), the inference output is evaluated using the large language model, and the data below the pre-set threshold or the inference error is integrated into the prediction error set

[0052] Prediction error set is represented as:

[0053]

[0054] where s is the input of the model, answer s is the label corresponding to the input of the model;

[0055] In the S13, the prediction error set is evaluated, the input features of the training data are analyzed, and the gap between the model inference result ModelOut s and the supervised label is analyzed, the reason for the inference error of the current prompt_i is automatically analyzed using the large language model, and the output is in the form of fixed format text, and the gradient set G i is calculated.

[0056] The gradient set G i is represented as:

[0057]

[0058] In the S14, the back propagation process is represented as: using the large language model to expand the current prompt_i, adding the features extracted from the gradient set G i to the prompt i, obtaining a new prompt i+1, based on the new prompt i+1 and the batch i+1, performing the next round of circulation, until the preset maximum iteration step is reached, and outputting the current new prompt_i+1, that is, the modified prompt.

[0059] Specifically, referring to Figure 3 , the main purpose of step S1 is to optimize the input prompt based on a small amount of supervised examples, and to realize automatic prompt optimization of the large language model based on small-scale training data.

[0060] The prompt optimization module is the core module, similar to training a deep neural network based on a back propagation algorithm, and the prompt optimization module is divided into three algorithm steps: (1) forward propagation: performing inference on a small batch of data based on the current prompt; (2) calculating gradients: based on the small batch of data with inference errors, using the large language model to generate features of error examples in the form of text; (3) back propagation: based on the features generated by calculating the gradient, modifying the current prompt, and performing the next round of circulation.

[0061] Further, in the S2, the following steps are specifically included:

[0062] S21. Clustering and merging the current new prompt_i+1 by the prompt simplification module based on the large language model, to obtain a classification result after class merging;

[0063] S22. According to the classification results after class merging and the original prompt, recombine to obtain an updated prompt for prompting the optimization module iteration to obtain an optimized prompt;

[0064] In S21, for the rule description text in the current new prompt_i+1, the classification is performed according to the semantics and the classification results are output. For each classification result, the redundant part in the same classification result is merged and deleted based on the large language model to obtain the classification results after class merging, thereby improving the information density of the current prompt.

[0065] Specifically, referring to Figure 4 The main purpose of step S2 is to optimize the super-long prompt that appears in the training process of step S1, automatically simplify and rewrite, and improve the information density of the optimized prompt and the reasoning efficiency of the large language model. For the rule description text in the prompt, the rule description text is clustered and merged based on the large language model thinking chain.

[0066] In the prompt optimization module, since the current prompt needs to be expanded each time the optimization step is performed, in the case of a large amount of training data, the prompt may be super-long. The super-long prompt may cause many additional problems, such as increased computational overhead and loss of context. When the prompt is too long, the large language model may generate content that is not coherent or lacks context consistency. The large language model may lose focus in long text and generate content that is inconsistent with the previous text or unrelated to the question, thereby reducing the quality of the output.

[0067] Further, in S3, the following steps are included:

[0068] S31. Select two prompts in the optimized prompt, and randomly rewrite them through the local search module to obtain n rewritten prompts;

[0069] S32. According to the n rewritten prompts obtained, evaluate them through the local search module to obtain the optimal prompt on the validation set;

[0070] In S31, the local search module starts iteration with one current prompt. For k rules of the rule description text in the selected two prompts, randomly extract some rules, randomly rewrite them using the large language model, and repeat n times to obtain n rewritten prompts, which are regarded as n local searches around the current optimal prompt.

[0071] In S32, according to the n rewritten prompts obtained, evaluate them based on the predetermined rules and the large language model on the evaluation set, select the two prompts with the best performance to repeat the iteration of the local search module, until the maximum number of iterations is reached, and obtain the optimal prompt on the validation set.

[0072] Specifically, referring to Figure 4 , the current two prompts selected are prompt fst, i.e., prompt 1, and prompt sed, i.e., prompt 2, and the main purpose of step S3 is to prevent overfitting of the prompt optimization algorithm and improve the generalization performance of the prompt. After the training of the prompt optimization module is completed, searching is performed around the optimal prompt obtained at present to obtain the optimal prompt on the validation set;

[0073] In the automatic prompt optimization technology, overfitting refers to that the generated prompt performs well on the training set but performs poorly on unseen data, that is, the optimization result of the prompt excessively fits the training data involved in the optimization process and lacks generalization ability. For the prompt automatic optimization technology that directly depends on the sample features and has less training data, the overfitting phenomenon will be more serious. The purpose of the local search module is to alleviate this overfitting phenomenon. The specific steps include randomly rewriting the prompt and evaluating the prompt.

[0074] In this embodiment, first, about one hundred pieces of supervised data and an initial prompt to be optimized are collected. Then, based on the prompt optimization module and the prompt refinement module, optimization of the input prompt is completed. Finally, based on the local search module, the generalization ability of the prompt optimized based on the sample is improved.

[0075] In the product labeling task in the e-commerce field, if the manual prompt optimization method is used, not only a large amount of manpower will be wasted, but also the manually optimized prompt cannot completely cover the many rules required by the product labeling task. Using the prompt optimization method proposed in the present application, the prompt optimized by the large language model can better utilize the capabilities of the large language model and achieve better results on the target task.

[0076] Although the present application is described in terms of a limited number of embodiments, those skilled in the art, with the benefit of the above description, will appreciate that other embodiments are contemplated within the scope of the application described herein. In addition, it should be noted that the language used in the specification is chosen primarily for readability and instructional purposes and can not have been selected to convey an exclusive or exhaustive meaning. Therefore, many modifications and variations of the present application are possible in light of the above teachings without departing from the scope and spirit of the appended claims. The disclosure of the application is illustrative only and not limiting of the scope of the application, which is defined by the appended claims.

Claims

1. A large language model automatic prompting optimization method based on example feedback, characterized in that, The method comprises the following steps: S1. Constructing a prompt optimization module based on a large language model, inputting preprocessed input data, performing prompt optimization based on example feedback, and obtaining modified prompts; S2. Constructing a prompt refinement module based on a large language model, refining and rewriting the super-long prompts in the modified prompts, obtaining updated prompts, and transmitting the updated prompts to the prompt optimization module for iteration to obtain optimized prompts; S3. Constructing a local search module based on a large language model, performing local search and optimization on the optimized prompts, and obtaining optimal prompts; In S1, the following steps are included: S11. Preprocess the input data and input the preprocessed input data into the prompt optimization module based on the large language model; S12. Perform forward propagation by prompting the optimization module to obtain a set of prediction errors ; S13. According to the prediction error set , gradient calculation is performed by prompting the optimization module to obtain a gradient set ; S14. According to the gradient set by prompting the optimization module to backpropagate, output the modified prompt.

2. The method of claim 1, wherein the method is based on a large language model automatic prompt optimization method based on example feedback. In S11, the input data includes an initial prompt and training data, the initial prompt is prompt_0, which is a text for clearly describing the target task, and the training data is supervised data; In the S12, based on the large language model, the data in the batch i is evaluated using the prompt i , i = 1, 2,..., n, to obtain a model inference result , based on the supervised label of the data in the batch i and the pre-set threshold, the inference output is evaluated using the large language model, and the data below the pre-set threshold or the inference error is incorporated into a prediction error set ; Set of prediction errors is represented as: ； wherein, is an input to the model, is a label corresponding to the model input; In step S13, a large language model is used to analyze the prediction error set. The evaluation involves analyzing the input features of the training data and the model inference results. The gap between the current prompt and the supervised label is analyzed automatically using a large language model to determine the cause of the inference error. The result is output as text in a fixed format, and the gradient set is calculated. ; Gradient set is represented as: ； In the S14, the back propagation process is represented as: expanding the current prompt_i using a large language model, adding the extracted features in the S13 to the prompt_i to obtain a new prompt_i+1, and based on the new prompt_i+1 and a batch_i+1, performing a next round of iteration until a preset maximum iteration step number is reached, and outputting the current new prompt_i+1 as the modified prompt. In the S14, the back propagation process is represented as: expanding the current prompt_i using a large language model, adding the extracted features in the S13 to the prompt_i to obtain a new prompt_i+1, and based on the new prompt_i+1 and a batch_i+1, performing a next round of iteration until a preset maximum iteration step number is reached, and outputting the current new prompt_i+1 as the modified prompt.

3. The method of claim 2, wherein the method is based on a large language model automatic prompt optimization method based on example feedback. In S2, the following steps are included: S21. Clustering and merging the current new prompt_i+1 by the prompt refinement module based on the large language model to obtain the classification results after class merging; S22. Re-combining the updated prompts according to the classification results after class merging and the original prompts to obtain the updated prompts for the prompt optimization module iteration to obtain the optimized prompts; In S21, the rule description text in the current new prompt_i+1 is classified and the classification results are output according to the semantics, and for each classification result, the redundant part in the same classification result is merged and deleted based on the large language model to obtain the classification results after class merging.

4. The method of claim 3, wherein the method is characterized by, In S3, the following steps are included: S31. Selecting two prompts in the optimized prompts, randomly rewriting them by the local search module to obtain n rewritten prompts; S32. According to the n rewritten prompts, the local search module is used to evaluate them to obtain the optimal prompt on the validation set; In S31, for the k rules of the rule description text in the selected two prompts, some rules are randomly extracted, randomly rewritten by the large language model, and repeated n times to obtain n rewritten prompts, which are regarded as n local searches around the current optimal prompt; In S32, according to the n prompts, the predetermined rules and the large language model are used to evaluate them on the evaluation set, and the two prompts with the best performance are selected to repeat the iteration of the local search module until the maximum iteration step is reached to obtain the optimal prompt on the validation set.

Images