Large model automatic prompt tuning method and device using gradient optimization and medium

Abstract

The invention discloses a large model automatic prompt tuning method and device using gradient optimization and a medium, and the method comprises the steps: scoring an execution rule in each test data cue word and a corresponding question and answer model output answer through an evaluation model, and carrying out the gradient descent optimization of the total evaluation score of the cue word and the evaluation score of each execution rule, in the tuning process, judging whether the current average score and turn meet a set threshold value or not; if not, outputting the cue word of the current round as the optimal cue word; and if yes, selecting the optimization direction of the optimal cue word, refining the optimization direction, generating a new cue word based on the refined optimization direction, repeating the step until a set threshold value is met, stopping the optimization round, and outputting the optimal cue word. The convergence problem in the long cue word optimization process can be efficiently and accurately solved, and the pertinence and effectiveness of cue word tuning are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of model question-answering evaluation, and in particular to a method, device and medium for automatically prompting tuning of a large model using gradient optimization. Background Art

[0002] With the rapid development of artificial intelligence, large language models (LLMs) have become an important research direction in the field of natural language processing. Large language models (LLMs) refer to language models that contain hundreds of billions (or more) of parameters and are trained on large amounts of text data. These models have powerful language understanding and generation capabilities, and have shown good results in tasks such as machine translation, text summarization, and dialogue generation. Large language models are also gradually playing a pivotal role in various industries. Prompt refers to the input text or instructions provided to the generative AI model. It can be a question, a sentence, a piece of text, or a complete paragraph to guide the model to generate the corresponding answer or output. The role of the prompt is to tell the model what task needs to be completed or what kind of result to produce.

[0003] The prior art proposes the concept of Chain of Thoughts Prompting, which enhances complex reasoning capabilities by introducing intermediate reasoning steps. This method can be combined with few-sample prompts to achieve better results, and is particularly suitable for tasks that require more complex reasoning. That is, a series of intermediate reasoning steps are described, and the reasoning process is manually constructed, so that large models can achieve more ideal results in tasks such as arithmetic reasoning, common sense reasoning, and symbolic reasoning by learning these reasoning processes. It can be seen that the same base model may have very different task execution effects under different prompt words. However, the process of building an adapted prompt word is a time-consuming and labor-intensive process, so Zhou et al. (2022) proposed the Automatic Prompt Engineer, a framework for automatic instruction generation and selection. The instruction generation problem is constructed as a natural language synthesis problem, and LLMs are used as a solution to the black-box optimization problem to generate and search for candidate solutions. Since then, many people have made certain optimizations to address the shortcomings of this work. Instructions and corresponding scores are added during the guidance process to allow the model to understand the process of score improvement, thereby improving prompt optimization. Analysis of the shortcomings of the original instructions is added during the guidance process and used to guide the optimization of the new prompt. The non-convergence problem of long prompt optimization is solved by splitting the original prompt into several sentences and modifying only one of them at a time.

[0004] However, after many optimizations, APE technology still has some obvious shortcomings, mainly in the following aspects:

[0005] When solving non-standard tasks such as generative tasks, such as translation tasks, text summarization, dialogue systems, etc., traditional APE technology often needs to set different evaluation rules for different tasks during the optimization process, that is, there is a lack of a unified task evaluation standard.

[0006] Secondly, when performing complex tasks, the input prompt words are usually long and complex. Although previous technologies have alleviated the convergence problem in long optimization to a certain extent, in the optimization process, since the model optimizes each sentence separately, not only is the process complicated and cumbersome, it also brings unnecessary overhead. The examples for guiding modification are not specifically targeted at the problematic sentences, but more of a general and extensive analysis, which weakens the ability to guide the optimal prompt words.

[0007] Finally, the optimization process of traditional APE is actually a black box process, which leads to the lack of interpretability of our optimization process. Summary of the invention

[0008] The technical problem to be solved by the present invention is that the traditional optimization process lacks interpretability. The purpose is to provide a large model automatic prompt tuning method, device and medium using gradient optimization. By proposing an automated prompt tuning framework for long prompts in complex scenarios, the execution rules in each test data prompt word and the corresponding question-answering model output answer are scored through an evaluation model under the framework, and the total evaluation score of the prompt word and the evaluation score of each execution rule are gradient descent tuned. In each modification process, we only modify the worst rule and use specific examples of the rule to further guide it. Therefore, the convergence problem in the long prompt optimization process can be solved more efficiently and accurately. The framework constructed by this solution is an interpretable process for the ape process, which improves the pertinence and effectiveness of prompt tuning.

[0009] The present invention is achieved through the following technical solutions:

[0010] The first aspect of the present invention provides a large model automatic prompt tuning method using gradient optimization, comprising the following specific steps:

[0011] Initialization parameter settings;

[0012] Obtain the execution rules in each test data prompt word and the corresponding question-answering model output answer;

[0013] Construct an evaluation model to score the execution rules in each test data prompt word and the corresponding question-answer model output answer, and obtain the total evaluation score of the prompt word and the evaluation score of each execution rule;

[0014] Perform gradient descent optimization on the total evaluation score of the prompt word and the evaluation score of each execution rule:

[0015] Get the average score corresponding to the prompt word of the current round, and determine whether the current average score and round meet the set threshold;

[0016] If not, the prompt word of the current round is output as the optimal prompt word;

[0017] If it is satisfied, the optimization direction of the best prompt word is selected, the optimization direction is refined, and a new prompt word is generated based on the refined optimization direction. This step is repeated until the set threshold is met, the optimization round is stopped, and the best prompt word is output.

[0018] Furthermore, the initialization parameter settings specifically include: initializing the current round, the maximum iteration round, the initial prompt word effect score, the target threshold, the step size of each optimization, the number of boundary attempts, and the initial prompt word.

[0019] Furthermore, the target threshold is used to describe the effect score that the user expects the prompt word to achieve, the step size is used to indicate the number of words added, deleted or modified each time, the number of boundary attempts is used to indicate how many rounds the score remains unchanged and reaches the upper limit of the model, and the initial prompt words include background introduction, task description, execution rules and task body.

[0020] Furthermore, when the evaluation model is scored, the evaluation model scores the execution rules in the prompt words according to the answers output by the question-answering model. The scoring process includes: calculating the total score of each test data under the prompt words, and taking the average of n data, and obtaining the total evaluation score of the evaluation model for the prompt words and the evaluation scores of each rule based on the average of the n data.

[0021] Furthermore, when the evaluation model scores the execution rules in each test data prompt word and the corresponding model output answer, it also includes:

[0022] Determine whether the average score of the current round of evaluation scores and the round meet the set threshold. If the average score does not change for k consecutive rounds, it is determined that the optimal boundary of the model has been reached, and the prompt word of the current round is output as the optimal prompt word.

[0023] Furthermore, the optimization direction of selecting the best prompt word specifically includes: obtaining all rules of the current prompt word, generating multiple forward optimization spaces according to all the rules, and selecting the rule with the lowest score among all the rules as the current optimization direction.

[0024] Furthermore, the refinement and optimization direction specifically includes:

[0025] Move forward according to the current optimization direction, and obtain the number of words that can be modified each time as the step length of each step;

[0026] Perform score analysis on the prompt words corresponding to the rule with the lowest score;

[0027] Select the three test data with the lowest scores of the rule and the corresponding answers as examples;

[0028] The evaluation model performs analysis based on the examples, and the question-answering model optimizes the step size of the rule based on the analysis results to generate N new prompt words.

[0029] Furthermore, each time the step size of the rule is optimized, only the rule with the lowest average score in the prompt is modified.

[0030] A second aspect of the present invention provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, a large model automatic prompt tuning method utilizing gradient optimization is implemented.

[0031] A third aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a large model automatic prompt tuning method using gradient optimization.

[0032] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0033] 1. This paper proposes an automatic prompt tuning framework for long prompts in complex scenarios. Our framework has a unified format for all tasks and proposes a unified evaluation method for our framework.

[0034] 2. Under the framework of the present invention, only the execution rules, which are the key part of guiding the model to accurately generate answers, are tuned. In each modification process, we only modify the worst rule and use the specific examples of the rule to further guide it. Therefore, the convergence problem in the long prompt optimization process can be solved more efficiently and accurately.

[0035] 3. Under the framework of the present invention, the ape process is an explainable process, which further improves the pertinence and effectiveness of prompt tuning. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the technical solutions of the exemplary embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present invention and should not be regarded as limiting the scope. For those of ordinary skill in the art, other relevant drawings can be obtained based on these drawings without creative work. In the drawings:

[0037] Figure 1 This is the optimization process in the embodiment of the present invention;

[0038] Figure 2 This is the prompt optimization process in the embodiment of the present invention;

[0039] Figure 3 This is a prompt optimization and simplification process in an embodiment of the present invention. DETAILED DESCRIPTION

[0040] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with embodiments and drawings. The exemplary embodiments of the present invention and their description are only used to explain the present invention and are not intended to limit the present invention.

[0041] As a possible implementation, Figure 1 As shown, in order to solve the problems existing in the existing APE optimization process, this embodiment provides a large model automatic prompt tuning method using gradient optimization. First, this embodiment proposes a basic framework for long prompts in complex scenarios, and proposes a unified evaluation rule for this framework. In this embodiment, all prompts can be regarded as four parts, namely background introduction (env), task description (task), execution rules (rules), and task subject (object). We believe that rules are the key to guiding whether the model can accurately generate answers. Therefore, when judging the pros and cons of generative tasks, we use the evaluation model to score the answers and the degree of satisfaction with the rules, thereby obtaining a standardized evaluation rule.

[0042] On this basis, we proposed our specific optimization method. For the test set, we can get the overall average score and the average score of each rule. During each modification, we only modify the rule with the lowest average score in the prompt, and pass in k test examples and scoring analysis of the rule and the lowest score, so that the model can better understand the deficiencies and modify the rule. In this process, we not only achieved the explainability of the modified sentence, but also achieved the explainability of our modification process.

[0043] Finally, we regard the above process as a gradient descent optimization process, and abstract it into the mathematical space to achieve visualization and stability of the optimization process. If each rule is regarded as the direction of optimization in the optimization process, and the number of words that can be modified each time is regarded as the step size of each step, the process is abstracted into a process in which the score corresponding to each prompt descends toward the optimal point. At the initial point, we can get the average score corresponding to prompt0 and the rule that should be modified. Then each autonomous modification of the model can be regarded as a continuous attempt in this direction. The prompt closest to the optimal point obtained by this modification is used as the new prompt. (Our gradient descent idea) The above process is repeated until the optimal boundary of this model is reached or the upper limit of the specified execution rounds is reached. The implementation process is as follows: Figure 2 As shown, if we confirm the direction of each step, Figure 2 The process shown can be transformed into Figure 3 As shown in the process, after the transformation, the performance of the model is improved, the initial prompt0 is closer to the optimal boundary of the model, and the optimal boundary of the model is closer to the optimal point.

[0044] Specifically, this embodiment provides a detailed optimization step:

[0045] 1. Initial parameter setting

[0046] Set the current round t = 0, the maximum iteration round T, the initial prompt word effect score aver-score_0t = 0, the target threshold threshold = P, the step length of each optimization l, the number of boundary attempts k, and the initial prompt word. Among them, the score of the initial prompt overall effect is aver-score_00 = 0, the threshold threshold describes the effect score that the user expects the prompt to achieve, the step length indicates the number of words added, deleted, and modified each time, and the number of boundary attempts indicates how many rounds the score remains unchanged and reaches the upper limit of the model. The initial prompt word structure includes background introduction, task description, execution rules, and task body.

[0047] 2. Testing and Scoring

[0048] The prompt word of each test data (hereinafter referred to as promptt-1) and the corresponding question-answering model output are input into the evaluation model (the evaluation model used in this embodiment can be GPT-4), and the evaluation model scores according to the answer output by the question-answering model and the execution rules (rules) in promptt-1. The full score of each rule is 10 points. The total score of each test data under promptt-1 is calculated, and the average of n data is taken to obtain the evaluation model's total evaluation score aver-score_0t for promptt-1 and the evaluation score aver-score_it for each rule.

[0049] 3. Gradient descent optimization process

[0050] In each round of optimization, the following detailed steps are performed:

[0051] 3.1 Conditional Judgment

[0052] Analyze the current round of scores and determine whether the current average score aver-score_t and the round meet the predetermined conditions. If they do, proceed to the next step; if not, output prompt-t of the current round as the optimal prompt. Note that if the average score does not change for consecutive k rounds, we believe that we have reached the optimal boundary of the model (i.e. Figure 1 Figure 2 The optimal boundary that this model can achieve in ), and also outputs prompt-t of the current round as the optimal prompt

[0053] 3.2 Selecting the optimization gradient

[0054] All rules are considered as the forward optimization space of the task, and the rule with the lowest score is selected as the current optimization direction. That is, the direction with the steepest gradient is selected for optimization in gradient descent.

[0055] 3.3 Use evaluation to further refine the optimization direction and optimize according to the specified step size

[0056] The test data corresponding to the rule with the lowest score is further analyzed. After the analysis, the three test data with the lowest score of the rule and the answer are selected as few-shot. The evaluation model gives the corresponding analysis according to the input content. The question-answering model uses the analysis to modify and optimize the rule N times according to the specified step length, and generates N new prompt-t. That is, in gradient descent, a new point is generated by taking a certain step length along the gradient direction.

[0057] 3.4 Optimization

[0058] The N new prompt-t are tested again and evaluated and scored by the evaluation model. If the N new prompt-t do not improve, return to modify again until the score improves and then proceed to the next step. If the number of re-modifications reaches the boundary attempt number k, it is considered that the optimal boundary within the number of attempts has been reached and the optimization is terminated. Among these N prompts, the prompt with the highest score is selected as the updated prompt-t.

[0059] 4. Update and loop:

[0060] Update the round number t and repeat the loop of step 3 until the termination condition is met or the maximum iteration round number T is reached.

[0061] As a possible implementation method, this embodiment provides a specific application scenario, which includes multiple experimental contents. The experiment revolves around a complex task, namely recommending financial products based on user portraits. Each prompt word (prompt) consists of four parts: background introduction (env), task description (task), execution rules (rules) and task subject (object). Experiments one to three aim to show that the prompt word tuning scheme we proposed is not inferior to the traditional scheme when the standard answer is known, and it can converge faster. Experiment four demonstrates the applicability of our scheme in complex backgrounds without standard answers.

[0062] Experiment 1: Apply our proposed cue word tuning scheme to the traditional standard answer context.

[0063] Experiment 2: Use the OPRO prompt word tuning scheme mentioned by Chengrun et al. in "LARGELANGUAGE MODELSASOPTIMIZERS". Input the same original prompt word and customer feature data into the big model, let the big model select the investment plan for each customer, and get the accuracy under the standard answer. Input this prompt word, the corresponding investment plan and accuracy selected for each customer, and the customer's actual choice into GPT4, let it give a score and modification suggestions for the entire execution rule (rules), input this modification suggestion and score into the big model, let it optimize and generate new prompt words by itself, and then apply the new prompt word to the recommendation of the investment plan to get a new accuracy, and add the new prompt word, investment plan and accuracy generated by this iteration to the set of original prompt word, investment plan and accuracy, and input it into GPT4 together with the customer's actual choice, let GPT4 score again and give modification suggestions. There are four main differences between this and the solution we proposed: First, it can only be used in tasks with standard answers; Second, GPT4 scores the entire rules instead of scoring each rule separately; Third, GPT4 gives modification suggestions instead of the analysis mentioned in our solution, but correspondingly, the large model only explores the process of modifying the prompt words by itself once in each round, while we explore three times in each round and take the one with effective modification; Fourth, the data set input to GPT4 each time is a collection of prompt words, investment plan choices and accuracy rates generated in all historical iterations.

[0064] Experiment 3: Use the meta-prompting LLMs two-step task prompt word tuning scheme mentioned by Qinyuan Ye et al. in "Prompt Engineering a Prompt Engineer". Input the same original prompt word and customer feature data into the big model, let the big model select the investment plan for each customer, and get the accuracy under the standard answer. Then proceed in two steps. In the first step, input this prompt word, the corresponding investment plan and accuracy selected for each customer, and the customer's actual choice into GPT4, and ask GPT4 whether this prompt word needs to be modified; in the second step, if it needs to be modified, let GPT4 give a score and positive modification suggestions for the entire execution rule (rules). Next, input this modification opinion and score into the big model, let it optimize and generate new prompt words by itself, and then apply the new prompt word to the choice of investment plan to get a new accuracy, and input this new prompt word, investment plan and accuracy together with the customer's actual choice into GPT4, let GPT4 score again and give modification opinions. There are four main differences between this method and the solution we proposed: First, it can only be used in tasks with standard answers. Second, GPT4 scores the entire rules instead of scoring each rule separately. Third, GPT4 gives positive modification suggestions instead of the analysis mentioned in our solution, but in contrast, the large model only explores the process of modifying the prompt word once in each round, while we explore three times in each round and take the one with effective modification. Fourth, GPT4's analysis process is carried out in two steps. First, it needs to be asked whether the prompt word needs to be modified.

[0065] Experiment 4: Apply our proposed cue word tuning scheme to a non-traditional context where no standard answer is provided.

[0066] Table 1: The accuracy of this solution and the control solution in each round of the iteration process when the standard answer is provided

[0067]

[0068] Table 2: The score of each rule in the iterative process of this solution without providing a standard answer

[0069] Rules First round Round 2 Round 3 Total score 77.8 78.4 85 The first rule 85 85 90 The second rule 78 80 85 The third rule 82 70 80 Rule 4 74 82 88 Rule 5 70 75 82

[0070] According to the above content, it can be seen that the method adopted in this embodiment improves the pertinence and effectiveness of prompt tuning.

[0071] As a possible implementation method, this embodiment provides a tuning process in a specific scenario corresponding to the experimental process of the above experiment, such as Figure 1As shown, in the context of not providing a standard answer, the maximum iteration rounds are set to 8, the number of boundary attempts is 3, and the maximum threshold score is 85. If the results do not improve or the effect decreases after more than three times, or the maximum iteration rounds or the maximum threshold score is reached, the test is terminated.

[0072] Corresponding to Experiment 4 Experimental process:

[0073] Round 1:

[0074] First, the original prompt words "Select a suitable investment plan based on the customer's financial situation, investment goals and risk preferences; ensure that the selected investment plan matches the customer's long-term financial planning and life goals; the selected investment plan should include consideration of market dynamics and economic indicators to achieve diversified asset allocation; based on the customer's investment goals, select a plan that matches the customer's expected investment return and time frame; formulate flexible and adjustable investment plans based on the customer's personalized financial situation and liquidity needs" and customer characteristic data (for example, User 1: "Age": "30", "Gender": "Male", "Annual Income": "100000", "Occupation": "Programmer", "Marital Status": "Single", "Do you have a personal house": "None", "Risk Tolerance": "Medium", "Investment Goal": "Long-term appreciation") are input into the big model, allowing the big model to formulate an investment plan for each customer. This original prompt word and the investment plan formulated by the large model are input into GPT4, which scores each execution rule of the original prompt word and gives corresponding analysis (for example: Execution rule 1: [Score: 85; Analysis: The investment plan generally takes into account the customer's financial situation, investment objectives and risk preferences, but in some users' plans, the match between risk tolerance and investment portfolio is slightly insufficient.]).

[0075] Next, we enter the process of modifying the execution rules. We use GPT4 to give the score and analysis of each execution rule, and then use GPT4 to make corresponding optimization suggestions based on the analysis. Finally, we input the analysis and optimization suggestions into the big model, allowing the big model to modify the prompt words based on this. The requirement is that only the execution rule with the lowest GPT4 score can be modified.

[0076] Finally, let the big model formulate the customer's investment plan again according to the new prompt words obtained after the modification, and input this plan and the new prompt words into GPT4 again, so that GPT4 can make a new score for it. Determine whether the overall score has improved. If it has improved and neither the maximum iteration round nor the maximum threshold score has been reached, the iteration will enter the second round. Otherwise, determine whether the termination condition has been met. If so, terminate.

[0077] Second round: First, let the big model propose modification suggestions and modify the prompt words based on GPT4's analysis. Then, let the big model formulate an investment plan based on the modified prompt words. Finally, input the new prompt words and the formulated investment plan into GPT4 and let it score each execution rule. Determine whether the overall score has improved. If not, let the big model revise it again and determine whether the termination conditions have been met and whether to enter the third round of iteration.

[0078] Round 3: ...

[0079] In this way, we can get the best prompt words and formulate the best investment plan.

[0080] When comparing this solution with the control solution in the context of providing standard answers, the maximum number of iterations is set to 11, the number of boundary attempts is set to 3, and the maximum threshold score is set to 0.9. If the results do not improve or the effect decreases after more than three attempts, or the maximum number of iterations or the maximum threshold score is reached, the test is terminated.

[0081] Corresponding to Experiment 1 Experimental process:

[0082] Round 1:

[0083] First, the original prompt words "Select a suitable investment plan based on the customer's financial situation, investment goals and risk preferences; ensure that the selected investment plan matches the customer's long-term financial planning and life goals; the selected investment plan should include consideration of market dynamics and economic indicators to achieve diversified asset allocation; based on the customer's investment goals, select a plan that matches the customer's expected investment return and time frame; formulate a flexible and adjustable investment plan based on the customer's personalized financial situation and liquidity needs" and customer characteristic data (for example, user 1: "age": "30", "gender": "male", "annual income": "100000", "Occupation": "Programmer", "Marital status": "Single", "Do you have a personal house": "None", "Risk tolerance": "Medium", "Investment goal": "Long-term appreciation") are input into the big model, and the big model selects an investment plan for each customer (including: A: 90% stock investment, 10% bond investment; B: 70% stock investment, 30% bond investment; C: 50% stock investment, 50% bond investment; D: 30% stock investment, 70% bond investment; E: 10% stock investment, 90% bond investment.), and compares it with the customer's actual choice to get the recommendation accuracy. The original prompt word, investment plan selection, accuracy, and the customer's actual choice are input into GPT4 together, and GPT4 scores each execution rule of the original prompt word and gives corresponding analysis (for example: Execution rule 1: [Score: 85; Analysis: Most users' investment plans are consistent with their risk tolerance and investment goals, but some users' plan selections are not accurate enough.]), and then the analysis is passed to GPT4 to make optimization suggestions.

[0084] Next, we enter the modification process of the execution rules. Based on the analysis and optimization suggestions of GPT4's feedback, the big model proposes three modification suggestions and modifies the prompt words three times. The modification with the highest accuracy is taken as the effective modification, and the other two are discarded. In this process, the modification with low scores or no improvement in accuracy is discarded to increase the number of explorations. The score and analysis given by GPT4 for each execution rule are input into the big model, and the big model is allowed to modify the prompt words on its own. The requirement is that only the execution rule with the lowest GPT score can be modified.

[0085] Finally, let the big model recommend investment plans to customers again based on the new prompt words obtained through effective modification, and obtain the accuracy rate. Determine whether the accuracy rate has been improved. If it has been improved and has not reached the maximum iteration round or the maximum threshold score, the iteration will enter the second round. Otherwise, determine whether the termination condition has been met. If so, terminate.

[0086] Second round: First, input the new prompt words, the investment plan selection given by the big model, the accuracy rate, and the actual choice of the customer into GPT4, and let GPT4 score each execution rule of the new prompt words and give corresponding analysis. Next, let the big model propose modification opinions and modify the prompt words according to GPT4's analysis, and test the modification. Take one of the three modifications (the one with the highest accuracy rate) as the effective modification. If the accuracy rate does not improve, return to modify again until it improves or reaches the maximum number of attempts of 3 times. Finally, whether the termination condition is met, determine whether to enter the third round of iteration.

[0087] Round 3: ...

[0088] Repeat this process to obtain the optimal prompt word and recommendation accuracy.

[0089] As a possible implementation, this embodiment provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, a large model automatic prompt tuning method using gradient optimization is implemented.

[0090] As a possible implementation, this embodiment provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a large model automatic prompt tuning method using gradient optimization.

[0091] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. Automatically suggest tuning method using large model with gradient optimization, characterized by: The specific steps include: Initialization parameter settings; Obtain the execution rules in each test data prompt word and the corresponding question-answering model output answer; Construct an evaluation model to score the execution rules in each test data prompt word and the corresponding question-answer model output answer, and obtain the total evaluation score of the prompt word and the evaluation score of each execution rule; Perform gradient descent optimization on the total evaluation score of the prompt word and the evaluation score of each execution rule: Get the average score corresponding to the prompt word of the current round, and determine whether the current average score and round meet the set threshold; If not, the prompt word of the current round is output as the optimal prompt word; If it is satisfied, the optimization direction of the best prompt word is selected, the optimization direction is refined, and a new prompt word is generated based on the refined optimization direction. This step is repeated until the set threshold is met, the optimization round is stopped, and the best prompt word is output.

2. The large model automatic prompt tuning method using gradient optimization according to claim 1 is characterized in that: The initialization parameter settings specifically include: initializing the current round, the maximum iteration round, the initial prompt word effect score, the target threshold, the step size of each optimization, the number of boundary attempts, and the initial prompt word.

3. The large model automatic prompt tuning method using gradient optimization according to claim 2 is characterized in that: The target threshold is used to describe the effect score that the user expects the prompt word to achieve, the step size is used to indicate the number of words added, deleted or modified each time, the number of boundary attempts is used to indicate how many rounds the score remains unchanged and reaches the model upper limit, and the initial prompt words include background introduction, task description, execution rules and task body.

4. The large model automatic prompt tuning method using gradient optimization according to claim 2 is characterized in that: When the evaluation model is scored, the evaluation model scores according to the answer output by the question-answering model and the execution rules in the prompt word. The scoring process includes: calculating the total score of each test data under the prompt word, and taking the average of n data, and obtaining the total evaluation score of the evaluation model for the prompt word and the evaluation score of each rule based on the average of the n data.

5. The large model automatic prompt tuning method using gradient optimization according to claim 1 is characterized in that: When the evaluation model scores the execution rules in each test data prompt word and the corresponding model output answer, it also includes: Determine whether the average score of the current round of evaluation scores and the round meet the set threshold. If the average score does not change for k consecutive rounds, it is determined that the optimal boundary of the model has been reached, and the prompt word of the current round is output as the optimal prompt word.

6. The large model automatic prompt tuning method using gradient optimization according to claim 1 is characterized in that: The selecting the optimization direction of the best prompt word specifically includes: obtaining all rules of the current prompt word, generating multiple forward optimization spaces according to all the rules, and selecting the rule with the lowest score among all the rules as the current optimization direction.

7. The large model automatic prompt tuning method using gradient optimization according to claim 6 is characterized in that: The refinement and optimization directions specifically include: Move forward according to the current optimization direction, and obtain the number of words that can be modified each time as the step length of each step; Perform score analysis on the prompt words corresponding to the rule with the lowest score; Select the three test data with the lowest scores of the rule and the corresponding answers as examples; The evaluation model performs analysis based on the examples, and the question-answering model optimizes the step size of the rule based on the analysis results to generate N new prompt words.

8. The large model automatic prompt tuning method using gradient optimization according to claim 1 is characterized in that: Each time the step size of the rule is optimized, only the rule with the lowest average score in the prompt is modified.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the large model automatic prompt tuning method using gradient optimization as described in any one of claims 1 to 8 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the large model automatic prompt tuning method using gradient optimization as described in any one of claims 1 to 8 is implemented.

Images