Large language model cue word automatic optimization method

Abstract

The invention discloses an automatic optimization method for cue words of a large language model, and relates to the technical field of cue word optimization. Comprises: setting an initial cue word; constructing training data, gradient generation cue word templates, cue word editing templates, optimizers and task models; randomly sampling a small batch of data from the training data as a training set, and predicting a sample of the training set by the task model according to the initial cue word to obtain an answer; comparing the answer with a sample label of a training set to obtain an error example set; inputting the error example set and the initial cue word into a gradient generation cue word template, and generating a gradient analysis request; the optimizer model receives and analyzes the gradient analysis request to generate a second natural language gradient; and inputting the second natural language gradient and the initial cue word into a cue word editing template, and modifying the initial cue word by the optimizer model according to the cue word editing template to generate a second candidate cue word. The technical problem that the efficiency of manually writing prompt words is low is solved.

Description

Technical Field

[0001] The present invention belongs to the technical field of prompt word optimization, and in particular relates to a method for automatically optimizing prompt words of a large language model. Background Art

[0002] With the widespread adoption of large language models (LLMs) in natural language processing, their performance has become heavily dependent on input prompts. Traditional prompt design relies primarily on manual trial and error, which is time-consuming and labor-intensive, and difficult to guarantee optimality. In recent years, automated prompt optimization technology has emerged, aiming to improve model performance on specific tasks through automated methods for prompt generation and optimization. Existing techniques primarily utilize reinforcement learning or gradient-based methods for prompt optimization. However, these methods have the following problems: high resource consumption: large language models are involved in the entire optimization and reasoning process, resulting in high computational costs; poor generalization ability: the prompt word optimization process lacks effective control over the diversity and semantic consistency of the prompt words, and the optimization process is prone to falling into local optimality. The prompt words are overly dependent on the data examples for training the current task, resulting in overfitting or unclear optimization effects; insufficient task adaptation: simple tasks and complex tasks have different requirements for model capabilities, and a single model is difficult to balance efficiency and accuracy; manual writing is inefficient and the effect is unstable: manual writing of prompt words relies on manual experience, which is inefficient and difficult to adapt to complex and changing task requirements. Different tasks have different requirements for prompt words, and manual adjustment makes it difficult to quickly find the optimal prompt words, resulting in the inability to fully utilize the model performance.

[0003] Existing optimization methods suffer from the following drawbacks: Optimization techniques that rely on the LLM's internal state cannot be used in scenarios where API-based LLM services are requested; reinforcement learning methods are complex to train and lack interpretability; and methods based on Monte Carlo search and other methods lack semantic guidance, are prone to falling into local optima, and struggle to effectively improve prompt word quality. Difficulty adapting to complex task scenarios: Existing prompt word optimization methods struggle to adapt to the diversity and complexity of complex tasks such as instruction following, code generation, and mathematical reasoning, and are unable to effectively improve LLM performance in these scenarios. The direction of prompt word optimization is difficult to control: During prompt word optimization, prompt words tend to be optimized toward containing more correct examples, which can lead to overfitting and reduce the model's generalization ability. Currently, there is a lack of effective methods to control this optimization direction. Model selection lacks specificity: When selecting models for prompt word optimization and task execution, the performance differences between large and small models at different task complexities are not fully considered. Using large models for simple tasks wastes resources, while using small models for complex tasks may not meet the task requirements, resulting in inefficient resource utilization and poor overall optimization results. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a method for automatically optimizing prompt words of a large language model to at least solve some of the above technical problems.

[0005] To achieve the above object, the technical solution adopted by the present invention is as follows: A method for automatically optimizing prompt words of a large language model includes the following steps: S1. Set a prompt word as the initial prompt word; build training data, gradient generation prompt word template, prompt word editing template, optimizer model and task model; S2. Randomly sample small batches of data from the training data as the training set. The task model predicts the answers to the samples in the training set based on the initial prompt words. The answers are compared with the sample labels in the training set one by one to identify examples with incorrect predictions. The examples with incorrect predictions are collected to form an error example set. S3, inputting the error example set and the initial prompt word into the gradient generation prompt word template to generate a gradient analysis request; the optimizer model receives and analyzes the gradient analysis request to generate a second natural language gradient; S4. Input the second natural language gradient and the initial prompt word into the prompt word editing template. The optimizer model modifies the initial prompt word according to the editing prompt word template to generate a second candidate prompt word.

[0006] Furthermore, it also includes: S5. Using a beam search algorithm to screen the second candidate prompt word to obtain an optimized prompt word; S6. Use the task model to evaluate the optimized prompt words on a pre-built test set to determine whether the optimized prompt words are effective; S7. Determine whether the optimized prompt word meets the termination condition. If the termination condition is met, output the optimized prompt word. If the termination condition is not met, repeat steps S2-S7 using the optimized prompt word as the initial prompt word.

[0007] Furthermore, in S1, the optimizer model is a 70B large language model, and the task model is a 7B large language model.

[0008] Furthermore, S3 includes: S31, inputting the error example set and the initial prompt word into a gradient generation prompt word template to generate a gradient analysis request; the gradient generation prompt word template includes the following structural requirements: the initial prompt word, specific samples of the error example set, the number of error reasons of the error examples, and fixed format output constraints; S32: The optimizer model receives the gradient analysis request and performs the following analysis: identifying common features in error examples, analyzing the causal relationship between the initial prompt word and the error type, and diagnosing the specific defect of the initial prompt word, thereby generating a first natural language gradient containing multiple lines of natural language text. The first natural language gradient includes: defect type classification, specific defect description, relevant error examples, and improvement suggestions. S33. According to the fixed-format output constraint in the gradient generation prompt word template, a regular expression is used to extract, from the first natural language gradient, the same number of improvement direction suggestions as the number of error causes of the error examples as the second natural language gradient.

[0009] Furthermore, S4 includes: S41, inputting the second natural language gradient and the initial prompt word into the prompt word editing template; S42, integrating elements of the prompt word editing template, including: an initial prompt word, a set of error examples, a second natural language gradient, and the number of candidate prompt words to be generated; S43, setting output format constraints in the prompt word editing template; S44, the optimizer model modifies the initial prompt word along the direction of the second natural language gradient according to the prompt word editing template guidance to generate a first candidate prompt word; S45 . According to the output format constraint in the prompt word editing template, use a regular expression to extract from the first candidate prompt words the same number of candidate prompt words as the number of candidate prompt words to be generated as the second candidate prompt words.

[0010] Furthermore, S5 includes: S51, the task model uses the second candidate prompt word to predict all samples in the test set to obtain the first prediction answer of the task model; S52: Compare the first predicted answer with the sample labels of the test set one by one and calculate the performance index to obtain a first performance index; S53 , using a beam search algorithm to select second candidate prompt words with the best performance index and a number equal to a preset beam width as optimized prompt words.

[0011] Furthermore, S6 includes: S61, using the task model to predict all samples of the optimized prompt word in the test set to obtain a second prediction answer of the task model; S62, comparing the second predicted answer of the task model with the sample labels of the test set one by one and calculating the performance index to obtain a second performance index; S63: The task model compares and evaluates the first performance indicator and the second performance indicator. If the second performance indicator is greater than or equal to the first performance indicator, the optimization prompt word is valid; otherwise, the optimization prompt word is invalid.

[0012] Furthermore, S7 includes: S71. If the optimized prompt word is valid, if the second performance indicator reaches the preset threshold or the second performance indicator does not decrease relative to the first performance indicator in five consecutive iterations, then the optimized prompt word is output; otherwise, steps S2-S7 are repeated using the optimized prompt word as the initial prompt word; S72: If the optimized prompt word is invalid, repeat steps S2-S7 using the optimized prompt word as the initial prompt word.

[0013] Compared with the prior art, the present invention has the following beneficial effects: The present invention is scientifically and rationally designed and easy to use. By combining a gradient-generated prompt word template and an optimizer model, the present invention generates a second natural language gradient, accurately points out problems with the prompt word, and makes the improvement of the prompt word more targeted and semantically reasonable. Furthermore, by combining the editing prompt word template, the optimizer model, and the generated second natural language gradient, the present invention generates a second candidate prompt word, thus solving the technical problem of low efficiency in manually writing prompt words.

[0014] The present invention uses a beam search algorithm and combines the task model to evaluate the performance indicators of the second candidate prompt words on the test set, effectively screening the second candidate prompt words, improving the search efficiency and reducing unnecessary computing resource consumption while ensuring the optimization effect.

[0015] This paper adopts the 70B large language model as the optimizer, leveraging its powerful language understanding and analysis capabilities to generate high-quality optimization suggestions; the 7B model is used as the task model, leveraging its efficiency and cost-effectiveness in simple tasks to quickly evaluate the effect of prompt words, realizing the complementary advantages of large and small models and reducing computing resource consumption.

[0016] By limiting the number of candidate prompt words in the prompt word editing template, the present invention effectively avoids the prompt words from excessively including examples, guides them to select typical examples, enhances the generalization ability of the prompt words, and ensures that the model can perform stably on different data. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a flow chart of the steps of the present invention. DETAILED DESCRIPTION

[0018] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0019] In the description of the present invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings and are intended solely to facilitate and simplify the description of the present invention. They are not intended to indicate or imply that the devices or components referred to must have a specific orientation or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the present invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0020] In the description of the present invention, it should be noted that, unless otherwise expressly specified or limited, the terms "mounted," "connected," and "connected" should be understood in a broad sense. For example, they may refer to fixed connections, detachable connections, or integral connections; of course, they may also refer to mechanical connections or electrical connections; in addition, they may refer to direct connections, indirect connections through an intermediate medium, or internal connections between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on specific circumstances.

[0021] like Figure 1 As shown, a method for automatically optimizing prompt words of a large language model includes the following steps: S1. Set a prompt word as the initial prompt word; build training data, gradient generation prompt word template, prompt word editing template, optimizer model and task model; S2. Randomly sample small batches of data from the training data as the training set. The task model predicts the answers to the samples in the training set based on the initial prompt words. The answers are compared with the sample labels in the training set one by one to identify examples with incorrect predictions. The examples with incorrect predictions are collected to form an error example set. S3, inputting the error example set and the initial prompt word into the gradient generation prompt word template to generate a gradient analysis request; the optimizer model receives and analyzes the gradient analysis request to generate a second natural language gradient; S4. Input the second natural language gradient and the initial prompt word into the prompt word editing template. The optimizer model modifies the initial prompt word according to the editing prompt word template to generate a second candidate prompt word.

[0022] An example of a gradient-generated prompt template for a zero-shot classification task is as follows: I'm trying to write a zero-shot classifier on hint words.

[0023] My current prompt words are: {prompt} But this prompt fails when processing the following example: {error_samples} Please give {num_feedbacks} reasons why this prompt word may be wrong, and use <start>and <end>Wrap every reason.

[0024] An example of a second natural language gradient is as follows: 1. The prompt words lack description of the key features of the text; 2. The prompt word instructions are unclear; 3. The prompt words do not clearly explain the task; 4. The prompt words lack effective examples; 5. The prompt words do not effectively constrain the output of the task model.

[0025] The example content of the prompt word editing template for the zero-shot classification task is as follows: I'm trying to write a zero-shot classifier; My current prompt words are: {prompt} But it gives an error when processing the following example: {error_samples} Based on these examples, the problem with this prompt word is {gradient}; Based on the above information, I wrote {steps_per_gradient} different improvement hints, each hint is used <start>and <end>pack; Note: To make the model output more accurate, the new prompt words should not contain incorrect examples and their correct answers, and ensure that the number of examples in the new prompt words (if any) does not increase to avoid overfitting; {steps_per_gradient} new hints are: This embodiment combines a gradient-generated prompt word template with an optimizer model to generate a second natural language gradient, accurately identifying problems with the prompt word and making the prompt word improvement more targeted and semantically reasonable. Furthermore, by combining editing the prompt word template with the optimizer model and generating the second natural language gradient, a second candidate prompt word is generated, addressing the technical issue of low efficiency in manually writing prompt words.

[0026] The present invention is based on a large language model prompt word automatic optimization method, wherein one embodiment further includes: S5. Using a beam search algorithm to screen the second candidate prompt word to obtain an optimized prompt word; S6. Use the task model to evaluate the optimized prompt words on the test set to determine whether the optimized prompt words are effective; S7. Determine whether the optimized prompt word meets the termination condition. If the termination condition is met, output the optimized prompt word. If the termination condition is not met, repeat steps S2-S7 using the optimized prompt word as the initial prompt word.

[0027] This embodiment uses a beam search algorithm to quickly find a relatively better prompt word among a large number of prompt word candidates, rather than performing a second round of iteration among all prompt words in a greedy search, thereby improving optimization efficiency and reducing computing resource consumption.

[0028] The present invention is based on a large language model prompt word automatic optimization method. In one embodiment, in S1, the optimizer model is a 70B large language model, and the task model is a 7B large language model.

[0029] This embodiment uses a 70B large language model as the optimizer, leveraging its powerful language understanding and analysis capabilities to generate high-quality optimization suggestions. The 7B model is used as the task model, leveraging its efficiency and cost-effectiveness in simple tasks to quickly evaluate the effectiveness of prompt words, achieving complementary advantages between the large and small models and reducing computing resource consumption.

[0030] Preferably, S3 includes: S31, inputting the error example set and the initial prompt word into a gradient generation prompt word template to generate a gradient analysis request; the gradient generation prompt word template includes the following structural requirements: the initial prompt word, specific samples of the error example set, the number of error reasons of the error examples, and fixed format output constraints; S32: The optimizer model receives the gradient analysis request and performs the following analysis: identifying common features in error examples, analyzing the causal relationship between the initial prompt word and the error type, and diagnosing the specific defect of the initial prompt word, thereby generating a first natural language gradient containing multiple lines of natural language text. The first natural language gradient includes: defect type classification, specific defect description, relevant error examples, and improvement suggestions. S33. According to the fixed-format output constraint in the gradient generation prompt word template, use a regular expression to extract, from the first natural language gradient, the same number of improvement direction suggestions as the number of error examples and error causes in the gradient generation prompt word template as the second natural language gradient.

[0031] The present invention is based on a large language model prompt word automatic optimization method. In one embodiment, S4 includes: S41, inputting the second natural language gradient and the initial prompt word into the prompt word editing template; S42, integrating elements of the prompt word editing template, including: an initial prompt word, a set of error examples, a second natural language gradient, and the number of candidate prompt words to be generated; S43, setting output format constraints in the prompt word editing template; S44, the optimizer model modifies the initial prompt word along the direction of the second natural language gradient according to the prompt word editing template guidance to generate a first candidate prompt word; S45 . According to the output format constraint in the prompt word editing template, use a regular expression to extract from the first candidate prompt words the same number of candidate prompt words as the number of candidate prompt words to be generated in the prompt word editing template as the second candidate prompt words.

[0032] For its own dataset, the prompt words will be optimized towards containing more correct examples. When iterating the prompt words, the optimizer tends to put the incorrect examples input to it and the corresponding correct answers into the optimized prompt words. This is equivalent to telling the task model as many examples as possible, which is a disguised way of telling the task model the answers, which is a cheating behavior. The number of examples once reached 260. This situation will be more obvious for small model combinations (such as 7B-7B). As the number of iterations increases, this phenomenon will become more serious. This is because the optimizer hopes that the task model can answer as many correct answers as possible to highlight its own status and role. Therefore, it will intentionally or unintentionally integrate the answer examples of the training set into the optimized prompt words.

[0033] This embodiment effectively prevents prompt words from excessively including examples by limiting the number of candidate prompt words in the prompt word editing template and not adding examples (zero samples) in the prompt word editing template. This guides the prompt word to select typical examples, enhances the generalization ability of the prompt word, and ensures that the model can perform stably on different data.

[0034] The present invention is based on a large language model prompt word automatic optimization method. In one embodiment, S5 includes: S51, the task model uses the second candidate prompt word to predict all samples in the test set to obtain the first prediction answer of the task model; S52: Compare the first predicted answer with the sample labels of the test set one by one and calculate the performance index to obtain a first performance index; S53: Use a beam search algorithm to select the second candidate prompt words with the best performance index and a number equal to the beam width as the optimized prompt words.

[0035] Among them, the performance indicator is accuracy or F1 value.

[0036] The present invention uses a beam search algorithm and combines the task model to evaluate the performance indicators of the second candidate prompt words on the test set, effectively screening the second candidate prompt words, improving the search efficiency and reducing unnecessary computing resource consumption while ensuring the optimization effect.

[0037] The present invention is based on a large language model prompt word automatic optimization method. In one embodiment, S6 includes: S61, using the task model to predict all samples of the optimized prompt word in the test set to obtain a second prediction answer of the task model; S62, comparing the second predicted answer of the task model with the sample labels of the test set one by one and calculating the performance index to obtain a second performance index; S63: The task model compares and evaluates the first performance indicator and the second performance indicator. If the second performance indicator is greater than or equal to the first performance indicator, the optimization prompt word is valid; otherwise, the optimization prompt word is invalid.

[0038] Among them, the performance indicator is accuracy or F1 value.

[0039] The task model has high computational efficiency and cost-effectiveness in simple tasks and can quickly provide evaluation results. For example, in text sentiment analysis tasks, the task model can accurately analyze and judge the emotional tendency of the text, and thus calculate performance indicators based on the evaluation results. If the task is more complex, such as generating feedback for prompt words based on incorrect examples, the task model may not be able to complete the task accurately. This is because the task model has more complex semantic relationships and long-distance dependencies due to differences in parameter scale, as well as disadvantages when handling tasks that require deep logical reasoning and complex contextual understanding. On the other hand, it is unstable in multi-task learning and domain transfer, and has poorer adaptability to unseen complex tasks. This is where the advantages of the optimization model in complex task processing are reflected. Its advantages in parameter scale and knowledge capacity enable it to perform better in tasks such as complex reasoning, multimodal interaction, long text processing, and high-precision question and answer.

[0040] In this embodiment, the task model is mainly used to preliminarily evaluate the effect of optimizing prompt words and provide feedback for iterative optimization.

[0041] Preferably, S7 includes: S71. If the optimized prompt word is valid, if the second performance indicator reaches the preset threshold or the second performance indicator does not decrease relative to the first performance indicator in five consecutive iterations, then the optimized prompt word is output; otherwise, steps S2-S7 are repeated using the optimized prompt word as the initial prompt word; S72: If the optimized prompt word is invalid, repeat steps S2-S7 using the optimized prompt word as the initial prompt word.

[0042] Finally, it should be noted that the above embodiments are merely preferred embodiments of the present invention and are intended to illustrate the technical solutions of the present invention, rather than limiting them, and certainly not limiting the patent scope of the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some or all of the technical features therein may be replaced by equivalents. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present invention. In other words, any changes or refinements made to the main design concept and spirit of the present invention that have no substantive significance, provided that the technical problems they solve are still consistent with those of the present invention, should be included in the protection scope of the present invention. In addition, the direct or indirect application of the technical solutions of the present invention to other related technical fields should also be included in the patent protection scope of the present invention.< / end> < / start> < / end> < / start>

Claims

1. A method for automatically optimizing prompt words of a large language model, characterized in that: The steps include: S1. Set a prompt word as the initial prompt word; build training data, gradient generation prompt word template, prompt word editing template, optimizer model and task model; S2. Randomly sample small batches of data from the training data as the training set. The task model predicts the answers to the samples in the training set based on the initial prompt words. The answers are compared with the sample labels in the training set one by one to identify examples with incorrect predictions. Collect examples of prediction errors to form an error example set; S3, inputting the error example set and the initial prompt word into the gradient generation prompt word template to generate a gradient analysis request; the optimizer model receives and analyzes the gradient analysis request to generate a second natural language gradient; S4. Input the second natural language gradient and the initial prompt word into the prompt word editing template. The optimizer model modifies the initial prompt word according to the editing prompt word template to generate a second candidate prompt word.

2. The automatic optimization method for prompt words of a large language model according to claim 1, characterized in that: Also includes: S5. Using a beam search algorithm to screen candidate prompt words to obtain optimized prompt words; S6. Use the task model to evaluate the optimized prompt words on a pre-built test set to determine whether the optimized prompt words are effective; S7, determining whether the optimization prompt word satisfies the termination condition; If the termination condition is met, the optimization prompt word is output; If the termination condition is not met, the optimized prompt word is used as the initial prompt word and steps S2-S7 are repeated.

3. The automatic optimization method for prompt words of a large language model according to claim 2, characterized in that: In S1, the optimizer model is a 70B large language model, and the task model is a 7B large language model.

4. The method for automatically optimizing the prompt words of a large language model according to claim 1, wherein S3 include: S31, inputting the error example set and the initial prompt word into a gradient generation prompt word template to generate a gradient analysis request; the gradient generation prompt word template includes the following structural requirements: the initial prompt word, specific samples of the error example set, the number of error reasons of the error examples, and fixed format output constraints; S32. The optimizer model receives the gradient analysis request and performs the following analysis: the optimizer model identifies common features of specific samples in the error example set and analyzes the causal relationship between the initial prompt word and the common features; the optimizer model diagnoses the specific defects in the initial prompt word and generates a first natural language gradient containing multiple lines of natural language text; the first natural language gradient includes: defect type classification, specific defect description, relevant error examples, and improvement suggestions; S33. According to the fixed-format output constraint in the gradient generation prompt word template, a regular expression is used to extract, from the first natural language gradient, the same number of improvement direction suggestions as the number of error causes of the error examples as the second natural language gradient.

5. The method for automatically optimizing prompt words of a large language model according to claim 1, wherein S4 include: S41, inputting the second natural language gradient and the initial prompt word into the prompt word editing template; S42, integrating elements of the prompt word editing template, including: an initial prompt word, a set of error examples, a second natural language gradient, and the number of candidate prompt words to be generated; S43, setting output format constraints in the prompt word editing template; S44, the optimizer model modifies the initial prompt word along the direction of the second natural language gradient according to the prompt word editing template guidance to generate a first candidate prompt word; S45 . According to the output format constraint in the prompt word editing template, use a regular expression to extract from the first candidate prompt words the same number of candidate prompt words as the number of candidate prompt words to be generated as the second candidate prompt words.

6. The method for automatically optimizing prompt words of a large language model according to claim 2, wherein S5 include: S51, the task model uses the second candidate prompt word to predict all samples in the test set to obtain the first prediction answer of the task model; S52: Compare the first predicted answer with the sample labels of the test set one by one and calculate the performance index to obtain a first performance index; S53 , using a beam search algorithm to select second candidate prompt words with the best performance index and a number equal to a preset beam width as optimized prompt words.

7. The method for automatically optimizing prompt words of a large language model according to claim 6, wherein S6 include: S61, using the task model to predict all samples of the optimized prompt word in the test set to obtain a second prediction answer of the task model; S62, comparing the second predicted answer of the task model with the sample labels of the test set one by one and calculating the performance index to obtain a second performance index; S63: The task model compares and evaluates the first performance indicator and the second performance indicator. If the second performance indicator is greater than or equal to the first performance indicator, the optimization prompt word is valid; otherwise, the optimization prompt word is invalid.

8. The method for automatically optimizing prompt words of a large language model according to claim 7, characterized in that: S7 includes: S71: If the optimization prompt word is valid, if the second performance indicator reaches a preset threshold or the second performance indicator does not decrease relative to the first performance indicator in five consecutive iterations, then output the optimization prompt word; Otherwise, repeat steps S2-S7 using the optimized prompt word as the initial prompt word; S72: If the optimized prompt word is invalid, repeat steps S2-S7 using the optimized prompt word as the initial prompt word.

Images