Prompt template automatic optimization method and device and storage medium
By iteratively querying the inference API and random linear projection technology of large language models, the prompt template of large language models is optimized, which solves the problem of prompt template design in the existing technology, and realizes automatic optimization of prompt templates without fine-tuning the model, improving the effect of task completion.
Patent Information
- Application Number
- CN202411923207.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-23
- Publication Date
- 2025-04-25
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
In the prior art, it is difficult to design effective prompt templates to guide large language models to complete downstream tasks, especially when large language models are not fine-tuned, how to automatically optimize prompt templates to improve the effect is a common problem.
By iteratively querying the inference API of large language models, random linear projection is used to project the space of the original prompt template to a smaller subspace, and optimize the continuous prompt in this subspace with a derivativeless optimizer, thereby optimizing the prompt template.
It realizes that the prompt template is automatically optimized without fine-tuning the large language model, which improves the effect of the prompt template, expands the scope of use of closed-source large language models, and improves the utilization value.
Smart Images
Figure CN119886092A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a prompt template automatic optimization method in the field of large language models, and in particular to a prompt template automatic optimization method, device and storage medium. Background Art
[0002] In recent years, large language models (LLMs) have become the mainstream technology in the field of natural language processing. Based on the self-attention mechanism with powerful fitting ability and massive text training data, large language models such as GPT3 and LLaMa can learn the extensive semantic associations contained in the language, and have achieved good results in many natural language processing sub-fields such as intelligent question and answer, machine reading, and text summarization.
[0003] Due to the complexity of downstream natural language processing tasks, when dealing with many new tasks, fine-tuning based on existing large language models may achieve better technical results, but it also requires a large number of samples and powerful computing resources. Existing large models such as GPT3 require dozens of graphics cards to load the model. At the same time, due to reasons such as commercial confidentiality protection, many large language model R&D teams no longer disclose the parameters of their models. The public can only interact with the large language model by accessing the Application Programming Interface (API). The above restrictions make it impossible for anyone except the model R&D team to fine-tune the model.
[0004] Many documents show that the potential of existing large language models themselves is still to be explored, and prompt learning technology can use the rich semantic knowledge already contained in the model itself without fine-tuning the large language model, and only rely on prompt templates to guide the model to complete downstream tasks. Unlike the training phase learning that requires the use of reverse gradient to update model parameters, prompt learning does not need to update the parameters of the large language model it uses, and directly makes predictions based on the existing capabilities of the large language model.
[0005] In practice, technicians need to design different prompt templates according to different tasks, so as to induce the large language model to generate responses that meet user expectations as much as possible. How to design effective prompt templates is still a common problem faced by the industry. Although it is not difficult to manually write prompt templates, it is unrealistic to use one template to solve all problems. The prompt templates designed by technicians with different experiences will have very different effects in practice. Therefore, the industry needs to study how to automatically optimize the prompt template to obtain a prompt template with better effect. Summary of the invention
[0006] The embodiments of the present invention provide an automatic prompt template method, device, electronic device and storage medium, wherein the continuous prompts in front of the task prompt template are optimized by iteratively querying the reasoning API of a large language model, and then the prompt template is optimized. In order to process high-dimensional continuous prompt vectors, a random linear projection is used to project the space of the original prompt template onto a smaller subspace, and then a derivative-free optimizer is used in this smaller subspace to solve the optimization problem.
[0007] To achieve the above-mentioned purpose, a first aspect of an embodiment of the present application proposes a prompt template automatic optimization method for optimizing an original task prompt template for completing a task. The method comprises the following steps:
[0008] S100) modifying the original task prompt template to obtain a modified task prompt template;
[0009] S200) using random linear projection to project the space of the original task prompt template onto a subspace smaller than the space of the original task prompt template, and using a derivative-free optimizer in the subspace to obtain an optimized continuous prompt; and
[0010] S300) Combining the optimized continuous prompt with the modified task prompt template to obtain an optimized prompt template.
[0011] In some embodiments, step S200 includes the following steps:
[0012] S210) obtaining an initial prompt p0 of the task;
[0013] S220) Find the optimal solution space z * ;and
[0014] S230) obtaining the optimization continuous prompt through p0+Az*;
[0015] In step S220, a random projection matrix A∈R is used. D×d Projecting on the space of the original task prompt template obtains the subspace z(z∈R d ), wherein the value of the random projection matrix A is set by sampling from a uniform distribution, wherein D is the dimension value of the space of the original task prompt template, and wherein d is the dimension value of the subspace.
[0016] In some embodiments, step S220 uses a covariance matrix adaptive evolution strategy to optimize to obtain the best z*, where where F is the API for large language model inference and L is the optimized loss function.
[0017] In some embodiments, the optimized loss function L = αLhinge +βL CE , where α and β are coefficients, α+β=1, where L CE (Y p ,Y g )=-logsoftmax Yg (Y p ), Where the output Y of the given large language model on the candidate label set of the task p and the true label word Y for a specific sample g , where r is a preset parameter, and the value of r is 2.
[0018] In some embodiments, step S210 includes the following steps:
[0019] S211) selecting a plurality of matching task prompt templates to match the task;
[0020] S212) decomposing the words corresponding to the plurality of task prompt templates to obtain a word set;
[0021] S213) excluding nouns and verbs in the word set to obtain a screening word combination;
[0022] S214) obtaining an average word vector of the selected word combination; and
[0023] S215) Extract multiple characters that are closest to the average word vector cosine similarity of the filtered word combination as the initial prompt p0.
[0024] In some embodiments, in step S215, n characters closest to the average word vector cosine similarity of the filter word combination are extracted as the initial prompt p0, where the value range of n is [the number of elements of the filter word combination, 3*the number of elements of the filter word combination].
[0025] In some embodiments, the size of the smaller subspace is 0.07 to 0.13 times the size of the space of the original hint template.
[0026] In some embodiments, the size of the smaller subspace is 0.1 times the size of the space of the original hint template.
[0027] To achieve the above-mentioned purpose, the second aspect of an embodiment of the present application proposes an electronic device, which includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the method described in the first aspect is implemented.
[0028] To achieve the above-mentioned purpose, a third aspect of an embodiment of the present application proposes a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described in the first aspect is implemented.
[0029] The prompt template automatic optimization method, device, electronic device and storage medium proposed in this application are different from the traditional fine-tuning method that can only be executed on the server side where the large language model is deployed. The method allows users to perform local optimization without mastering the parameters of the large language model to meet their own specific needs, expand the scope of use of closed-source large language models, and improve their utilization value. BRIEF DESCRIPTION OF THE DRAWINGS
[0030] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying any creative work.
[0031] Figure 1 is an application diagram of a prompt template automatic optimization method according to a preferred embodiment of the present invention, wherein the prompt template is optimized by iteratively querying the reasoning API of a large language model.
[0032] Figure 2 It is a flow chart of the prompt template automatic optimization method according to the above preferred embodiment of the present invention.
[0033] Figure 3 It is a flow chart of optimizing continuous prompts in the prompt template automatic optimization method according to the above preferred embodiment of the present invention.
[0034] Figure 4 It is a flowchart of obtaining an initial prompt for a task in the method for optimizing continuous prompts in the prompt template automatic optimization method according to the above preferred embodiment of the present invention.
[0035] Figure 5 It is a flowchart of finding the optimal solution space in the method of optimizing continuous prompts in the prompt template automatic optimization method according to the above preferred embodiment of the present invention.
[0036] Figure 6 The system architecture for optimizing prompt templates is explained. DETAILED DESCRIPTION
[0037] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.
[0038] It should be noted that, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification, claims and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.
[0039] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.
[0040] First, some nouns involved in this application are analyzed:
[0041] BERT stands for Bidirectional Encoder Representation from Transformers, which is a pre-trained language representation model.
[0042] Suppose the task prompt template for completing a specific task (such as sentiment classification) is X, modify X (for example, add some trigger words and a special symbol [MASK], and map the task corresponding label Y to some words in the large language model vocabulary (for example, the labels "positive", "neutral", and "negative" of sentiment classification can be mapped to "excellent", "okay", and "boring"). Let the modified task prompt template and label be represented as Xm and Yg. For example, the original task prompt template X is: Transformers is a movie full of science fiction and fierce fighting, and the modified task prompt templates Xm and Yg are: Transformers is a movie full of science fiction and fierce fighting, it really [MASK].
[0043] Assume that the application programming interface (API) used for large language model inference is F, concatenate the continuous prompt p (a vector) and a batch_size Xm together and input them into F. The large language model returns the result, which is the output vector at the [MASK] position, that is, Yp = F (p: Xm), where : represents the concatenation operation. Usually, the traditional gradient descent optimization method can be used to calculate the loss of this batch of data through Yp to find the best prompt, but on the one hand, these data are not necessarily differentiable, so this method cannot be used in some occasions; on the other hand, since the dimension of the embedding vector input to the large language model is usually greater than 1000, some continuous prompts P (P∈R D) can have an embedding dimension of tens of thousands, which makes non-gradient descent optimization very difficult. D Dimensions optimized for interest.
[0044] Existing literature has shown that large language models have lower core dimensions. For example, in a 1000-dimensional vector, only 50 dimensions are the most important. Setting the values of the remaining 950 dimensions to 0 is almost the same as the original 1000-dimensional vector. These 50 dimensions are the core dimensions of the vector. Therefore, in a smaller subspace, where the dimension value d<< the dimension value D of the original prompt space, and using the random projection matrix A∈R D×d Projecting on the original prompt space, we get the subspace z(z∈R d ). Preferably, d is in the range of [0.07, 0.13] times D, and more preferably, d is 1 / 10 of D. In other words, the size of d in the smaller subspace is 0.07 to 0.13 times D of the original prompt space. More preferably, d is 0.1 times D.
[0045] Therefore, the optimization goal becomes to find an optimal solution space z for z * , this solution space can achieve the minimum loss function value by projecting z with a projection matrix, combining it with a fixed-length prompt p0 (a vector), and feeding it into the large language model with Xm. z is a solution in the Z search space.
[0046] Let the API of large language model inference be F and the optimized loss function be L.
[0047]
[0048] In formula 1, Ym represents the label mask. It is worth mentioning that the masking mechanism prevents the model from seeing future information, which is achieved by multiplying the mask matrix and the attention weight matrix element by element, making some element values in the attention weight matrix very small, so small that they can be ignored.
[0049] Reference to the attached drawings of the specification Figure 1 In order to handle the high dimensionality of continuous prompts, the preferred embodiment of the present application uses a random linear projection to project the original prompt space into a subspace smaller than the original prompt space, and then uses a derivative-free optimizer in this smaller subspace to solve the optimization problem. Unlike traditional fine-tuning methods that can only be performed by the service, black-box fine-tuning allows users to optimize prompt words for specific tasks locally on resource-limited devices (even without a GPU).
[0050] It is worth mentioning that the derivative-free optimizer adopts a genetic algorithm or a PSO or PDFO optimization algorithm.
[0051] The drawings in the specification Figure 2 A prompt template automatic optimization method according to a preferred embodiment of the present invention is explained, which is used to automatically optimize an original task prompt template for completing a task. The prompt template automatic optimization method includes the following steps:
[0052] S100) modifying the original task prompt template to obtain a modified task prompt template;
[0053] S200) using random linear projection to project the space of the original task prompt template onto a subspace smaller than the space of the original task prompt template, and using a derivative-free optimizer in the subspace to obtain an optimized continuous prompt;
[0054] S300) Combining the optimized continuous prompt with the modified task prompt template to obtain an optimized prompt template.
[0055] According to the preferred embodiment of the present invention, step S300 combines the optimized continuous prompt with the modified task prompt template by directly splicing, for example, the prompt template is "Please write an introduction about {title}, with a tone of {tone}."; the continuous prompt is title: artificial intelligence, tone: formal; the spliced result is "Please write an introduction about artificial intelligence, with a formal tone."
[0056] Step S200 uses a random linear projection to project the space of the original task prompt template onto a smaller subspace, and then uses a derivative-free optimizer in the smaller subspace to solve the optimization problem of the continuous prompt. In other words, step S200 uses a random linear projection to project the space of the original task prompt template onto a subspace smaller than the space of the original task prompt template, and uses a derivative-free optimizer in the subspace to obtain an optimized continuous prompt to optimize the continuous prompt in front of the modified task prompt template.
[0057] Reference to the attached drawings of the specification Figure 3 The step S200 of optimizing the continuous prompts in front of the task prompt template includes the following steps:
[0058] S210) obtaining the initial prompt p0 of the task;
[0059] S220) Find the optimal solution space z * ;and
[0060] S230) Obtain the optimization continuous prompt through p0+Az*.
[0061] Step S220 uses the random projection matrix A∈R D×d Projecting on the space P of the original task prompt template, we get the subspace z(z∈R d ), where the value of the random projection matrix A is set by sampling from a uniform distribution.
[0062] Specifically, in step S210, for a given task, a batch of matching task prompt templates are prepared in advance, and the nouns and verbs therein are identified by using word segmentation and part-of-speech analysis techniques. Suppose the set of all words corresponding to the matching task prompt template obtained by word segmentation technology is S, and the set of nouns and verbs is S1. Let S2 = S-S1, and input all words in S2 into the bert model to obtain their corresponding word vectors. Add the word vectors of the words in S2 and calculate the average to obtain the average word vector, and then extract n characters from the vocabulary used by the large language model. The condition that these n characters must meet is: the corresponding word vector is closest to the cosine similarity of the average word vector of the above words. Take these n characters as the initial prompt p0.
[0063] According to this preferred embodiment of the present invention, the value range of n is [the number of elements in the S2 set, 3*the number of elements in the S2 set].
[0064] Reference to the attached drawings of the specification Figure 4 The step S210 of obtaining the initial prompt of the task includes the following steps:
[0065] S211) selecting multiple matching task prompt templates to match a task;
[0066] S212) decomposing the words corresponding to the plurality of matching task prompt templates to obtain a word set;
[0067] S213) excluding nouns and verbs in the word set to obtain a screening word combination;
[0068] S214) obtaining an average word vector of the selected word combination;
[0069] S215) Extract multiple characters that are closest to the average word vector cosine similarity of the filtered word combination as the initial prompt p0.
[0070] According to this preferred embodiment of the present invention, step S211 selects a batch (5) of matching task prompt templates to match the task.
[0071] Step S220 achieves the optimal solution space z by achieving the minimum loss function value * In other words, the minimum loss function is used as the optimal solution z for the subspace z. *The basis is where a projection matrix is used to project the search space Z, combined with the initial prompt p0 of the task obtained in step S210, and the modified task prompt template Xm in step S100, and fed into the large language model.
[0072] Let the API of large language model inference be F and the optimized loss function be L.
[0073]
[0074] According to the preferred embodiment of the present invention, step S220 uses a covariance matrix adaptive evolution strategy (CMA-ES) for optimization to obtain the best z*.
[0075] Reference to the attached drawings of the specification Figure 5 , the step S220 of finding the optimal solution space includes the following steps:
[0076] S221) using random linear projection to project the space of the original hint template onto a subspace smaller than the space of the original hint template; and
[0077] S222) Use a derivative-free optimizer to perform optimization in this subspace.
[0078] The drawings in the specification Figure 6 A specific implementation of the derivative-free optimizer described in step S222 is explained, and an example of optimizing continuous prompts for an optimization algorithm that does not rely on gradients is given. Combining parameter-efficient tuning and a non-gradient optimization algorithm based on random embedding, the vision of using the inference API to complete downstream tasks (removing large model parameter tuning) can be achieved. The large model service provider only needs to perform model inference (i.e. provide computing power), and the optimization of task performance is completed by the user himself (i.e. optimize prompts based on the inference results).
[0079] Step S221 uses the random projection matrix A∈R D×d Projecting on the original prompt space P obtains a smaller subspace z(z∈R d ), where the dimension of z is d< <D。
[0080] In step S221, the search space Z is limited to Z = [-5, 5] d , indicating that each dimension of the d-dimensional vector takes a value between -5 and 5. It is worth mentioning that z is a solution in the Z search space.
[0081] Set the value of the random projection matrix A by sampling from a uniform distribution.
[0082] Step S222 searches for the optimal solution space through the minimum loss function value.
[0083] According to this preferred embodiment of the present invention, two loss functions are set to increase optimization constraints, thereby having a positive impact on model convergence.
[0084] Specifically, the preferred embodiment of the present invention combines cross-entropy loss and hinge loss to calculate the minimum loss function value, and then finds the optimal solution space z*.
[0085] More specifically, given the output Y of the large language model on the candidate label set of the task p and the true label word Y for a specific sample g , the cross entropy loss function formula is defined as:
[0086] L CE (Y p ,Y g )=-logsoftmax Yg (Y p )
[0087] Hinge loss adopts multi-category expansion: r is a preset parameter, preferably, the value of r is 2:
[0088]
[0089] Where i and j are different samples.
[0090] The optimized loss function L = αL hinge +βL CE , where α and β are coefficients, α+β=1.
[0091] Step S230 obtains the optimized continuous prompt through p0+Az*.
[0092] During the reasoning phase, users can combine continuous prompts with task-specific prompt templates, input them into a large language model, and return the answer.
[0093] Unlike the traditional fine-tuning method that can only be performed by the server, the preferred embodiment of the present application allows users to perform local optimization without mastering the parameters of the large language model to meet their own specific needs, expand the scope of use of the closed-source large language model, and improve its utilization value.
[0094] Iteratively query the large language model inference API to optimize the continuous prompts in front of the input text. In order to handle high-dimensional continuous prompt vectors, a random linear projection is used to project the space of the original prompt template into a smaller subspace. Then, some optimization methods that do not use traditional derivative optimization are used in this smaller subspace to solve the optimization problem.
[0095] It is worth mentioning that the technical solution of the present application, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including multiple instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, and other media that can store programs.
[0096] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the rights of the present invention is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present invention should be within the scope of the rights of the present invention.
Claims
1. A prompt template automatic optimization method, used to optimize an original task prompt template for completing a task, characterized in that: The method comprises the following steps: S100) modifying the original task prompt template to obtain a modified task prompt template; S200) using random linear projection to project the space of the original task prompt template onto a subspace smaller than the space of the original task prompt template, and using a derivative-free optimizer in the subspace to obtain an optimized continuous prompt; and S300) Combining the optimized continuous prompt with the modified task prompt template to obtain an optimized prompt template.
2. The automatic optimization method of prompt template according to claim 1, characterized in that: Step S200 includes the following steps: S210) obtaining an initial prompt p0 of the task; S220) Find the optimal solution space z * ;and S230) obtaining the optimization continuous prompt through p0+Az*; In step S220, a random projection matrix A∈R is used. D×d Projecting on the space of the original task prompt template obtains the subspace z(z∈R d ), wherein the value of the random projection matrix A is set by sampling from a uniform distribution, wherein D is the dimension value of the space of the original task prompt template, and wherein d is the dimension value of the subspace.
3. The automatic optimization method of prompt template according to claim 2, characterized in that: Step S220 uses the covariance matrix adaptive evolution strategy to optimize to obtain the best z*, where where F is the API for large language model inference and L is the optimized loss function.
4. The automatic optimization method of prompt template according to claim 3, characterized in that: The optimized loss function L = αL hinge +βL CE , where α and β are coefficients, α+β=1, where L CE (Y p ,Y g )=-logsoftmax Yg (Y p ), Where the output Y of the given large language model on the candidate label set of the task p and the true label word Y for a specific sample g , where r is a preset parameter, and the value of r is 2.
5. The automatic optimization method of prompt template according to claim 4, characterized in that: Step S210 includes the following steps: S211) selecting a plurality of matching task prompt templates to match the task; S212) decomposing the words corresponding to the plurality of matching task prompt templates to obtain a word set; S213) excluding nouns and verbs in the word set to obtain a screening word combination; S214) obtaining an average word vector of the selected word combination; and S215) Extract multiple characters that are closest to the average word vector cosine similarity of the filtered word combination as the initial prompt p0.
6. The automatic optimization method of prompt template according to claim 5, characterized in that: In step S215, n characters that are closest to the average word vector cosine similarity of the filter word combination are extracted as the initial prompt p0, where the value range of n is [the number of elements of the filter word combination, 3*the number of elements of the filter word combination].
7. The automatic optimization method for prompt templates according to any one of claims 1 to 6, characterized in that: The size of the smaller subspace is 0.07 to 0.13 times the size of the space of the original hint template.
8. The automatic optimization method of prompt template according to claim 7, characterized in that: The size of the smaller subspace is 0.1 times the size of the space of the original hint template.
9. A computer device, characterized in that: The device includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; Memory, used to store computer programs; The processor is used to implement the steps of the prompt template automatic optimization method described in any one of claims 1 to 8 when executing the program stored in the memory.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the prompt template automatic optimization method according to any one of claims 1 to 8 are implemented.