Prompt adaptation method, for optimizing outputs of a large language model (LLM)

Abstract

The disclosure relates to a computer implemented prompt adaptation method, and apparatus, for optimizing outputs of a large language model (LLM). The method comprises inputting a query into a trained encoder-decoder based prompt adaptor. The method comprises obtaining an adapted prompt from the prompt adaptor for input into the LLM.

Description

PROMPT ADAPTATION METHOD, FOR OPTIMIZING OUTPUTS OF A LARGE LANGUAGE MODEL (LLM)TECHNICAL FIELD

[0001] The present disclosure relates to trainable prompt adaptors for large language models.BACKGROUND

[0002] One of the main challenges of exploiting Large Language Models (LLMs) effectively is to communicate and guide them to generate accurate responses for specific tasks. Prompt engineering (PE) allows human users, via natural language, to instruct LLMs to produce desired outputs / behaviors. This typically involves multiple interactions with different prompts. The knowledge / expertise of the users in the LLM domain plays an important role to achieve the best possible results.

[0003] Several techniques have been proposed for designing prompts, to improve the abilities of LLMs for extracting relevant information (without the needs for re-training and fine-tuning). Some examples of these prompts include prompts for machine translation, question-answering, reasoning, etc. Existing prompt engineering (PE) techniques include zero-shot or few-shot prompting, chain-of-thought (CoT) prompting, self-consistency, Reasoning + Action (ReAct), etc.

[0004] Prompts optimization can be performed either manually or automatically. Several prompt tuning techniques, such as discrete prompt search / generation methods, prompt scoring, paraphrasing and continuous gradient-based methods have been proposed to optimize the prompt instructions. A method called AutoPrompt was developed based on gradient-based search to generate automated prompts. Prompt tuning was proposed to learn / train soft prompts using a frozen pre-trained language model and additional tunable tokens through backpropagation for different downstream tasks, where task-specific prompts are required. It could outperform fewshot prompting and show competitive performance to model tuning.

[0005] Automatic prompt engineer (APE) method generates and selects instructions automatically using LLMs. Instruction candidates are generated by an LLM based on a few examples of input-output pairs, where the most relevant / optimal instructions are selected according to evaluations scores and iterative Monte Carlo search. Model-generated instructions with APE can demonstrate human-level performance on zeroshot prompting over different tasks. Since prompt evaluation with LLMs is computationally expensive, an offline inverse reinforcement learning (Prompt-OIRL) method was proposed to effectively evaluate prompt for arithmetic reasoning. In this method, an offline reward model is used to optimize query-dependent prompt objective to achieve optimal prompt without LLM evaluation.

[0006] Lately, an effective approach called optimization by prompting (OPRO) was proposed to utilize LLMs as optimizers for prompt optimization tasks, where natural language is used to describe the optimization problem. In this approach, LLMs are instructed to generate new solutions based on the previous solutions and description of optimization problem. In each iteration (optimization step), prompts containing the last solutions and their values are used to generate other solutions according to the objective function. Then, these solutions are evaluated and integrated to the prompt for the subsequent step until maximizing the task performance.

[0007] Despite the astonishing success of LLMs, there are several difficulties in using them in practice. Particularly, the usage of LLMs when there are not enough resources (high-end GPUs, large enough RAM, etc.). Therefore, it is of paramount importance to have small yet capable LLMs that can be run either on CPUs or on affordable GPUs.

[0008] In general, LLMs have high sensitivity to their input prompt. Moreover, the smaller models have even a higher sensitivity to their input prompt. Therefore, PE becomes even more important for the smaller models. Currently, PE is done manually for the most part. Although, there are many papers on automatic PE, most of the research is focused on how to improve the prompts for a specific task.SUMMARY

[0009] There is a need for creating trainable prompt adaptors for multiple domains (e.g., mathematics, finance, programming, etc.) and specifically for small LLMs.

[0010] Also, one of the main problems with existing automatic prompt engineering (APE) solutions is that they are predominantly online methods. Therefore, they are computationally expensive and not energy / cost efficient. They are slow and not realistic due to the cost and resource constraints in most practical applications. They are also not focused on APE for small LLMs.

[0011] There is provided a computer implemented prompt adaptation method, for optimizing outputs of a large language model (LLM). The method comprises inputting a query into a trained encoder-decoder based prompt adaptor. The method comprises obtaining an adapted prompt from the prompt adaptor for input into the LLM.

[0012] There is provided an apparatus for optimizing outputs of a large language model (LLM). The apparatus comprises processing circuits and a memory. The memory contains instructions executable by the processing circuits whereby the apparatus is operative to input a query into a trained encoder-decoder based prompt adaptor. The apparatus is operative to obtain an adapted prompt from the prompt adaptor for input into the LLM.

[0013] There is provided a non-transitory computer readable media having stored thereon instructions for optimizing outputs of a large language model (LLM). The instructions comprise inputting a query into a trained encoder-decoder based prompt adaptor. The instructions comprise obtaining an adapted prompt from the prompt adaptor for input into the LLM.

[0014] The method and apparatus provided herein present improvement to the way LLM prompting operates.BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 is a block diagram illustrating query and answer generation during inference.

[0016] Figure 2 is a schematic illustration of a prompt chain.

[0017] Figure 3 is a block diagram showing how multiple prompts pairs can be generated for training the prompt adaptor, from multiple prompt roots or seeds (Q, A).

[0018] Figure 4 is a schematic illustration of the encoder-decoder model proposed by Vaswani et al, in the paper entitled “Attention is All you Need”.

[0019] Figure 5 is a block diagram illustrating prompt dataset generation and prompt adaptor training.

[0020] Figure 6 is a schematic illustration of prompt template generation and inference using template prompts.

[0021] Figure 7 is a flowchart of a method for optimizing outputs of a large language model (LLM).

[0022] Figure 8 is a schematic illustration of a hardware in which steps and / or method described herein can be executed.

[0023] Figure 9 is a schematic illustration of a virtualization environment in which the different steps and hardware components described herein can be deployed.DETAILED DESCRIPTION

[0024] Various features will now be described with reference to the drawings to fully convey the scope of the disclosure to those skilled in the art.

[0025] Sequences of actions or functions may be used within this disclosure. It should be recognized that some functions or actions, in some contexts, could be performed by specialized circuits, by program instructions being executed by one or more processors, or by a combination of both.

[0026] Further, computer readable carrier or carrier wave may contain an appropriate set of computer instructions that would cause a processor to carry out the techniques described herein.

[0027] The functions / actions described herein may occur out of the order noted in the sequence of actions or simultaneously. Furthermore, in some illustrations, some blocks, functions or actions may be optional and may or may not be executed; these are generally illustrated with dashed lines.

[0028] At least some aspects of the techniques described herein may be implemented using artificial intelligence, which comprises a variety of techniques as would be apparent to a person skilled in the art, including machine learning techniques.Machine learning techniques include deep learning and Neural Network (NN), or Artificial Neural Network (ANN). Both NN and ANN may be used interchangeably herein. In some contexts, an Artificial Neural Network could include biological portions.

[0029] Further, looking forward, in a virtual world (e.g., the metaverse, digital twins, etc.), the techniques described herein could be applied in relevant virtual scenarios.

[0030] Referring to figure 1, the solution 100 described herein can train a prompt adaptor (PA) / tuner 105 for any LLM 110. The solution can also generate a large dataset of prompts automatically from a small and labeled dataset of Questions and Answers (Q& As) for any domain and different downstream tasks.

[0031] This large dataset can then be used to train the small encoder-decoder model (the PA) 105. During inference, the user 115 enters a prompt and the PA 105 modifies the prompt to increase the accuracy of the answer of LLM 110.

[0032] Figure 2 illustrates a prompt chain 200. Prompt chains can be used to generate data to train the prompt adaptor 105. In figure 2, P stands for prompt, Q stands for query / question, OG stands for observation generator and is a small LLM model that is being deployed for inference (it can also be called actor LLM). PG stands for prompt generator and is a large LLM model that is used only for training in the solution (it can also be called critique LLM). In online APE methods, it is used during inference. O stands for observation and is the answer generated by the OG. R stands for reason and includes the reason why PG thinks that OG made a mistake. It is used to generate better prompts for the OG. E stands for error and is the discrepancy between a ground truth answer A and the observation O.

[0033] The prompt chain starts with a query 220 which is input into the OG 110. An answer 225 is generated and input into another LLM model, the prompt generator (PG) 205, which is used to generate a prompt 230 for this user query.

[0034] It should be noted that these LLM models (OG and PG) could be the same model, if a large LLM was used for training. In that case, the model would both generate answers and create prompts from the user queries.

[0035] The prompt 230 is input into the OG 110 again, which produces a second answer 235, which goes again in PG 205, which in turn produces a prompt P’ 230’. This can be repeated a number of times and further answers 245 produce further prompts P” 230”. This goes on until the generated prompt is good enough.To evaluate if a generated prompt is good enough, a vector is used (A, O, E, R), where A is the ground truth answer, O is the observation, which is output / answer generated by the OG 110, E is the error, which is the difference between the ground truth and the observation, which can be calculated with the help of the large LLM (PG 205). R is a reasoning explaining why there is an error and how to decrease the error. The reasoning is therefore used to generate a better prompt. This vector is generated at each iteration in the prompt chain.

[0036] In the current state of the art, such prompt chains are used to generate prompts on the fly, in an online manner which is very expensive especially when using large LLMs.

[0037] Herein the prompt chain is used to generate a training data set, which is much more efficient especially in scenarios where there are constraints on resources. With this training data set, a smaller model can be trained to do the automatic prompt engineering.

[0038] Figure 3 illustrates how the training data set of prompts can be generated. It is assumed there is a starting training data set {(Q, A) } 310 of questions and answers. This starting training data set contains tasks or questions (Q) and associated ideal answer (A), previously called ground truth answers.

[0039] For example, there could be ten pairs of questions and answers 320a, 320b, 320i, 320j. For each question answer pair, using 320a as an example, the large LLM model PG 205 is used to generate a number of prompts all at once. Until this point, no chaining is used. For example, 100 prompts could be generated and from these prompts a subset of the better ones 330a 330b,..., 330v are kept.

[0040] From each of this subset of prompts 33a-330v, a prompt chain is created as explained previously (illustrated by the lines of blocks going towards the right). Then, for each of these prompts, the error is measured and compared with a threshold. When a prompt improves on the previous prompt, the pair of prompts is deemed to be a good pair and is flagged and saved in the database 340. For example, the pairs 350, 351, 360, 361, 330v, 331 are saved in the database. The prompt chain can be terminated after a fixed number of iterations. In each horizontal sequence, there can be one pair, many pairs, or no pairs. The initial prompt usually influences this result.

[0041] Once enough prompts pairs have been generated using this technique, these pairs 370 can be used to train the PA. Each pair consists of a prompt, followed by a better prompt.

[0042] This part of the process is slow, but it is used only for training the PA.

[0043] Turning to figure 4, which is taken from “Vaswani, Ashish; Shazeer, Noam; Parmar, Niki; Uszkoreit, Jakob; Jones, Llion; Gomez, Aidan N; Kaiser, Lukasz;Polosukhin, Illia (2017). " Attention is All you Need”, 31st Conference on Neural Information Processing Systems (NIPS 2017), arXiv: 1706.03762”, a high-level description of how the prompt adaptor is trained is provided.

[0044] The encoder / decoder 400 of figure 4 is behind models such as the T5 model or Bidirectional and Auto-Regressive Transformers (BART).

[0045] Usually, to train these models, the data set comprises pairs, where the first entry is a text, and the second entry is a summary. At each stage, the text is the inputto the encoder decoder and then the model generates as output another text. Then the output is compared with the summary. The closer the output is to the summary, the smaller the error is, which is similar to having a gradient descent to train these models using these pairs.

[0046] In the present case, instead of “text and summary” pairs, “prompt and improved prompt” pairs are used to train the encoder decoder.

[0047] Once the learning is done, referring to figure 1, the prompt adaptor 105 is used to improve the user prompt for prompting the small LLM OG 110.

[0048] The prompt adapter 105 is not very’ large; for example, a Bart model has around 140 million parameters, this number could vary depending on the version, but the number of parameters is in millions, not billions. Because of the reduced number of parameters, it is more resource efficient to use a prompt adapter instead of a large LLM.

[0049] The following paragraphs provide a more formal description of the method explained previously, using an example.

[0050] Assuming a small LLM needs to be deployed for domain specific tasks and a labeled dataset D_qa of Q& As is available. Assuming also that the small LLM has less than IB parameters, D_qa has 100 question and answer pairs (Q i, A_i), and the domain is mathematics. For example, the first Q& A could be Q=“Solve the equation XA2-1 = 0” and the ground truth answer could be A=“Step 1: rewrite the equation as XA2=1, Step 2: take square root from both sides: X= + sqrt(l), X=-sqrt(l), Step 3: replace sqrt(l) with 1 in Step 2. According to steps 1-3, X=1 and X=-l are the solutions”. Here, “domain specific” is mentioned only because this is a typical scenario due to the fact that small LLMs are not suitable for general intelligence. Nevertheless, the proposed solution is not restricted to a single domain and can work for various tasks that belong to multiple domains, e.g. finance, mathematics, programming, etc., as long as there are enough Q& As from these domains in D_qa.

[0051] Given the small LLM and the labeled dataset, one goal is to create prompts, automatically and on the fly, out of any future question / query to improve the accuracy of the answers of the small LLM. This is achieved in two major steps i) creation of a prompt dataset consisting of pairs of the form (prompt, improved prompt) using prompt chains, and ii) training of an encoder-decoder model on the generated prompt dataset that can be used to adapt questions / queries to tailor prompts for the small LLM.

[0052] Problem formulation and solution steps: A prompt chain is formulated as a sequence {X i} of quadruples of prompt, observation (answer / response to the prompt), error (the discrepancy between observation and ground truth answer), and reasoning / speculation for this discrepancy, denoted by X i = (P i, O_i, E_i, R_i). In the formulation, the LLM that generates this sequence is different from the LLM that generates the observations (O_i). The former is a large LLM while the latter is a small LLM. This is for two reasons: i) using the same LLM for both tasks introduce bias, ii) large LLMs are much better in reasoning tasks and general intelligence while small LLMs are much better in solving specific problems and answering queries in specific domains and not suitable for general intelligence or complex reasoning. The large LLM is the prompt generator (PG) while the small LLM is the observation / response generator (OG). The large LLM can also be thought of as the critique and the small LLM as the actor because the large LLM also provides reasoning / criticism to further improves the next sequence of prompts.

[0053] A prompt chain is very sensitive to its first element X_0 (generally 330 in figure 3) because if the initial prompt is misleading, it will take more iterations to fix it and, in some cases, PG might never be able to fix it due to accumulation of errors. Therefore, it is not enough to have one sequence X_0, X_l,..., X_n but rather multiple sequences denoted by {X^j_i} where j is the sequence number. Each sequence starts from a different initial guess / prompt. LLMs are stochastic in nature, that is each run will produce slightly different initial prompt, but it is possible to further increase their stochasticity by changing the temperature parameter. In practice, this process can be sped up by creating M (e.g. M=20) initial guesses at the beginning and selecting the top V (e.g. V=5) initial guesses. Note that the question / query Q itself should be included as an initial guess (as a trivial or bare minimum prompt).

[0054] The prompt sequence is guided by (E_i, R_i) because PG is trying to decrease the discrepancy between the observation of OG and the ground truth, i.e. E_i, by considering the reasoning / speculation behind this discrepancy, i.e. R_i. Overall, {E_i } is objective as long as the evaluation criteria is clear. However, {R_i} is subjective. For example, introducing roles (e.g. “answer this query as an accountant or as a superhero”) in a prompt can improve the accuracy of the answers. It should not be expected for the PG to realize this through {R_i} because there is no good reasoning for this phenomenon and the black-box nature of these models does not allow to find the real reason for the importance of roles in accuracy. This suggests that generatinggood prompts involves some exploration. That is why the prompt that is used to create the M initial guesses, at the beginning of a sequence, is quite important. To add variety to these initial guesses, the following factors should be considered:- Length: long (verbose) vs. short (concise)- Level of detail: low vs. highTone: formal vs. informalStyle: informational vs. educational- Audience: general vs. expertClarity: clear vs. vague- Persona: general vs. particularContext: with vs. withoutGradation: step-by-step vs. immediateSemantics: paraphrase vs. rephrase- Randomness: sentence permutation and random token masking

[0055] More factors can be included depending on the domain and use case. Up to this point, improved prompts have been created for OG using PG. However, it is not practical computationally to run this pipeline for each task / query due to the large number of iterations and the size of PG. Therefore, a simple and light weight model is needed that can adapt the prompts quickly to improve the accuracy of OG. This model is the prompt adaptor (PA). In the solution, the purpose behind PG is to create a training dataset for the PA so that prompts can be adapted efficiently. Therefore, the approach can be considered offline as the trained model PA is just called during inference and then the output of PA, i.e. the improved prompt, is fed to the OG.

[0056] For training a domain adaptor, a large dataset D prompt of the form (Prompt, Improved_prompt) is required which will be fed to an encoder-decoder model for training.

[0057] As already explained previously, the training procedure is quite similar to training a text summarizer In text summarization, the training dataset consists of (text, summary) pairs and an encoder-decoder model such as BART can be trained / fine-tuned to summarize any text. Ideally, the encoder-decoder model should not be too large.

[0058] The question is how one can generate a D_prompt from a small D_qa. Here, it is suggested to use a large LLM (e.g. more than 70B parameters) to generate D_prompt.

[0059] Referring to figure 5, the data generation is accomplished through many chains of prompts as follows:1. For each pair (Q_i, A_i), i=l, N, follow Steps 2-12:2. PG’s prompt: “Given Q i, write M prompts PAk_i (educated guesses), k=l,..., M, that can improve the accuracy of the answer generated by a small LLM (OG). Make sure that the prompts vary and are dissimilar by considering the following factors: [c.f. the factors mentioned above].”,3. Feed each prompt to OG and get back the answers / observations O^k_i, 4. PG’s prompt: “For each PAk_i, report the discrepancy EAk_i between OAk_i and A_i (ground truth answer) by a number and choose the top prompts with lowest error EAk_i. Furthermore, provide reasoning for the effectiveness of the top 5 prompts ”,5. For each quadruple (P^j_iq, O^j_iq, E^j_iq, R^j_iq), j =1,..., V, q=1, follow the steps 6-12:6. PG’s prompt: “Given (PAj_iq, OAj_iq), create a new improved prompt PAj_i(q+l) such that the difference EAj_iq between OAj_iq and A_i decreases. Take into account RAj_iq when generating the new prompt.”,7. Feed P^j_i(q+1) to OG and receive the answer which is denoted by O^j_i(q+1).8. PG’s prompt: “Given (PAj_i(q+l), OAj_i(q+l)), calculate the difference E^j_i(q+1) between OAj_i(q+l) and A_i and give a reasoning RAj_i(q+l).”, 9. If 0< T< EAj_iq - E^j_i(q+1), where T is a pre-defined threshold, push the pair (PAj_iq, PAj_i(q+l)) to D prompt,10. Push (P^j_iq, E^j_iq) and (P^j_i(q+1), E^j_i(q+1)) to D_eval,11. Go to Step 6 and increment q,12. Terminate after I (e.g. I =50, i.e. q =1,..., I) iterations. Keep track of rolling standard deviation, denoted by S, of the sequence {E^j_i1, E^j_i2,... }. If D_prompt is empty or S is too small, slightly increase the temperature of LLM to increase randomness / exploration.Note that the temperature value must be reset to the default value upon termination after I iterations.

[0060] As a result, two datasets, D prompt and D eval are generated. The first dataset will be used to train a prompt adaptor (PA) while the second dataset will be used to train a prompt evaluator (PE). Training a prompt evaluator is optional. The prompt evaluator is a binary classifier. Prompts with errors less than a threshold are classified as good and the rest are classified as bad by the prompt evaluation. The prompt evaluator can be used to provide a labeled dataset with bad and good prompts that can later be used to train a binary classifier.

[0061] A large prompt dataset can therefore be generated in an efficient way from a small, labeled dataset of Q& As because there are V sequences with the maximum length of I for each Q& A. It should be noted that the sequence could be much shorter (even of length zero) depending on how much improvement can be made, but if V=5 and 1=50 is set, then 250 (5*50) prompts can potentially be generated for each Q& A.

[0062] The threshold T in Step 9 plays an important role. If the criterion was 0< E j_iq - E^j_i(q+1), the result would be prompts that are only small improvements, and the quality of the domain adaptor would suffer.

[0063] To train a domain adaptor, an encoder-decoder model is fine-tuned. For example, Bidirectional and Auto-Regressive Transformers (BART) can be fine-tuned on D prompt. BART is pretrained by corrupting documents and then optimizing a reconstruction loss, which is a cross-entropy between the decoder’s output and the original document. During fine-tuning, prompts are the inputs to the encoder and the decoder generates an adapted prompt autoregressively and the improved prompt can be interpreted as the ground truth that guides the fine-tuning in the right direction. Once the PA model (M P A) is trained, it can be used during inference to improve user’s prompts.

[0064] Turning to figure 6, extensions can be provided. Up to here, a domain adaptor has been discussed, a prompt template / guideline can also be derived using both D_prompt and D eval. To achieve this, a handful of top prompt improvements are selected and push to D_ (top prompt). Then these prompt pairs are provided to PG: “Given D_ (top prompt), provide general guidelines on how to improve prompts for domain C, also provide a prompt template according to the guidelines for domain C”. In the example, domain C is mathematics, but it could be any domain. Therefore, not only a prompt adaptor can be provided, but also a prompt template that can be used for future queries in case there is a need to stop using the prompt adaptor for energy efficiency reasons.

[0065] Further, a database of prompt templates can be created for different domains / categories that are tailored to the OG’s LLM. During inference, a simple pipeline can be created:1. Prompt to OG: “find the domain / category of the query Q”,2. Go to database of prompt templates and find the template that corresponds to the category of Step 1,3. Prompt to OG:” Given query Q and prompt template P template of Step 2, write a prompt P improved for query Q”,4. Prompt to OG: “P improved of Step 3”.

[0066] Assuming different datasets 605a-605m have been created for different domains, for example, domain 1 could be mathematics, DI, domain 2 could be finances, D2, and domain m could be biology, Dm, a large LLM (PG205)is fed these examples prompts and the LLM is asked to create template prompts 610a, 610b, 610m. The template prompts are based on good prompts that were generated with the method explained previously and are specific to a domain, for example mathematics for DI.

[0067] During inference, these templates can be used, when a user enters a query, and the small LLM (OG) can decide, based on the question, which category the question relates to. For example, if the question involves solving an equation, it's clear that the domain is related to mathematics.

[0068] Once the domain is determined, the prompt template can be used. A prompt template could, for example, indicate at the beginning of the template to add “assume you are a mathematician” or “assume you are a financial analyst with extensive experience”. The actual question from the user could follow with possibly other instruction as part of that template. The result of this concatenation can then become an improved prompt.

[0069] The solution described herein has many advantages. It is cost and energy efficient because it is an offline method. It is fast. It is much faster to call a rather small PA model once than calling a large LLM many times in a loop. It is relatively easy to maintain. There is no significant data / concept drift for these types of applications. As long as the context / domain and the LLM model do not change, PA should perform reasonably well. If the context / domain changes a lot, it suffices to train the PA with a larger and more diverse training dataset. It can be trained in a weakly-supervised manner. The Q& A dataset is labeled but the training dataset of PAis pseudo-labeled (labeled by another LLM). Furthermore, the Q& A dataset is much smaller than the prompt dataset used for training the PA. Hence, it falls in the category of weakly-supervised learning methods. The dataset of prompt pairs together with their error sequence can be reused for downstream tasks, including creating prompt templates or prompt guidelines automatically. This can further lead to more cost and energy efficient solution for PE.

[0070] Turning to figure 7, there is provided a method 700, more specifically a computer implemented prompt adaptation method, for optimizing outputs of a large language model (LLM). The method comprises inputting, step 701, a query into a trained encoder-decoder based prompt adaptor. The method comprises obtaining, step 713, an adapted prompt from the prompt adaptor for input into the LLM.

[0071] The LLM may be a small LLM. In the context of the present disclosure, small can mean internal parameters in millions instead of in billions according to the state of the art at the date of filing. This definition should be expected to change as progresses are made in the field. Small could also be interpreted as a significantly smaller than the biggest models available. Significantly smaller could mean, as an example, anything between one thousandth the size and half the size of the biggest models available.

[0072] The LLM may be trained to be domain specific. The query may be a text query. The query may be obtained from a user or from a system.

[0073] The method may further comprise training, step 702, the encoder-decoder based prompt adaptor using pairs of prompts, each pair of prompts comprising a prompt and an improved prompt.

[0074] The method may further comprise generating, step 703, the pairs of prompts using a prompt chain. The method may comprise inputting, step 704, a root prompt into the LLM and obtaining an output. The method may comprise, iteratively, step 705, until a stop criterion is met, computing, step 706, an error by comparing the output with a ground truth associated with the root prompt and generating a new prompt using a second LLM, and using, step 708, the new prompt as the next input to the LLM for obtaining the output. The method may further comprise, upon detection, step 709, in the prompt chain, of a pair of prompts consisting of a prompt and an improved prompt, storing the pair of prompts for training the encoder-decoder based prompt adaptor.

[0075] The pairs of prompts for training the encoder-decoder based prompt adaptor may be generated for a plurality of root prompts.

[0076] The root prompt may be selected from a plurality of prompts initially generated by the second LLM and labeled with associated ground truth.

[0077] Computing the error may further comprise generating, step 707, a reasoning to be used by the second LLM to generate the new prompt. The second LLM is the PG (prompt generator) which is the bigger model. By asking the second LLM to generate a reasoning for each prompt generation, the system is more logically consistent. Furthermore, this reasoning can be inspected by a human to make the process less of a black box.

[0078] The detection of a pair of prompts consisting of a prompt and an improved prompt, may comprise comparing, step 710, a difference between the error of the prompt and the error of the improved prompt with a threshold and storing the pair of prompts if the difference is greater than the threshold.

[0079] The stop criterion may be met after a predetermined number of iterations or after a predetermined error measurement is achieved. The stop criterion could be either a hard threshold (for example, after 100 iterations or when the error decreases by 5%) or it could be a softer threshold (for example, stop when the error is not decreasing anymore or when it decreases too slowly).

[0080] The method may further comprise training, step 711, the encoder-decoder based prompt adaptor using the stored pairs of prompts.

[0081] The method may further comprise generating, step 712, prompt templates associated with different domains using the second LLM.

[0082] It should be noted that methods and steps described herein are, generally, computer implemented methods and steps. The term computer may be interpreted as having different meanings, such as explained next, for example.

[0083] Referring to figure 8, there is provided an apparatus (HW) 801, in which functions and steps described herein can be implemented.

[0084] The apparatus 801 (which may go beyond what is illustrated in figure 8), may be a user device, such as a smartphone, a tablet, a computer, a wearable such as a watch or glasses, a connected vehicle, including but not limited to a bicycle, a car, a truck, a plane, a drone, a boat or any other flying or floating vehicle, etc.

[0085] The apparatus 801 may be a server, a network node, a radio base station, or any other computing device which may be part of a cloud computing system, edge computing system, or which may be a standalone device.

[0086] The apparatus is operative to adapt prompts for optimizing outputs of a large language model (LLM). This is done by inputting a query into a trained encoderdecoder based prompt adaptor and obtaining in response, from the prompt adaptor, an adapted prompt for input into the LLM. The prompt adaptor is based on the encoderdecoder architecture and is trained using pairs of prompts obtained through prompt chaining as explained above. This provides the following technical effects. Using improved prompts allows obtaining more relevant and / or more accurate results from an LLM, more rapidly, by skipping the numerous iterations usually needed to end up having a good prompt. When obtained through a method such as the one described here, multiple prompts can be improved automatically, in large number and very rapidly, thereby allowing increased efficiency for a system in which this is implemented.

[0087] The apparatus 801 comprises processing circuitry 803 and memory 805. The memory 805 can contain instructions executable by the processing circuitry 803 whereby functions and steps described herein may be executed to provide any of the relevant features and benefits disclosed herein.

[0088] The apparatus 801 may also include non-transitory, persistent, machine-readable storage media 807 having stored therein software and / or instruction 809 executable by the processing circuitry 803 to execute functions and steps described herein. The apparatus may also include network interface(s) and a power source.

[0089] The instructions 809 may include a computer program for configuring the processing circuitry 803. The computer program may be stored in a physical memory local to the device, which can be removable, or it could alternatively, or in part, be stored in the cloud. The computer program may also be embodied in a carrier such as an electronic signal, optical signal, radio signal, or computer readable storage medium.

[0090] Referring to figure 9, there is provided a virtualization environment 900 in which functions and steps described herein can be implemented.

[0091] The virtualization environment 900 (which may go beyond what is illustrated in figure 9), may comprise systems, networks, servers, nodes, devices, etc., that are in communication with each other either through wire or wirelessly, e.g. through anetwork interface component (NIC) comprising physical network interface(s). Some or all of the functions and steps described herein may be implemented as one or more virtual components (e.g., via one or more applications, components, functions, virtual machines, containers, etc.) executing on one or more physical apparatus in one or more networks, systems, environment, etc.

[0092] A virtualization environment provides hardware 901 comprising processing circuitry 903 and memory 905. The memory 905 can contain instructions executable by the processing circuitry 903 whereby functions and steps described herein may be executed to provide any of the relevant features and benefits disclosed herein.

[0093] The hardware 901 may also include non-transitory, persistent, machine-readable storage media 907 having stored therein software and / or instruction 909 executable by the processing circuitry 903 to execute functions and steps described herein.

[0094] The instructions 909 may include a computer program for configuring the processing circuitry 903. The computer program may be stored in a removable memory, such as a portable compact disc, portable digital video disc, or other removable media. The computer program may be stored in a physical memory local to the hardware 901, which can be removable, or it could alternatively, or in part, be stored in the cloud. The computer program may also be embodied in a carrier such as an electronic signal, optical signal, radio signal, or computer readable storage medium.

[0095] Referring again to figures 8 and 9, there is provided an apparatus 801, 901 for optimizing outputs of a large language model (LLM). The apparatus comprises processing circuitry 803, 903 and a memory 805, 905. The memory contains instructions executable by the processing circuits whereby the apparatus is operative to input a query into a trained encoder-decoder based prompt adaptor. The apparatus is operative to obtain an adapted prompt from the prompt adaptor for input into the LLM.

[0096] The apparatus is further operative to execute any of the steps described herein, for example steps 702 to 712.

[0097] Referring again to figures 8 and 9, there is provided a non-transitory computer readable media 807, 907 having stored thereon instructions 809, 909 for optimizing outputs of a large language model (LLM). The instructions comprise inputting a queryinto a trained encoder-decoder based prompt adaptor. The instructions comprise obtaining an adapted prompt from the prompt adaptor for input into the LLM.

[0098] The instructions may comprise further instructions for executing any of the steps described herein, including steps 702 to 712.

[0099] Modifications will come to mind to one skilled in the art having the benefit of the teachings presented in the foregoing description and the associated drawings. Therefore, it is to be understood that modifications, such as specific forms other than those described above, are intended to be included within the scope of this disclosure. The previous description is merely illustrative and should not be considered restrictive in any way. The scope sought is given by the appended claims, rather than the preceding description, and all variations and equivalents that fall within the range of the claims are intended to be embraced therein. Although specific terms may be employed herein, they are used in a generic and descriptive sense only and not for purposes of limitation.

Claims

CLAIMS1. A computer implemented prompt adaptation method, for optimizing outputs of a large language model (LLM), comprising:- inputting (701) a query into a trained encoder-decoder based prompt adaptor; andobtaining (713) an adapted prompt from the prompt adaptor for input into the LLM.

2. The method of claim 1, wherein the LLM is a small LLM.

3. The method of claim 1 or 2, wherein the LLM is trained to be domain specific.

4. The method of any one of claims 1 to 3, wherein the query is a text query.

5. The method of any one of claims 1 to 4, wherein the query is obtained from a user or from a system.

6. The method of any one of claims 1 to 5, further comprising training (702) the encoder-decoder based prompt adaptor using pairs of prompts, each pair of prompts comprising a prompt and an improved prompt.

7. The method of claim 6, further comprising generating (703) the pairs of prompts using a prompt chain, comprising:- inputting (704) a root prompt into the LLM and obtaining an output; iteratively (705), until a stop criterion is met:computing (706) an error by comparing the output with a ground truth associated with the root prompt and generating a new prompt using a second LLM; and- using (708) the new prompt as the next input to the LLM for obtaining the output; and- upon detection (709), in the prompt chain, of a pair of prompts consisting of a prompt and an improved prompt, storing the pair of prompts for training the encoder-decoder based prompt adaptor.

8. The method of claim 7, wherein the pairs of prompts for training the encoderdecoder based prompt adaptor are generated for a plurality of root prompts.

9. The method of claim 7 or 8, wherein the root prompt is selected from a plurality of prompts initially generated by the second LLM and labeled with associated ground truth.

10. The method of any one of claims 7 to 9, wherein computing the error further comprises generating (707) a reasoning to be used by the second LLM to generate the new prompt.

11. The method of any one of claims 7 to 10, wherein detection of a pair of prompts consisting of a prompt and an improved prompt, comprises comparing (710) a difference between the error of the prompt and the error of the improved prompt with a threshold and storing the pair of prompts if the difference is greater than the threshold.

12. The method of any one of claims 7 to 11, wherein the stop criterion is met after a predetermined number of iterations or after a predetermined error measurement is achieved.

13. The method of any one of claims 7 to 12, further comprising training (711) the encoder-decoder based prompt adaptor using the stored pairs of prompts.

14. The method of any one of claims 7 to 12, further comprising generating (712) prompt templates associated with different domains using the second LLM.

15. An apparatus (801, 901) for optimizing outputs of a large language model (LLM) comprising processing circuits (803, 903) and a memory (805, 905), the memory containing instructions executable by the processing circuits whereby the apparatus is operative to:- input (701) a query into a trained encoder-decoder based prompt adaptor; and obtain (713) an adapted prompt from the prompt adaptor for input into the LLM.

16. The apparatus (801, 901) of claim 17, further operative to execute the steps of any one of claims 2 to 16.

17. A non-transitory computer readable media (807, 907) having stored thereon instructions (809, 909) for optimizing outputs of a large language model (LLM), the instructions comprising:- inputting (701) a query into a trained encoder-decoder based prompt adaptor;andobtaining (713) an adapted prompt from the prompt adaptor for input into the LLM.

18. The non-transitory computer readable media of claim 19, having stored thereon further instructions for executing the steps of any one of claims 2 to 16.

Images