A vocabulary-constrained discrete prompt learning method and system

Abstract

The application relates to the field of artificial intelligence and natural language processing technology, and discloses a discrete prompt learning method and system based on a vocabulary constraint. The method comprises the following steps: constructing an initial prompt and mapping the initial prompt into prompt embeddings, inputting a target model to calculate corresponding gradients; searching for candidate vocabulary elements with a closer distance to the positions of the prompt embeddings in the vocabulary of the target model, selecting vocabulary constraint elements, weighting and fusing the gradient update results and the vocabulary element word vectors to obtain updated prompt embeddings; discretizing the updated prompt embeddings into discrete prompts composed of vocabulary elements, and adjusting the prompt length according to a first-order importance score; and connecting multiple models in series, constructing proxy targets and proxy gradients with the optimized intermediate outputs from the downstream to the upstream to gradually optimize the prompts at each step. The application considers both the continuous optimization efficiency and the interpretability, editability and deployability of the discrete prompts, alleviates the gradient block caused by the discrete intermediate outputs, and improves the optimization effect of the multi-step task prompts.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence and natural language processing, and specifically relates to a discrete prompting learning method and system based on vocabulary constraints. Background Technology

[0002] Prompt words, by attaching suggestive information before input data, guide pre-trained models or large language models to perform specific tasks, reducing model adaptation costs compared to directly updating all model parameters. However, manually designed prompt words heavily rely on human intuition and extensive trial-and-error processes, resulting in high development costs while failing to guarantee optimal prompts.

[0003] To automate the prompt optimization process, heuristic-based local editing techniques or large-model-based optimization techniques can be used. Heuristic-based methods, such as Black-Box Tuning (BB Tuning), embed the prompt into a low-dimensional subspace and optimize it using derivative-free search. GrIPS (Gradient-free Instructional Prompt Search) treats the prompt as editable text, searching among candidate prompts through predefined phrase-level editing operations. Large-model-based methods use a large model to guide prompt optimization. Some algorithms, such as TextGard (text gradient) and RiOT (Residual Optimization Tree), utilize the reflective capabilities of large models to generate update signals. Others, such as GPO (Gradient-inspired LLM-based Prompt Optimizer), improve the prompt optimization process by generating multiple candidate solutions and utilizing optimization trajectories. Although these methods have significantly improved prompts through iterative verification and evaluation, they generally lack a clear optimization direction.

[0004] Some methods model cue optimization as a reinforcement learning process with task-specific rewards, introducing a more explicit optimization direction to guide the search for high-quality cue words. For example, RLPrompt (Reinforcement Learning Prompt) uses downstream task rewards to learn lexical generation strategies, while TEMPERA (Test-Time Prompt Editing via Reinforcement Learning) extends this paradigm to test-time editing for specific queries. Given the slow convergence of reinforcement learning-based methods, StablePrompt introduces the APPO (Adaptive Proximal Policy Optimization) algorithm to constrain policy updates while maintaining exploration capabilities. However, due to the sparse reward signals and high evaluation costs, reinforcement learning-based methods often converge slowly. Gradient-based methods model cue optimization as a training problem and directly update cue word embeddings, providing a more efficient alternative.

[0005] Gradient-based approaches include continuous and discrete hints. Continuous hints use gradient descent in a continuous space to update lexical embeddings. Early methods directly concatenated trainable embedding vectors before the input sequence, while prefix-tuning methods further concatenate trainable vectors layer by layer before the key-value states of the attention mechanism. Although continuous hint optimization is a differentiable process, the optimized hints can only be represented as high-dimensional embeddings in a continuous space, lacking interpretability and being non-editable. Furthermore, they are unsuitable for multi-step tasks that convey information using natural language lexical units.

[0006] Discrete prompts consistently generate lexical units within the vocabulary. For example, AutoPrompt (Automatic Prompt) replaces lexical units with one of the candidate lexical units based on gradients during training, but strictly limiting the update range to discrete lexical units makes the search prone to getting stuck in local optima. PEZ (Hard Prompts Made EaZy), on the other hand, continuously updates word embeddings like continuous prompt learning methods, mapping them to the nearest discrete prompt lexical unit at each validation step. However, continuous prompt embeddings may drift outside the vocabulary embedding space, resulting in multiple different continuous embeddings mapping to the same vocabulary during the discretization process, which limits the search space to some extent.

[0007] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of the present invention, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0008] To address or at least alleviate one or more of the above problems, a discrete cue learning method and system based on vocabulary constraints is provided. This method can obtain interpretable, editable, and deployable vocabulary cue forms while preserving the optimization efficiency of continuous space, and alleviates the problem that gradients cannot propagate across discrete intermediate outputs in multi-step tasks.

[0009] To achieve the above objectives, according to a first aspect of the present invention, a discrete prompting learning method based on vocabulary constraints is provided, comprising the following steps.

[0010] S1. Construct initial prompt words for the task to be optimized, map the lexical units in the initial prompt words to the word vectors corresponding to the target model to obtain the prompt embedding, and input the prompt embedding into the target model. Calculate the task loss based on the difference between the target model output and the task target, and calculate the gradient corresponding to the prompt embedding.

[0011] S2. In the word vector set corresponding to the vocabulary of the target model, retrieve candidate word elements that are close to each position in the prompt embedding; based on the consistency between the displacement direction of the candidate word element relative to the prompt embedding at the current position and the negative gradient direction at the current position, select vocabulary constraint word elements from the candidate word elements to constrain the current update direction; perform weighted fusion of the update result obtained based on gradient descent and the word vectors of the vocabulary constraint word elements to obtain the updated prompt embedding.

[0012] S3. Discretize the updated prompts into discrete prompts composed of lexical units from the vocabulary of the target model, and delete lexical units with scores lower than a preset threshold based on the importance score of each prompt embedding position to the task loss, to obtain discrete prompts with adjusted length that can be directly input into the target model to execute the task, i.e., optimized prompt words.

[0013] Further, initial prompt words are constructed for the task to be optimized, and the lexical units in the initial prompt words are mapped to word vectors corresponding to the target model to obtain prompt embeddings. The prompt embeddings are then input into the target model. The task loss is calculated based on the difference between the output of the target model and the task objective, and the gradient corresponding to the prompt embeddings is calculated.

[0014] An initial prompt word is constructed for the task to be optimized, and the initial prompt word is concatenated with the input data and then input into the target model, with the optimization objective being to minimize the task loss.

[0015] Based on the lexical sequence corresponding to the initial prompt word, each lexical is mapped to an embedding vector through the lexical embedding matrix of the target model, and a prompt embedding matrix is ​​constructed according to the order of each lexical in the prompt word.

[0016] The cue embedding matrix is ​​input into the target model for forward computation, and the gradient corresponding to each embedding vector in the cue embedding matrix is ​​calculated based on the task loss.

[0017] Furthermore, in the word vector set corresponding to the vocabulary of the target model, candidate word units that are close to each position in the prompt embedding are retrieved, including...

[0018] For each cue embedding position, calculate the Euclidean distance between the cue embedding at that position and the word embedding vector corresponding to each word in the target model vocabulary, and select a preset number of words with the smallest distance to form a candidate word set.

[0019] In the candidate lexical set, based on the directional consistency between the displacement direction of the candidate lexical relative to the current prompt embedding and the negative gradient direction of the current position, the candidate lexical that is most consistent with the negative gradient direction is selected as the vocabulary constraint lexical.

[0020] Further, selecting the candidate lexical unit that is most consistent with the negative gradient direction as the word list constraint lexical unit includes: calculating the cosine value of the angle between the displacement vector of the candidate lexical unit relative to the current prompt embedding and the negative gradient vector of the current position, and selecting the candidate lexical unit with the largest cosine value as the word list constraint lexical unit.

[0021] Furthermore, based on the importance score of each prompt embedding position to the task loss, words and phrases with scores below a preset threshold are deleted, including...

[0022] For each updated cue embedding position, calculate the absolute value of the inner product between the cue embedding position and the corresponding gradient, and normalize the absolute values ​​of the inner products of all cue embedding positions to obtain the first-order importance score of each cue embedding position.

[0023] When the first-order importance score of the prompt embedding position is lower than the preset threshold, the word corresponding to the prompt embedding position is deleted.

[0024] Furthermore, deleting words with scores below a preset threshold includes: iteratively performing scoring and deletion operations until the length of the prompt word reaches a preset minimum length, or the first-order importance score of each prompt embedding position is not lower than the preset threshold.

[0025] Furthermore, it also includes S4: when a task contains multiple models or processing steps that are executed sequentially, the multiple task models are chained together in a preset order, and the output of the previous task model is used as the input of the next task model to form a multi-step task processing flow; the prompts are optimized starting from the last processing step, the intermediate outputs obtained from the downstream steps are used as the optimization targets of the upstream steps, a proxy loss is constructed for the upstream steps and the proxy gradient is calculated, and the discrete prompts corresponding to each processing step are optimized step by step in the order from downstream to upstream.

[0026] Furthermore, multiple task models are connected in a preset order, including...

[0027] The output of the previous task model is concatenated with the prompt of the next task model and used as the input of the next task model, so that each processing step is executed sequentially.

[0028] With the goal of minimizing the final task loss, the prompts for each processing step are jointly optimized.

[0029] Furthermore, the intermediate output obtained from the downstream optimization step is used as the optimization objective of the upstream step. A proxy loss is constructed for the upstream step, and the proxy gradient is calculated, including...

[0030] The intermediate output lexical units obtained after downstream optimization are used as the optimization targets of upstream steps.

[0031] For the prompt words in the upstream step, a proxy loss is constructed based on the difference between the optimization objective and the actual intermediate output of the upstream step. The proxy gradient of the prompt words in the upstream step is calculated based on the proxy loss, and the prompt words in the upstream step are optimized based on the proxy gradient.

[0032] To achieve the above objectives, according to a second aspect of the present invention, a discrete prompting learning system based on vocabulary constraints is provided, comprising:

[0033] The gradient calculation module is used to construct initial prompt words for the task to be optimized, map the lexical units in the initial prompt words to word vectors corresponding to the target model to obtain the prompt embedding, input the prompt embedding into the target model, calculate the task loss based on the difference between the output of the target model and the task target, and calculate the gradient corresponding to the prompt embedding.

[0034] The vocabulary constraint update module is used to retrieve candidate words that are close to each position in the prompt embedding from the word vector set corresponding to the vocabulary of the target model; select vocabulary constraint words from the candidate words to constrain the current update direction based on the directional consistency between the displacement direction of the candidate words relative to the prompt embedding at the current position and the negative gradient direction of the current position; and perform weighted fusion of the update result obtained based on gradient descent and the word vector of the vocabulary constraint words to obtain the updated prompt embedding.

[0035] The discretization and length adjustment module discretizes the updated prompts into discrete prompts composed of lexical units from the vocabulary of the target model, and deletes lexical units with scores below a preset threshold based on the importance score of each prompt embedding position to the task loss, thus obtaining discrete prompts with adjusted length that can be directly input into the target model to execute the task.

[0036] By adopting the above technical solution, the present invention has the following beneficial effects compared with the prior art.

[0037] This invention combines continuous gradient updates with vocabulary constraints to balance efficient search with practical deployment. It expands the search space by utilizing vocabulary constraints and gradient direction consistency selection to reduce the risk of local optima. It introduces length pruning based on first-order importance scores to automatically remove low-contribution vocabulary and obtain concise prompts that are suitable for the task. At the same time, it designs a reverse relay mechanism for multi-step tasks, transforming the downstream optimized intermediate output into a relay target that can be used upstream, effectively alleviating gradient blocking caused by discrete results. Overall, it significantly improves the prompt optimization effect and interpretability in multi-step tasks.

[0038] Furthermore, this invention combines continuous gradient updates with vocabulary lexical constraints, enabling discrete suggestion optimization to simultaneously possess efficient search capabilities in continuous space and deployability under vocabulary constraints; the optimized suggestions are lexical sequences within the vocabulary range, possessing interpretable, editable, and deployable characteristics.

[0039] Furthermore, this invention utilizes vocabulary constraints and gradient direction consistency selection to enable the optimization process to explore a wider search region than strict word-by-word discrete replacement, reducing the risk of getting trapped in local optima.

[0040] Furthermore, this invention, through prompt length pruning based on first-order importance scoring, can automatically delete low-contribution lexical units, reduce redundant prompts, and obtain a prompt length that is suitable for the task.

[0041] Furthermore, the present invention constructs a reverse relay mechanism in multi-step tasks, which converts the optimized intermediate output generated by the downstream steps into a relay target that can be used by the upstream steps, thereby alleviating the gradient blocking problem caused by discrete intermediate results and improving the prompting optimization effect in multi-step tasks.

[0042] The specific embodiments of the present invention will now be described in further detail with reference to the accompanying drawings. Attached Figure Description

[0043] The accompanying drawings, as part of this invention, are provided to further illustrate the invention. The illustrative embodiments and descriptions of the invention are used to explain the invention, but do not constitute an undue limitation thereof. Clearly, the drawings described below are merely some embodiments; those skilled in the art can obtain other drawings based on these drawings without any creative effort.

[0044] Figure 1 This is a flowchart illustrating the discrete prompt learning method based on vocabulary constraints in this specific embodiment.

[0045] Figure 2 This is a schematic diagram illustrating the optimization process of the method described in this specific embodiment in a multi-step task.

[0046] Figure 3 This is a schematic diagram illustrating the operation of the algorithm described in this specific embodiment of the invention in a hub language machine translation task scenario based on multi-model concatenation.

[0047] Figure 4 This is a schematic diagram illustrating the operation of the algorithm described in this invention in a multimodal model-based image cue word inversion task scenario.

[0048] Figure 5 This is a schematic diagram comparing the image instances reconstructed by the algorithm described in this invention with other optimization algorithms during the image prompt word inversion process.

[0049] Figure 6 This is a schematic diagram of the discrete prompting learning system based on vocabulary constraints in this specific embodiment. Detailed Implementation

[0050] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments will be clearly and completely described below with reference to the accompanying drawings. The following embodiments are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0051] In the following description, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0052] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of embodiments of the present invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0053] Please see Figure 1 This invention aims to solve the technical problems in existing discrete prompt optimization, such as limited search space, easy getting trapped in local optima, difficulty in automatically adjusting prompt length, and inability of gradients to propagate across discrete intermediate outputs in multi-step tasks. It provides a discrete prompt learning method based on vocabulary constraints, which includes the following steps.

[0054] S1. Construct initial prompts for the task to be optimized, map the lexical units in the initial prompts to word vectors corresponding to the target model to obtain the prompt embeddings, and input the prompt embeddings into the target model. Calculate the gradient corresponding to the prompt embeddings based on the task loss.

[0055] S2. In the word vector set corresponding to the vocabulary of the target model, retrieve candidate word elements that are close to each position in the prompt embedding; select vocabulary constraint word elements from the candidate word elements based on the directional consistency between the displacement direction of the candidate word element relative to the prompt embedding at the current position and the negative gradient direction of the current position; and perform weighted fusion of the update result obtained based on the gradient and the word vector of the vocabulary constraint word element to obtain the updated prompt embedding.

[0056] S3. Discretize the updated prompts into discrete prompts composed of lexical units from the vocabulary of the target model, and delete lexical units with scores lower than a preset threshold based on the importance score of each prompt embedding position to the task loss, so as to adjust the length of the discrete prompts.

[0057] S4. When a task contains multiple models or processing steps that are executed sequentially, the multiple task models are chained together in a preset order, and the output of the previous task model is used as the input of the next task model to form a multi-step task processing flow. The prompts are optimized starting from the last processing step. The intermediate outputs obtained from the downstream steps are used as the optimization targets of the upstream steps. A proxy loss is constructed for the upstream steps and the proxy gradient is calculated. The discrete prompts corresponding to each processing step are optimized step by step in the order from downstream to upstream.

[0058] Specifically, step S1, gradient calculation: construct initial prompts for the task to be optimized, divide the initial prompt words into multiple word units, and map each word unit to the word vector corresponding to the target model to form a prompt embedding matrix; input the prompt embedding matrix and the input data of the task to be processed into the target model, calculate the task loss based on the difference between the output result of the target model and the task target, and further calculate the gradient of the task loss relative to the prompt embedding matrix to obtain the gradient information corresponding to each prompt embedding position, providing an update direction for subsequent prompt optimization.

[0059] Specifically, step S2, word list constraint update: For each cue position in the cue embedding matrix, based on the distance between the cue embedding at the current position and the word vectors corresponding to each word in the target model word list, multiple candidate words with relatively close distances are retrieved from the word list; further, based on the consistency between the displacement direction of each candidate word relative to the cue embedding at the current position and the negative gradient direction at the current position, word list constraint words are selected from the candidate words to constrain the current update direction; weighted fusion is performed based on the gradient descent result at the current position and the word vectors corresponding to the word list constraint words to obtain the updated cue embedding, thereby making the cue embedding remain near the embedding space corresponding to the target model word list while continuously optimizing using gradients.

[0060] Specifically, step S3, discretization and prompt length adjustment: further calculate the corresponding importance score based on the contribution of each prompt embedding position to the current task loss, and delete words with importance scores below a preset threshold to remove prompt content that contributes little to task performance; stop deletion when the prompt length reaches the preset minimum length or the prompt optimization reaches the convergence condition, and obtain a discrete prompt with adjusted length that can be directly input into the target model to execute the task.

[0061] When a task contains multiple sequentially executed models or processing steps, step S4, reverse relay optimization, is performed: Based on the input-output relationship between multiple task models, the task models are chained together according to a preset execution order, so that the output of the previous task model is used as the input of the next task model, thus forming a multi-step task processing flow consisting of multiple processing steps; starting from the last processing step of the multi-step task processing flow, the corresponding prompts are optimized, and the optimized downstream processing results are obtained; the intermediate output obtained after the optimization of the downstream processing steps is used as the optimization target of the adjacent upstream processing steps, and a proxy loss is constructed based on the difference between the actual output of the upstream processing steps and the optimized intermediate output, and the proxy gradient corresponding to the upstream prompts is calculated based on the proxy loss; prompt optimization is performed step by step from the downstream processing steps to the upstream processing steps, so that the optimization information generated by the downstream processing steps is gradually transmitted to the upstream processing steps through the intermediate outputs, thereby realizing the reverse relay optimization of prompts corresponding to multiple processing steps.

[0062] It should be noted that the method described in this embodiment can be called RelayPrompt (Relay Prompt, a relay-based prompt word optimization method). In a single-step task, this method includes three sequentially executed processing stages: gradient calculation, vocabulary constraint update, and discretization and length adjustment. When the task contains multiple sequentially executed models or processing steps, a reverse relay optimization stage is also included. Each stage is described below.

[0063] (a) Gradient calculation stage.

[0064] (1.1) Initialize prompts: Construct initial prompts for the task to be optimized, and concatenate the prompts and input data together and input them into the task model.

[0065] (1.2) Calculate the prompt word embedding matrix: Based on the word sequence corresponding to the initial prompt word, map each word to the corresponding embedding vector through the word embedding matrix of the target model, and construct the prompt word embedding matrix.

[0066] (1.3) Calculate the gradient corresponding to the embedding matrix: Input the prompt word embedding matrix into the task model for forward calculation, and calculate the gradient corresponding to each embedding vector in the embedding matrix according to the task loss function.

[0067] Specifically, given a task dataset where each sample consists of a pair of inputs and outputs, after initializing the prompt words, the prompt words and inputs are fed together into the task model. The optimization objective is to find the optimal prompt words that minimize the task loss. Constrained by the task model's vocabulary, the optimal prompt words are sequences of lexical units whose length falls between a preset minimum and maximum length, and whose units all belong to the vocabulary. The task model's word segmenter converts the prompt words into a sequence of lexical unit IDs (IDs), and then maps each lexical unit ID to the embedding space using the model's lexical unit embedding matrix, obtaining the embedding vectors of each lexical unit, which together form the prompt embedding matrix. The gradient of the loss function relative to the prompt embedding matrix is ​​the gradient corresponding to the prompt embedding matrix.

[0068] The gradient calculation stage can be implemented through the following steps.

[0069] S11. For a given task dataset ,in , These are paired inputs and outputs, and the initialization prompt. Next, the prompt words and input are fed into the task model together, and the optimization goal is to find the optimal prompt words.

[0070]

[0071] in, It is a task model. These are the suggestion words that need optimization. The search space consists of all legal suggestion words. The index number of the dataset sample. The number of samples in the dataset. It is the loss function.

[0072] At the same time A vocabulary describing the task model, then satisfy.

[0073]

[0074] in, It is all lengths of A set of word sequence sequences and These are the minimum and maximum values ​​for the prompt word length, respectively.

[0075] S12, Prompt Words ,in Indicates the first Each word element, This indicates the prompt length. The prompt word is input into the word segmenter corresponding to the target task model. The word segmenter converts the prompt word into a corresponding word ID sequence according to the word segmentation rules preset by the target task model, so as to establish the correspondence between each word in the prompt word and the vocabulary of the target task model.

[0076]

[0077] in, prompt words The sequence of word IDs obtained after processing by the word segmenter For each word element of the prompt word, For the prompt word The ID of each word element Vocabulary for the target task model The size of the word is the total number of words in the vocabulary.

[0078] Subsequently, based on the lexical ID sequence in the lexical embedding matrix of the target task model... In the table lookup, each word ID is mapped to a word vector of the corresponding dimension.

[0079]

[0080] in, For the first each word element The corresponding word embedding vector.

[0081] The words are combined sequentially according to their order of arrangement in the prompt words. ,in, Given the word embedding vectors corresponding to each lexical unit in the prompt word, we obtain the prompt embedding matrix corresponding to the current prompt word. The cue embedding matrix is ​​used to represent the current discrete cue word in the continuous embedding space of the target model, and serves as the object of subsequent gradient calculation and cue embedding update.

[0082] S13. Embed the prompts obtained in step S12 into the matrix. The input data from the task dataset is fed into the target task model. Forward computation is performed through the target task model to obtain the corresponding task output. The task loss is calculated based on the difference between the task output and the task target. Backpropagation is performed on the task loss to calculate the gradient of the task loss relative to the cue embedding matrix, thus obtaining the gradient information corresponding to each cue embedding position.

[0083]

[0084] in, , For loss function Regarding the embedded matrix prompt The gradient matrix, This represents the loss function value of the target task model under the current prompt word.

[0085] The gradient corresponding to each cue embedding position is used to characterize the direction and degree of influence of the current position embedding on the task loss when the current position embedding changes. It serves as the basis for subsequent candidate word selection, word constraint update and cue embedding optimization, thereby using the gradient information provided by the task loss to guide the search of cue words in the continuous embedding space.

[0086] (ii) Vocabulary constraint update stage.

[0087] (2.1) Retrieve candidate lexical units: For each position of the cue embedding in the cue embedding matrix, calculate the Euclidean distance between each word embedding and the cue embedding at the current position in the word embedding set corresponding to the target model vocabulary, and select the closest one based on the distance. Each lexical element is selected as a candidate lexical element for the current position.

[0088] (2.2) Selecting lexical units based on gradient direction: For the candidate lexical units obtained in step (2.1), based on the directional consistency between the displacement direction of each candidate lexical unit relative to the current position and the negative gradient direction of the current position, the lexical unit with the highest directional consistency is selected as the word constraining lexical unit of the current position.

[0089] (2.3) Word vector hybrid update: Construct a word constrained word embedding matrix based on the word constrained words selected at each current position, and perform weighted fusion of the gradient descent update result of the current prompt embedding with the word vector corresponding to the word constrained word to obtain the updated prompt embedding matrix.

[0090] Specifically, for each cue embedding, the Euclidean distance between it and the corresponding word embedding vectors of each word in the task model vocabulary is calculated, and a predetermined number of word embedding vectors with the smallest distance are selected as the candidate word embedding set. In the candidate word embedding set, the candidate word element with the largest cosine value of the angle between the displacement direction and the negative gradient direction of the current position is selected as the constraint word element embedding, where the cosine value is the inner product of the displacement vector of the candidate word element relative to the current cue embedding and the negative gradient vector of the current position, divided by the product of the magnitude of the displacement vector and the magnitude of the negative gradient vector. The constraint word element embedding matrix is ​​composed of the constraint word elements selected at each position. The gradient descent update result and the constraint word element embedding matrix are weighted and fused according to the predetermined weights to obtain the updated cue embedding, where the fusion weights are used to control the ratio between gradient descent and vocabulary constraints.

[0091] The vocabulary constraint update phase can be implemented through the following steps.

[0092] S21. For each cue embedding in the cue embedding matrix obtained in step S12... Obtain the word embedding vectors corresponding to each word in the target task model vocabulary, and calculate the prompt embeddings respectively. Word embedding vectors corresponding to each lexical unit Euclidean distance between This measures the proximity of each word group to the current prompt embedding in the continuous embedding space.

[0093]

[0094] The word embedding vectors in the vocabulary are sorted according to the Euclidean distance, and the one with the smallest distance is selected. The word embedding vectors are used as the candidate word embedding set for the current position. This allows for the selection of a limited number of candidate words that are closely related to the current prompt embedding from the entire vocabulary, thereby reducing the search scope of the subsequent vocabulary constraint process.

[0095] S22. For the candidate word embedding set obtained in step S21 The gradient at the current position calculated in step S13 Determine the negative gradient direction of the current prompt embedding, and calculate the embedding of each candidate word relative to the current prompt embedding. The displacement direction is consistent with the negative gradient direction; from the candidate word embedding set, the candidate word embedding with the highest directional consistency with the negative gradient direction is selected as the word constrained lexical embedding corresponding to the current position, so that the selected word constrained lexical maintains both a close embedding distance with the current cue embedding and a high consistency with the current gradient descent direction. That is, for the candidate word embedding set that satisfies condition S21 ,choose Embedded as a constraint term.

[0096]

[0097] in, It is the gradient.

[0098] S23. Embedding the vocabulary constrained lexical elements selected for each prompt position. The lexical elements in the prompt words are combined according to their original positions to form a word list-constrained lexical embedding matrix. Based on the gradient information obtained in step S13, perform gradient descent update on the current cue embedding matrix, and then perform weighted fusion of the update result obtained by gradient descent with the word constrained lexical embedding matrix to obtain the updated cue embedding matrix.

[0099]

[0100] in, It's the learning rate. This controls the ratio between gradient descent and vocabulary constraints.

[0101] (III) Discretization and prompt length adjustment stage.

[0102] (3.1) Lexical pruning based on importance score: Based on the first-order importance score of each prompt embedding position on the task loss, lexicals whose contribution is lower than the preset threshold are deleted to obtain deployable discrete prompts.

[0103] Specifically, for each updated cue embedding, the absolute value of its inner product with the corresponding gradient is calculated, and the absolute value of the inner product at all positions is normalized to obtain the standardized first-order importance score of the loss at that position. If the score of a word is lower than a preset threshold, the word is deleted. This process is repeated iteratively until the cue converges or the cue length reaches the preset minimum length.

[0104] The discretization and prompt length adjustment stages can be implemented through the following steps.

[0105] S31. For each hint embedding in the updated hint embedding matrix obtained in step S23... The standardized first-order importance score of the cue position relative to the task loss is calculated based on the cue embedding and its corresponding gradient to measure the contribution of the current cue position to the change in task loss.

[0106]

[0107] in, For the first The normalized first-order importance score of each cue position. For the loss function with respect to the th A prompt embedding vector gradient vector, This indicates the index of the embedding location.

[0108] The importance score corresponding to each prompt location is compared with the preset pruning threshold. The algorithm compares the importance scores of different prompt positions. If the score is lower than the threshold, the corresponding word at that position is deleted to reduce prompts that contribute little to task performance. Words with importance scores not lower than the threshold are retained. This pruning process is repeated until the prompt optimization reaches a preset convergence condition or the current prompt length reaches a preset minimum prompt length. This allows us to obtain discrete prompt words with dynamically adjusted lengths.

[0109] Please see Figure 2When a task involves multiple sequentially executed models or processing steps, its optimization process also becomes a multi-step process.

[0110] (iv) Multi-step processing stage.

[0111] (4.1) Construct a multi-step task process: Connect multiple task models in a preset order and use the output of the previous task model as the input of the next task model to form a multi-step task processing process.

[0112] (4.2) Start optimizing the prompts from the last processing step, take the intermediate output obtained from the downstream step as the optimization target of the upstream step, construct the surrogate loss for the upstream step and calculate the surrogate gradient, and optimize the discrete prompts corresponding to each processing step in the order from downstream to upstream.

[0113] Specifically, multiple task models are chained together in a preset order, and the output of the previous task model is used as the input of the next task model to form a multi-step task processing flow. The prompts are optimized starting from the last processing step, and the intermediate outputs obtained from the downstream steps are used as the optimization targets of the upstream steps. A proxy loss is constructed for the upstream steps and the proxy gradient is calculated. The discrete prompts corresponding to each processing step are optimized step by step in the order from downstream to upstream.

[0114] Specifically, for each sample in the task dataset, it is processed sequentially by multiple models. The h-th model's output is obtained by concatenating the output of the previous step with the prompt word guiding the model and using it as input. The first model uses the original input as input. The joint optimization objective of the prompt words in each step is to minimize the loss between the final step output and the true label. Since the intermediate representations of multi-step tasks are discrete word units, the gradient of the final loss cannot be directly propagated to the upstream step. Therefore, this invention constructs a relay gradient: the updated word units obtained after optimization in the downstream step are used as the optimization objective of the upstream step. The surrogate loss of the upstream step is constructed using the loss between this optimization objective and the actual intermediate output of the upstream step, and the relay gradient of the prompt words in the upstream step is calculated from this. Thus, the prompt words of each step are gradually optimized from downstream to upstream.

[0115] Specifically, such as Figure 2 As shown, data input With the first prompt word Input task model To obtain intermediate output Intermediate output With the second cue word Input task model , to obtain the predicted output Based on the predicted output Update the second cue word to show the task loss between the actual output y and the output. And obtain the relay target corresponding to the updated second prompt word. Then, based on the intermediate output... With relay target Relay loss between the two, calculated from the first prompt word. The relay gradient is used to update the first prompt word. The task loss measures the difference between the predicted output and the actual output, while the relay loss measures the difference between the intermediate output and the relay target.

[0116] Following the above method, the relay targets obtained from the downstream processing steps are sequentially passed to the adjacent upstream processing steps, and the discrete prompt words corresponding to each processing step are optimized step by step in order from downstream to upstream. During the optimization process, the model parameters of each task model are kept frozen, and only the prompt words corresponding to each step are updated.

[0117] The multi-step processing stage can be implemented through the following steps.

[0118] S41. For the task dataset Each task sample is processed sequentially through multiple task models. These models are chained together according to a preset execution order, with the output of the previous model serving as the input for the next, thus forming a multi-step task processing flow. For the first... Task Model Set the corresponding prompt words. and the prompt words are compared with the input of the task model. By inputting the task model together, we obtain the first... Output corresponding to each processing step .

[0119]

[0120] in , It is guidance The prompt words, The index number is used to connect multiple task models sequentially in the above manner, and the loss between the final output and the task objective is used as the overall optimization objective for each task prompt word.

[0121]

[0122] in, They are respectively the 1st, ..., The optimal prompt word for each model It is the first The search space of prompt words for each model.

[0123] S42. For the multi-step task described in S41, since adjacent task models communicate through intermediate outputs in the form of discrete lexical units... Information transmission is performed, but the final task loss cannot be directly propagated to the prompts of the upstream task model through the discrete intermediate outputs. Therefore, optimization starts from the last task model, and the intermediate outputs obtained after optimizing the downstream task model are used. As the output of the corresponding upstream task model The optimization objective is to construct the proxy loss and calculate the relay gradient corresponding to the prompt word.

[0124]

[0125] in, This represents the updated lexical units obtained from optimizing the downstream model. By... As the optimization objective, construct The relay gradient is used to gradually optimize the prompts from downstream to upstream until the prompt words corresponding to each task model are optimized.

[0126] In other embodiments, the multi-step task is an image-based prompt word inversion task. This task includes multiple sequentially executed models or processing steps such as text encoding and image generation. The output tokens of the previous model serve as part of the input prompts for the next model, and the final optimization goal is to reconstruct the similarity between the image and the original image. Using the aforementioned reverse relay mechanism, optimization information is gradually propagated from the downstream image generation step to the upstream text encoding step, thereby obtaining better prompt words.

[0127] In some embodiments, each word in the discrete prompt belongs to the vocabulary of the target model. The discrete prompt is composed of a sequence of words in the vocabulary and can be directly edited, interpreted, and deployed in the target model. The discrete prompt is concatenated with the input data of the task to be optimized and then input into the target model. The target model executes the task to be optimized based on the discrete prompt. Users can also manually view and modify the optimized discrete prompt.

[0128] The invention will be further explained below with reference to comparative experiments.

[0129] The prompt optimization method described in this invention can be applied to different task scenarios, including few-shot text classification tasks based on classification models, text reasoning tasks based on language models, hub language machine translation tasks based on multi-model concatenation, and image prompt word inversion tasks based on multimodal models.

[0130] Table 1 shows the performance of the algorithm (RelayPrompt) described in this invention and other prompt word optimization algorithms in a few-shot text classification task scenario based on the RoBERTa-large (A Robustly Optimized Bidirectional Encoder Representations from Transformers Pretraining Approach) model. This scenario consists of three binary classification datasets (SST2 (Stanford Sentiment Treebank-2), MR (Movie Reviews), CR (Customer Reviews)) and three multi-class datasets (SST5 (Stanford Sentiment Treebank-5), Yelp5 (Yelp Review Full), AGNews (AG's News)). Other cue word optimization algorithms, such as BBTuning and GrIPS, use heuristic rules to update cue words, while GPO uses a large language model. In this process, GPO(GPT) and GPO(GLM) respectively use GPT-3.5-Turbo (Generative Pre-trained Transformer-3.5-Turbo) and GLM-5 (General Language Model-5) as cue word optimizers. Among reinforcement learning-based algorithms, this invention selects RLPrompt, TEMPERA, and StablePrompt for comparison. Gradient-based methods include Prompt Tuning, ID-SPAM (Input-Dependent Soft Prompting with aself-Attention Mechanism), AutoPrompt, and PEZ.

[0131] Table 1 shows the classification accuracy performance of the algorithm described in this invention (RelayPrompt) and other prompt word optimization algorithms in a few-shot text classification task based on a classification model. The best-performing data in Table 1 is marked in bold, and the second-best-performing data is marked with an underline. It can be seen that the algorithm described in this invention achieves the highest classification accuracy in both classification tasks in this scenario, with average accuracies of 91.5% (binary classification) and 60.7% (multi-class classification), respectively. On the SST5 and Yelp5 datasets, RelayPrompt outperforms the second-best baseline methods by 2.2% and 2.3%, respectively, demonstrating its effectiveness in complex tasks. For binary classification tasks, reinforcement learning-based methods show highly competitive performance, but their advantage diminishes in multi-class classification tasks. This is likely because as task complexity increases, reinforcement learning-based methods require a higher cost to converge.

[0132] Table 1: Classification accuracy performance of the algorithm described in this invention (RelayPrompt) and other prompt word optimization algorithms in few-shot text classification tasks based on classification models.

[0133] Table 2 shows the actual execution time (in minutes) and the number of trainable parameters for different prompt optimization methods in a few-shot text classification task scenario. The best-performing data in Table 2 is marked in bold, and the second-best-performing data is marked with an underline. The speedup ratio characterizes the running efficiency of the comparative optimization algorithm relative to the algorithm of this invention; it is defined as the ratio of the actual running time of the comparative optimization algorithm to the actual running time of the algorithm of this invention. It can be seen that on all datasets, RelayPrompt takes 2.6 to 8.2 minutes to learn the prompts, consistently faster than methods based on heuristic rules, large language models, or reinforcement learning. RelayPrompt is also 5.9 to 11.1 times faster than AutoPrompt. This high efficiency is mainly attributed to RelayPrompt's optimization in a continuous space, similar to continuous prompt optimization methods; therefore, its optimization time is comparable to Prompt Tuning and PEZ.

[0134] Table 2: Actual time consumption and number of trainable parameters of the algorithm described in this invention (RelayPrompt) and other prompt word optimization algorithms in a few-shot text classification task based on a classification model.

[0135] Table 3 shows the performance of the algorithm of this invention (RelayPrompt) and other prompt word optimization algorithms in a text reasoning task scenario based on a language model, using Llama-2-7B-Chat (Llama 2 7 Billion-Parameter Chat Model, Llama second-generation seven billion parameter dialogue model) as the task model. The best-performing data in Table 3 is marked in bold, and the second-best-performing data is marked with an underline. Three types of generative task datasets were selected for this scenario: GSM8K (GradeSchool Math 8K, a dataset of 8,000 elementary school math problems), BBH (BIG-Bench Hard, a BIG-Bench hard task benchmark), and MMLU (Massive Multitask Language Understanding, a large-scale multitask language understanding benchmark). The accuracy of the model's answer was used as the evaluation metric. In addition to the optimization algorithms mentioned above, the baseline methods for comparison also included non-optimization methods: Empty (no prompt words) and ManualPrompt (Manual Prompt, manually designed prompt words), representing the results using no prompt words and using initialized prompt words, respectively.

[0136] As shown in Table 3, RelayPrompt achieved the highest accuracy across all datasets, with an average accuracy of 35.2%, which is 4.2 percentage points higher than the second-ranked method, GPO (GLM). On the MMLU dataset, RelayPrompt achieved the largest performance improvement, leading the second-place method by 6.8 percentage points.

[0137] In model translation, translation quality is often limited if there is a lack of sufficient parallel language pairs between the source and target languages. An effective solution is to introduce a resource-rich hub language to connect the translation process.

[0138] Figure 3 This diagram illustrates the operation of the algorithm described in this invention in a hub language machine translation task based on a multi-model concatenation approach. Directly translating Arabic into Portuguese often results in low accuracy. Therefore, in this task flow, the Arabic sentence and prompt word A are first input into the first task model to translate into English, which serves as the hub language. Then, the English sentence and prompt word B are input into the second task model to translate into Portuguese. The translation result is evaluated based on the loss between the Portuguese translation and the corresponding standard translation (i.e., the loss in the diagram). Prompt words A and B are optimized to reduce this loss, thereby improving translation quality. During this process, the model parameters of the first and second task models remain frozen; only prompt words A and B are updated.

[0139] Table 3: The reasoning accuracy performance of the algorithm described in this invention (RelayPrompt) and other prompt word optimization algorithms using Llama-2-7B-Chat as the task model in a text reasoning task scenario based on a language model.

[0140] Table 4 shows the performance comparison of the method described in this invention with other methods in a multi-model concatenation-based hub language machine translation task scenario. The best-performing data in Table 4 is marked in bold, and the second-best-performing data is marked with an underline. The translation tasks include Arabic to Portuguese and Portuguese to Arabic, and the translation performance is evaluated using BLEU (Bilingual Evaluation Understudy), chrF++ (Character n-gram F-score plus Word n-grams), and COMET (Crosslingual Optimized Metric for Evaluation of Translation). BLEU measures the lexical matching degree between the candidate and reference translations, chrF++ measures the character and word-level matching degree, and COMET evaluates the overall semantic quality of the translation results. Higher scores on all three metrics indicate better translation quality. In the baseline methods, Direct represents directly translating the source language into the target language; SLP (Effective Pivot Language Strategies for Low-Resource Translation via Prompting Large Language Model) uses English as the pivot language; PIVOTE (Pivote, a pivot language-based translation method) uses English, Chinese, French, Russian, Spanish, and German as pivot languages ​​and utilizes an additional model, COMETkiwi (COMET Kiwi, a no-reference machine translation quality estimation model), to rank the translation candidates; GrIPS+Relay adapts the GrIPS method to the multi-step task framework described in this invention, and both GrIPS+Relay and the algorithm described in this invention use English as the pivot language. For all methods, including RelayPrompt, the Llama-3-8B-Instruct (Llama 3 8 Billion-Parameter Instruction Model, Llama third-generation eight billion parameter instruction model) model is used as the task model.

[0141] As shown in Table 4, RelayPrompt consistently outperforms all baseline methods. For example, in the Arabic-to-Portuguese translation task, RelayPrompt scores 4.15 BLEU, 0.03 chrF++, and 0.49 COMET higher than the second-best baseline method. Notably, PIVOTE performs the worst, which may be attributed to the influence of heterogeneous language noise. Using the multi-step task optimization framework described in this paper, GrIPS can also be extended to multi-step cue word optimization, achieving second-best results on two metrics in the Portuguese-to-Arabic translation task. This demonstrates the good versatility of the framework described in this embodiment, enabling the extension of existing cue word engineering methods to multi-step tasks.

[0142] Table 4: Translation performance of the algorithm described in this invention (RelayPrompt) and other hub language translation algorithms in a hub language machine translation task based on multi-model concatenation.

[0143] Figure 4 This demonstrates a multimodal model-based image cue word inversion task scenario. In this scenario, the original image and cue word A are input into a BLIP (Bootstrapping Language-Image Pre-training) model to generate an initial image description. Then, the initial image description and the original image are jointly input into a CLIP (Contrastive Language-Image Pre-training) model, and the image description is analyzed based on the graph-text loss between the image and the image description (i.e., the graph-text loss between the image and the image description). Figure 4 The image-text matching loss (image-text matching loss) measures the semantic consistency between the two. To reduce this loss, the image description is updated and the prompt word A is further optimized. A smaller image-text loss indicates a higher degree of semantic matching between the image description and the original image. After optimization, the final image description is input into a diffusion model to generate a reconstructed image. A larger-scale CLIP model is then used to calculate the similarity between the original and reconstructed images, and this similarity is used as the evaluation metric for this task.

[0144] Figure 5 Some image reconstruction results are shown, demonstrating that the algorithm of this invention (RelayPrompt) reconstructs images with finer-grained details from the original images, and the reconstruction effect is significantly better than the other two baseline methods.

[0145] Table 5 shows the CLIP similarity comparison of the reconstructed images between the algorithm described in this invention (RelayPrompt) and other algorithms in the image cue word inversion task based on a multimodal model. The best-performing data in Table 5 is marked in bold, and the second-best-performing data is marked with an underline. CLIP Inter. refers to the CLIP Interrogator method (an image cue word parsing tool), which adds similar lemmas to the image description based on the similarity measured in the embedding space. GrIPS+Relay is an algorithm obtained by adapting the GrIPS method to the multi-step task framework described in this invention. Four image datasets with different styles were selected for this scenario. As shown in the table, RelayPrompt achieved the highest reconstruction score on all four datasets. Compared to the suboptimal baseline PEZ, it improves scores by 4.25, 5.48, 5.80, and 5.06 on LAION (Large-scale Artificial Intelligence Open Network), MS COCO (Microsoft Common Objects in Context), Celeb-A (Celebrity Face Attributes), and Lexica.art (Lexica Image-Cue Word Pairs). Despite being equipped with the relay framework described in this embodiment, GrIPS+Relay performs poorly because it relies on heuristic search rather than gradient-based optimization.

[0146] Table 5: CLIP Similarity Table of Reconstructed Images and Original Images in Image Cue Inversion Task Based on Multimodal Model: The Algorithm (RelayPrompt) of this Invention and Other Cue Optimization Algorithms

[0147] Based on the same inventive concept, please see Figure 6 The present invention also provides a discrete prompting learning system based on vocabulary constraints, including a gradient calculation module, a vocabulary constraint update module, a discretization and length adjustment module, and a multi-step processing module.

[0148] The gradient calculation module is used to construct initial prompts for the task to be optimized, map the lexical units in the initial prompts to word vectors corresponding to the target model to obtain the prompt embedding, input the prompt embeddings into the target model, and calculate the gradient corresponding to the prompt embeddings based on the task loss.

[0149] The vocabulary constraint update module is used to retrieve candidate words that are close to each position in the prompt embedding from the word vector set corresponding to the vocabulary of the target model. Based on the directional consistency between the displacement direction of the candidate word relative to the prompt embedding at the current position and the negative gradient direction at the current position, the module selects vocabulary constraint words from the candidate words. The update result obtained based on the gradient is weighted and fused with the word vector of the vocabulary constraint words to obtain the updated prompt embedding.

[0150] The discretization and length adjustment module is used to discretize the updated prompts into discrete prompts composed of words in the vocabulary of the target model, and to delete words with scores lower than a preset threshold based on the importance score of each prompt embedding position to the task loss, so as to adjust the length of the discrete prompts.

[0151] The multi-step processing module is used when a task contains multiple models or processing steps that are executed sequentially. It connects multiple task models in a preset order, uses the output of the previous task model as the input of the next task model to form a multi-step task processing flow, optimizes the prompts starting from the last processing step, uses the intermediate outputs obtained from the downstream steps as the optimization targets of the upstream steps, constructs a proxy loss for the upstream steps and calculates the proxy gradient, and gradually optimizes the discrete prompts corresponding to each processing step in the order from downstream to upstream.

[0152] Unless otherwise defined, the technical or scientific terms used in this invention shall have the general meaning understood by one of ordinary skill in the art to which this invention pertains. Words such as “a,” “an,” “an,” “the,” “the,” and “these” used in this invention do not indicate quantitative limitations and may be singular or plural. The terms “comprising,” “including,” “having,” and any variations thereof used in this invention are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that comprises a series of steps or modules (units) is not limited to the listed steps or modules (units) but may include steps or modules (units) not listed, or may include other steps or modules (units) inherent to these processes, methods, products, or devices. Words such as “connected,” “linked,” and “coupled” used in this invention are not limited to physical or mechanical connections but may include electrical connections, whether direct or indirect. “A plurality” used in this invention refers to two or more. “And / or” describes the relationship between related objects, indicating that three relationships may exist; for example, “A and / or B” can represent: A alone, A and B simultaneously, and B alone. Normally, the character " / " indicates that the objects before and after it are in an "or" relationship. In this invention, the terms "first," "second," "third," etc., are used only to distinguish similar objects and do not represent a specific order of objects.

[0153] Obviously, the accompanying drawings are merely some examples or embodiments of the present invention. Those skilled in the art can apply the present invention to other similar situations based on these drawings without any creative effort. Furthermore, it is understood that although the work done in this development process may be complex and lengthy, certain design, manufacturing, or production modifications made according to the technical content disclosed in the present invention are merely conventional technical means and should not be considered as insufficient disclosure of the present invention.

[0154] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-described technical content to create equivalent embodiments without departing from the scope of the present invention. The implementation schemes in the above embodiments can also be further combined or replaced. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A discrete prompt learning method based on vocabulary constraints, characterized in that, Includes the following steps: S1. Construct initial prompt words for the task to be optimized, map the lexical units in the initial prompt words to the word vectors corresponding to the target model to obtain the prompt embedding, and input the prompt embedding into the target model. Calculate the task loss and the gradient corresponding to the prompt embedding based on the difference between the output of the target model and the task target. S2. In the word vector set corresponding to the vocabulary of the target model, retrieve candidate word elements that are close to each position in the prompt embedding; based on the consistency between the displacement direction of the candidate word element relative to the prompt embedding at the current position and the negative gradient direction at the current position, select vocabulary constraint word elements from the candidate word elements to constrain the current update direction. The updated result obtained based on gradient descent is weighted and fused with the word vectors of the vocabulary-constrained lexical units to obtain the updated prompt embedding; S3. Discretize the updated prompts into discrete prompts composed of lexical units from the vocabulary of the target model, and delete lexical units with scores lower than a preset threshold based on the importance score of each prompt embedding position to the task loss, to obtain discrete prompts with adjusted length that can be directly input into the target model to execute the task, i.e., optimized prompt words.

2. The method according to claim 1, characterized in that, For the task to be optimized, initial prompt words are constructed, and the lexical units in the initial prompt words are mapped to word vectors corresponding to the target model to obtain prompt embeddings. The prompt embeddings are then input into the target model. The task loss is calculated based on the difference between the target model output and the task objective, and the gradient corresponding to the prompt embeddings is calculated, including: Initial prompts are constructed for the task to be optimized, and the initial prompts are concatenated with the input data and then input into the target model, with minimizing task loss as the optimization objective; Based on the word sequence corresponding to the initial prompt word, each word is mapped to an embedding vector through the word embedding matrix of the target model, and a prompt embedding matrix is ​​constructed according to the order of each word in the prompt word. The cue embedding matrix is ​​input into the target model for forward computation, and the gradient corresponding to each embedding vector in the cue embedding matrix is ​​calculated based on the task loss.

3. The method according to claim 1, characterized in that, In the word vector set corresponding to the vocabulary of the target model, candidate word units that are close to each position in the prompt embedding are retrieved, including: For each cue embedding position, calculate the Euclidean distance between the cue embedding at that position and the word embedding vector corresponding to each word in the target model vocabulary, and select a preset number of words with the smallest distance to form a candidate word set; In the candidate lexical set, based on the directional consistency between the displacement direction of the candidate lexical relative to the current prompt embedding and the negative gradient direction of the current position, the candidate lexical that is most consistent with the negative gradient direction is selected as the vocabulary constraint lexical.

4. The method according to claim 3, characterized in that, Selecting the candidate lexical unit that is most consistent with the negative gradient direction as the word list constraint lexical unit includes: calculating the cosine value of the angle between the candidate lexical unit and the current position negative gradient vector relative to the current prompt embedding displacement vector, and selecting the candidate lexical unit with the largest cosine value as the word list constraint lexical unit.

5. The method according to claim 1, characterized in that, Based on the importance score of each prompt's embedding position to the task loss, words and phrases with scores below a preset threshold are removed, including: For each updated cue embedding position, calculate the absolute value of the inner product between the cue embedding position and the corresponding gradient, and normalize the absolute values ​​of the inner products of all cue embedding positions to obtain the first-order importance score of each cue embedding position. When the first-order importance score of the prompt embedding position is lower than the preset threshold, the word corresponding to the prompt embedding position is deleted.

6. The method according to claim 5, characterized in that, Deleting words with scores below a preset threshold includes iteratively performing scoring and deletion operations until the length of the prompt word reaches a preset minimum length, or the first-order importance scores of each prompt embedding position are not lower than the preset threshold.

7. The method according to claim 1, characterized in that, It also includes S4: When a task contains multiple models or processing steps that are executed sequentially, the multiple task models are chained together in a preset order, and the output of the previous task model is used as the input of the next task model to form a multi-step task processing flow. The optimization of the prompts begins with the last processing step. The intermediate output obtained from the downstream step optimization is used as the optimization target of the upstream step. A proxy loss is constructed for the upstream step and the proxy gradient is calculated. The discrete prompts corresponding to each processing step are optimized step by step in the order from downstream to upstream.

8. The method according to claim 7, characterized in that, Multiple task models are chained together in a preset order, including: The output of the previous task model is concatenated with the prompt of the next task model and used as the input of the next task model, so that each processing step is executed sequentially. With the goal of minimizing the final task loss, the prompts for each processing step are jointly optimized.

9. The method according to claim 7, characterized in that, The intermediate output obtained from the downstream optimization step is used as the optimization objective of the upstream step. A proxy loss is constructed and the proxy gradient is calculated for the upstream step, including: The intermediate output lexical units obtained after downstream optimization are used as the optimization targets of upstream steps. For the prompt words in the upstream step, a proxy loss is constructed based on the difference between the optimization objective and the actual intermediate output of the upstream step. The proxy gradient of the prompt words in the upstream step is calculated based on the proxy loss, and the prompt words in the upstream step are optimized based on the proxy gradient.

10. A discrete prompting learning system based on vocabulary constraints, characterized in that, include: The gradient calculation module is used to construct initial prompt words for the task to be optimized, map the lexical units in the initial prompt words to word vectors corresponding to the target model to obtain the prompt embedding, input the prompt embedding into the target model, calculate the task loss based on the difference between the output of the target model and the task target, and calculate the gradient corresponding to the prompt embedding. The vocabulary constraint update module is used to retrieve candidate word elements that are close to each position in the prompt embedding from the word vector set corresponding to the vocabulary of the target model; and select vocabulary constraint word elements from the candidate word elements to constrain the current update direction based on the directional consistency between the displacement direction of the candidate word element relative to the prompt embedding at the current position and the negative gradient direction at the current position. The updated result obtained based on gradient descent is weighted and fused with the word vectors of the vocabulary-constrained lexical units to obtain the updated prompt embedding; The discretization and length adjustment module discretizes the updated prompts into discrete prompts composed of lexical units from the vocabulary of the target model, and deletes lexical units with scores below a preset threshold based on the importance score of each prompt embedding position to the task loss, thus obtaining discrete prompts with adjusted length that can be directly input into the target model to execute the task.

Images