Gradient optimization-based hard prompt optimization method and device, equipment and storage medium
By performing gradient optimization on the initial hard cues to generate optimized hard cues, the problems of time-consuming hard cue writing and uninterpretable soft cues are solved, achieving automated and efficient hard cue optimization and reducing the training cost of large language models.
Patent Information
- Application Number
- CN202511397200.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2025-09-28
- Publication Date
- 2025-12-23
- Estimated Expiration
- 2045-09-28
AI Technical Summary
Existing hard cues require manual writing by experts, resulting in high training costs and time consumption for large language models, while soft cues lack intuitiveness and are difficult to understand.
By obtaining the embedding vector of the initial manually written hard cue, gradient optimization is performed to obtain the optimized embedding vector, which is then mapped to the vocabulary to generate a discrete word sequence, and finally the optimized hard cue is determined.
It reduces the time spent manually writing hard suggestions, improves the interpretability of gradient optimization, avoids the "black box" problem of soft suggestions, and reduces the resource consumption of training large language models.
Smart Images
Figure CN120874760B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of large language model, in particular to a hard prompt optimization method and device based on gradient optimization, equipment and storage medium. BACKGROUND
[0002] Prompt as a natural language processing technology has achieved remarkable results in text classification, question answering, dialogue and other tasks. According to the prompt information type and expression method of the prompt, it can be divided into hard prompt and soft prompt. Among them, the hard prompt is through a fixed prompt template, which directly embeds keywords or phrases to guide the large language model to generate text, and has the advantages of intuitive and easy to understand and stable effect, and is often used in dialogue, question answering and other tasks. Soft prompt is a parameterizable prompt template that can be adjusted according to specific tasks and requirements, and has the advantages of flexibility and strong generalization ability, and is widely used in text classification, question answering and other tasks.
[0003] However, the hard prompt needs to be manually written by experts, and usually involves a lot of trial and error process, so that the model construction cost is very time-consuming and high in cost; the soft cost is a group of continuous and learnable parameters, which usually does not have intuitive meaning, so that the process of how the large language model is guided to generate specific output is difficult to understand, and because the newly introduced parameters need to be loaded additionally during reasoning, the resource consumption cost of the large language model training is high. SUMMARY
[0004] The main purpose of the present application is to provide a hard prompt optimization method and device based on gradient optimization, equipment and storage medium, which aims to solve the technical problem that the existing prompt makes the cost of large language model training high.
[0005] To achieve the above purpose, the present application provides a hard prompt optimization method based on gradient optimization, which comprises:
[0006] An initial manually written hard prompt is obtained, and a first embedding vector corresponding to the hard prompt in a vocabulary table is determined;
[0007] The first embedding vector is taken as an optimization parameter for gradient optimization to obtain a second embedding vector after optimization;
[0008] The second embedding vector is mapped to the vocabulary table to obtain a discrete token sequence corresponding to the second embedding vector;
[0009] An optimized hard prompt is determined based on the discrete token sequence.
[0010] In an embodiment, the step of gradient optimizing the first embedding vector as an optimization parameter to obtain an optimized second embedding vector comprises:
[0011] adding random noise to the first embedding vector to obtain an initial optimization embedding vector;
[0012] taking the initial optimization embedding vector as an optimization parameter and obtaining a current optimization gradient of the optimization parameter based on a gradient-optimized loss function;
[0013] optimizing learning of the optimization parameter based on the optimization gradient and a preset learning rate to obtain an intermediate optimization embedding vector;
[0014] taking the intermediate optimization embedding vector when a gradient iteration termination condition is met as a second embedding vector.
[0015] In an embodiment, after the step of optimizing learning of the optimization parameter based on the optimization gradient and a preset learning rate to obtain an intermediate optimization embedding vector, the method further comprises:
[0016] if the gradient iteration termination condition is not met, taking the intermediate optimization embedding vector as an optimization parameter and returning to the step of obtaining a current optimization gradient of the optimization parameter based on a gradient-optimized loss function.
[0017] In an embodiment, the step of obtaining a current optimization gradient of the optimization parameter based on a gradient-optimized loss function comprises:
[0018] obtaining a preset training data set and randomly sampling training data in the preset training data set;
[0019] determining a partial derivative of the loss function with respect to the optimization parameter based on the sampled training data;
[0020] taking the partial derivative as the current optimization gradient of the optimization parameter.
[0021] In an embodiment, before the step of obtaining a current optimization gradient of the optimization parameter based on a gradient-optimized loss function, the method further comprises:
[0022] obtaining an embedding matrix of a large language model;
[0023] taking the optimization parameter as a prefix embedding of the large language model to obtain a sequence loss function;
[0024] obtaining a regularization loss function of the optimization parameter based on the embedding matrix;
[0025] obtaining a gradient-optimized loss function based on the sequence loss function and the regularization loss function.
[0026] In an embodiment, the step of mapping the second embedding vector to the vocabulary to obtain a discrete token sequence corresponding to the second embedding vector comprises:
[0027] obtaining a preset projection function;
[0028] mapping the second embedding vector to a token index in the vocabulary closest to the second embedding vector based on the preset projection function to obtain a discrete token sequence.
[0029] In an embodiment, the step of determining the first embedding vector corresponding to the hard prompt in the vocabulary comprises:
[0030] performing tokenization processing on the hard prompt based on a large language model tokenizer to obtain a hard prompt token;
[0031] determining the first embedding vector corresponding to the hard prompt token in the vocabulary of the large language model.
[0032] In addition, to achieve the above object, the present application also proposes a hard prompt optimization device based on gradient optimization, which comprises:
[0033] a tokenization module configured to obtain an initial manually written hard prompt and determine a first embedding vector corresponding to the hard prompt;
[0034] an optimization module configured to perform gradient optimization on the first embedding vector as an optimization parameter to obtain a second embedding vector after optimization;
[0035] a projection module configured to map the second embedding vector to the first embedding vector to obtain a discrete token sequence;
[0036] a decoding module configured to determine an optimized hard prompt based on the discrete token sequence.
[0037] In addition, to achieve the above object, the present application also proposes a hard prompt optimization device based on gradient optimization, which comprises: a memory, a processor, and a computer program stored on the memory and executable on the processor, the computer program being configured to implement the steps of the hard prompt optimization method based on gradient optimization as described above.
[0038] In addition, to achieve the above object, the present application also proposes a storage medium, which is a computer readable storage medium, and the storage medium stores a computer program, and the computer program is executed by a processor to implement the steps of the hard prompt optimization method based on gradient optimization as described above.
[0039] In addition, to achieve the above object, the application further provides a computer program product, which comprises a computer program, and the computer program, when executed by a processor, implements the steps of the hard prompt optimization method based on gradient optimization as described above.
[0040] The one or more technical solutions provided in the application have at least the following technical effects:
[0041] The application obtains an initial manually written hard prompt, determines a first embedding vector corresponding to the hard prompt in a vocabulary table, performs gradient optimization on the first embedding vector as an optimization parameter to obtain a second embedding vector after optimization, maps the second embedding vector to the vocabulary table to obtain a discrete token sequence corresponding to the second embedding vector, and decodes based on the discrete token sequence to obtain an optimized hard prompt. Since the initial manually written hard prompt is optimized, compared with the traditional scheme of writing a complete and comprehensive hard prompt, the time for manual writing is saved, making it more automated and intelligent. The initial manually written hard prompt is used as a guide for initialization, so that the explainability in the gradient optimization process is stronger. The tokenizer performs tokenization processing on the hard prompt and obtains the corresponding embedding vector in the vocabulary table, so that the initial manually written hard prompt can be iteratively optimized like a soft prompt, providing a basis for gradient iterative optimization of the hard prompt. By performing gradient optimization on the first embedding vector as an optimization parameter and mapping the second embedding vector after optimization back to the token of the vocabulary table, it is ensured that the final output is a sequence composed of natural language, avoiding the "black box" problem of the soft prompt and improving the explainability of the prompt word guided model process. BRIEF DESCRIPTION OF DRAWINGS
[0042] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the application and serve to explain the principles of the application together with the specification.
[0043] In order to more clearly illustrate the technical solutions in the embodiments of the application or the prior art, the accompanying drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, for those skilled in the art, other drawings can also be obtained based on these drawings without creative labor.
[0044] Figure 1 A flowchart is provided for the hard prompt optimization method based on gradient optimization of the application embodiment one;
[0045] Figure 2 A flowchart is provided for the hard prompt optimization method based on gradient optimization of the application embodiment two;
[0046] Figure 3A flowchart provided by the third embodiment of the hard prompt optimization method based on gradient optimization of the present application is shown in the figure.
[0047] Figure 4 A module structure diagram of the hard prompt optimization device based on gradient optimization of the embodiment of the present application is shown in the figure.
[0048] Figure 5 A device structure diagram of the hardware running environment involved in the hard prompt optimization method based on gradient optimization in the embodiment of the present application is shown in the figure.
[0049] The object implementation, functional features and advantages of the present application will be further described with reference to the accompanying drawings in conjunction with the embodiments. DETAILED DESCRIPTION
[0050] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and do not limit the present application.
[0051] In order to better understand the technical solutions of the present application, the following will be described in detail in conjunction with the drawings and specific embodiments of the specification.
[0052] The main solution of the embodiment of the present application is: obtaining an initial hard prompt manually written, and determining a first embedding vector corresponding to the hard prompt; performing gradient optimization on the first embedding vector as an optimization parameter to obtain a second embedding vector after optimization; mapping the second embedding vector to the first embedding vector to obtain a discrete token sequence; and decoding based on the discrete token sequence to obtain an optimized hard prompt.
[0053] In order to make up for the problems existing in hard prompts and soft prompts, the present application provides a solution to optimize hard prompts by using gradient iterative optimization, thereby saving the time of manually writing hard prompts, making hard prompts be able to be automatically optimized like soft prompts, and combining the optimization of hard prompts and soft prompts. At the beginning of gradient optimization, the hard prompt of the embodiment of the present application is guided and initialized by preliminary manual writing, so that the explainability in the gradient optimization process is stronger, the convergence speed of the model is accelerated, and it is helpful to avoid falling into local optimal solution, and the efficiency of the whole optimization process is improved. At the same time, the preliminary manually written hard prompt of the present application has intuitive meaning, which can more clearly explain the guiding way of the large language model to generate a specific output. In addition, the final optimization result of the present application is the hard prompt, which avoids the resource consumption of introducing additional parameters by the soft prompt, reduces the number of parameters loaded by the large language model during inference, and improves the system efficiency.
[0054] It should be noted that the execution subject of the embodiment can be a computing service device with data processing, network communication and program running functions, such as a computer, a server, or an electronic device, a virtual device, etc. capable of realizing the above functions. The following will take the hard prompt optimization based on gradient optimization (referred to as optimization device) as an example to illustrate the embodiment and the following embodiments.
[0055] Based on this, the application provides a hard prompt optimization method based on gradient optimization, which is described with reference to Figure 1 , Figure 1 The flowchart provided by Embodiment One of the hard prompt optimization method based on gradient optimization of the application.
[0056] In this embodiment, the hard prompt optimization method based on gradient optimization includes steps S10-S40:
[0057] Step S10, obtaining an initial manually written hard prompt, and determining a first embedding vector corresponding to the hard prompt in a vocabulary table.
[0058] It can be understood that the hard prompt (Hard Prompt) is also called discrete prompt, which refers to the instruction using human interpretable and reusable handmade words or marks to guide the output of the large language model. The pre-training semantics carried by the hard prompt can guide the training of the large language model.
[0059] It should be noted that the embedding vector (Embedding) is a continuous, low-dimensional, structured numerical vector based on the hard prompt. The first embedding vector mentioned above is the embedding vector corresponding to the initial manually written hard prompt. In order to obtain the first embedding vector, in the embodiment of the application, the step of determining the first embedding vector corresponding to the hard prompt in the vocabulary table includes: performing word segmentation processing on the hard prompt based on the tokenizer of the large language model to obtain hard prompt tokens; determining the first embedding vector corresponding to the hard prompt token in the vocabulary table of the large language model.
[0060] It can be understood that the tokenizer (Tokenizer) mentioned above is a component that can convert human-readable text (such as a hard prompt) into a sequence of basic units that can be understood by a large language model. Token is a basic unit after cutting human-readable text, which can be a single character or at least two characters to form a word, and the embodiment of the application does not limit this. The hard prompt token mentioned above is the token obtained by performing word segmentation processing on the hard prompt. For example, the hard prompt "Hello World" is cut into tokens "Hello" and "World". In the large language model, each unique token can be assigned a unique integer ID, i.e. token index (Token ID).
[0061] It should be understood that the above vocabulary table is a data table capable of storing the correspondence between wordpieces and wordpiece indexes, and the correspondence between wordpiece indexes and embedding vectors. Before training the large language model, the tokenizer can be trained on the text corpus first. By learning the statistical rules in the tokenizer corpus, it can be determined which word combinations are commonly used, thereby forming the vocabulary table corresponding to the large language model. Through the vocabulary table, the correspondence between the wordpieces, the wordpiece indexes and the embedding vectors in the embedding matrix can be determined. Specifically, through the tokenizer, the hard prompt can be divided into several wordpieces for processing, and by merging, searching and other operations on these wordpieces in the vocabulary table, the corresponding wordpiece indexes and embedding vectors of these wordpieces can be determined.
[0062] It should be noted that the tokenizer and the tokenization method used in the embodiments of the present application are not limited, and can be set based on the needs in actual applications.
[0063] It can be understood that when the Token index corresponding to the prompt word is obtained, the corresponding embedding vector (Embedding vector), i.e., the first embedding vector, can be obtained from the embedding matrix (Embedding matrix) of the large language model. The specific process of obtaining the embedding vector is not limited in the embodiments of the present application, and can be set according to the needs in actual applications.
[0064] It should be noted that the embedding matrix in the embodiments of the present application is a vocabulary embedding matrix, which is a core component in a large language model and can be used as a lookup table to map discrete Token indexes to continuous, high-dimensional vector representations to obtain embedding vectors. In the embodiments of the present application, the size of the embedding matrix can be |V|*d, where |V| is the size of the vocabulary table and d is the dimension size of the embedding vector corresponding to each Token index in the vocabulary table. The embedding matrix can be used as a lookup table, and each row (corresponding to a wordpiece index) in the embedding matrix can be a d-dimensional vector.
[0065] In a specific implementation, the optimization device can obtain an initial manually written hard prompt, perform word segmentation on the hard prompt based on a word segmenter of a large language model to obtain wordpieces corresponding to the hard prompt, obtain a wordpiece index corresponding to the wordpieces in a vocabulary table, and obtain a first embedding vector in an embedding matrix based on the wordpieces. Since the initial manually written hard prompt is optimized, compared with a traditional scheme of writing a complete and comprehensive hard prompt, the time for manual writing is saved, making the process more automated and intelligent. The initial manually written hard prompt is used as a guide for initialization, so that the explainability in the gradient optimization process is stronger. The hard prompt is segmented by the word segmenter, and the embedding vector is obtained based on the vocabulary table and the embedding matrix, so that the initial manually written hard prompt can be iteratively optimized like a soft prompt, providing a basis for gradient iterative optimization of the hard prompt.
[0066] In step S20, the first embedding vector is used as an optimization parameter for gradient optimization to obtain a second embedding vector after optimization.
[0067] In step S30, the second embedding vector is mapped to the vocabulary table to obtain a discrete wordpiece sequence corresponding to the second embedding vector.
[0068] It should be noted that gradient optimization is an iterative optimization process of gradually minimizing a loss function by calculating an optimization gradient and adjusting the parameters in the opposite direction based on the optimization gradient. The second embedding vector is an embedding vector obtained after gradient iterative optimization of the first embedding vector. The discrete wordpiece sequence is a discrete sequence composed of wordpieces corresponding to the second embedding vector in the vocabulary table.
[0069] It should be explained that, in order to obtain a discrete discrete wordpiece sequence, any point (second embedding vector after optimization) in the continuous vector space after gradient optimization can be mapped back to the original fixed and discrete set by projection, that is, the second embedding vector is mapped to the wordpieces of the vocabulary table to obtain the corresponding discrete wordpiece sequence. By projecting the optimized continuous vector back to the original embedding space, it is ensured that the final output is a sequence composed of natural language, avoiding the "black box" problem of soft prompts and improving the explainability of the prompt word guided model process.
[0070] In step S40, the optimized hard prompt is determined based on the discrete wordpiece sequence.
[0071] It can be understood that, by being based on the above vocabulary and embedding matrix, reverse decoding of the discrete token sequence can be implemented to obtain the optimized hard prompt. In some embodiments of the embodiments of the present application, a discrete token sequence corresponding to the second embedding vector can be determined on the embedding matrix. Based on the discrete token sequence, splicing can be performed to obtain the optimized hard prompt. By using the bidirectional mapping relationship between the token obtained by training the large language model and the Token index and the Token index and the embedding vector, the optimized hard prompt can be translated into natural language, thereby realizing automatic hard prompt optimization.
[0072] The embodiments of the present application obtain the initial manually written hard prompt, determine the first embedding vector corresponding to the hard prompt in the vocabulary, perform gradient optimization on the first embedding vector as an optimization parameter to obtain the second embedding vector after optimization, map the second embedding vector to the vocabulary to obtain the discrete token sequence corresponding to the second embedding vector, and decode based on the discrete token sequence to obtain the optimized hard prompt. Since the initial manually written hard prompt is optimized, compared with the traditional scheme of writing a complete and comprehensive hard prompt, the time of manual writing is saved, making it more automated and intelligent. The initial manually written hard prompt is used as a guide initialization, so that the explainability in the gradient optimization process is stronger. The tokenizer is used for tokenization processing of the hard prompt to obtain the corresponding embedding vector in the vocabulary, so that the initial manually written hard prompt can be iteratively optimized like a soft prompt, providing a basis for gradient iterative optimization of the hard prompt. By performing gradient optimization on the first embedding vector as an optimization parameter and mapping the second embedding vector after optimization back to the token of the vocabulary, it is ensured that the final output is a sequence composed of natural language, avoiding the "black box" problem of the soft prompt and improving the explainability of the prompt word guided model process.
[0073] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as the above embodiment one can refer to the above introduction, and will not be described in detail. On this basis, please refer to Figure 2 , Figure 2 for the flowchart of the second embodiment of the hard prompt optimization method based on gradient optimization of the present application.
[0074] As Figure 2 shown, in the embodiments of the present application, the step of performing gradient optimization on the first embedding vector as an optimization parameter to obtain the second embedding vector after optimization includes:
[0075] Step S21, adding random noise to the first embedding vector to obtain an initial optimization embedding vector.
[0076] It can be understood that the random noise can be a set of values randomly sampled from a predefined probability distribution (such as a Gaussian distribution, a uniform distribution, etc.). By adding random noise to the first embedding vector, it is possible to avoid falling into a local optimal solution.
[0077] In some embodiments of the present application, the first embedding vector can be denoted as P. Each of the may be a d-dimensional vector (the specific dimension d can be set based on the requirements in actual application). By adding random noise δ to the first embedding vector P, an initial optimization embedding vector P can be obtained. That is, .
[0078] Step S22, taking the initial optimization embedding vector as an optimization parameter, and obtaining a current optimization gradient of the optimization parameter based on a gradient-optimized loss function.
[0079] It should be noted that when gradient optimization is performed, the number of iterations T for iterative optimization can be set based on the requirements in actual application, and the present application does not limit this.
[0080] In some embodiments of the present application, for each step of the iterative process of gradient optimization, a small batch of training data (X, Y) can be randomly sampled from a preset training data set. Through the training data, the gradient g of the loss function with respect to the optimization parameter can be determined. Specifically, the step of obtaining a current optimization gradient of the optimization parameter based on a gradient-optimized loss function includes: obtaining a preset training data set and randomly sampling training data in the preset training data set; determining the partial derivative of the loss function with respect to the optimization parameter based on the sampled training data; and taking the partial derivative as the current optimization gradient of the optimization parameter.
[0081] It should be noted that the above-mentioned preset training data set can be a data set collected in advance for gradient optimization of the optimization parameter. Wherein, X represents the input of the large language model, and Y represents the expected output of the large language model.
[0082] It should be explained that the partial derivative of a multivariate function with respect to a certain independent variable is the rate of change of the multivariate function with respect to the independent variable when all other independent variables in the multivariate function are fixed. Specifically, by determining the partial derivative of the loss function with respect to the optimization parameter, it is possible to measure the rate of change of the loss function value with respect to the optimization parameter when all other independent variables (such as model input, model output, model parameter, etc.) are fixed. The rate of change can represent the change gradient of the optimization parameter, and can be taken as the current optimization gradient.
[0083] In some embodiments of the embodiments of the present application, in order to realize gradient optimization, before the step of obtaining the current optimization gradient of the optimization parameter based on the gradient optimization loss function, the method further comprises: obtaining an embedding matrix of the large language model; embedding the optimization parameter as a prefix embedding of the large language model to obtain a sequence loss function; obtaining a regularization loss function of the optimization parameter based on the embedding matrix; and obtaining a gradient optimization loss function based on the sequence loss function and the regularization loss function.
[0084] It should be noted that the above sequence loss function can be used to evaluate the task loss of the large language model, and ensure that the optimized hard prompt can effectively guide the large language model to complete a specific task. By embedding the optimization parameter as a prefix embedding of the large language model, the sequence loss function L1 can be obtained as follows:
[0085] ;
[0086] wherein, is the optimization parameter (such as an initial optimization embedding vector); is the input of the large language model; Y is the expected output of the large language model; is the frozen parameter of the large language model, i.e., the parameter of the large language model that is not updated.
[0087] It should be noted that the above regularization loss function can be used to ensure that the optimized hard prompt does not deviate too far from the true word, and to ensure the interpretability of the final result. The regularization loss L2 of the optimization parameter can be obtained through the embedding matrix, and can be specifically as follows:
[0088] ;
[0089] wherein, represents a hyperparameter, which can be set according to actual application; is a distance function, which is used to represent the distance of each embedding vector in the embedding matrix E.
[0090] It should be noted that the gradient optimization loss function L can be obtained based on the above sequence loss function and regularization loss function, and can be specifically as follows:
[0091] ;
[0092] It can be understood that the optimization gradient g of the loss function with respect to the optimization parameter can be calculated through the above loss function, and can be specifically as follows:
[0093] ;
[0094] wherein, This represents the partial derivative of the loss function L with respect to the optimization parameter P.
[0095] Step S23: Based on the optimization gradient and the preset learning rate, optimize the optimization parameters to obtain the intermediate optimized embedding vector;
[0096] Step S24: Use the intermediate optimized embedding vector that satisfies the gradient iteration termination condition as the second embedding vector.
[0097] It should be noted that the aforementioned preset learning rate is a parameter used to control the parameter update speed during gradient iteration optimization, and it directly determines the efficiency and effectiveness of model convergence. The value of the preset learning rate can be set according to the needs of actual applications, and this embodiment does not impose any restrictions on its specific value. Specifically, the formula for optimizing the parameters can be as follows:
[0098] ;
[0099] in, This indicates the preset learning rate. P represents the optimized parameters after optimization learning (such as intermediate optimized embedding vectors); P represents the optimized parameters before optimization learning.
[0100] It is understood that the aforementioned intermediate optimized embedding vector is the embedding vector obtained after optimizing the optimization parameters. When the gradient iteration termination condition is met, the intermediate optimized embedding vector can be used as the second embedding vector; if the gradient iteration termination condition is not met, the intermediate optimized embedding vector can be used as the optimization parameter for further gradient iteration optimization. Specifically, after the step of optimizing the optimization parameters based on the optimization gradient and a preset learning rate to obtain the intermediate optimized embedding vector, the process further includes: if the gradient iteration termination condition is not met, using the intermediate optimized embedding vector as the optimization parameter and returning to the step of obtaining the current optimization gradient of the optimization parameter using the gradient-optimized loss function.
[0101] It should be noted that the above-mentioned gradient iteration termination condition can be the number of iterations reaching the preset number of iterations T, the iteration reaching convergence, or other gradient iteration termination conditions. The embodiments of this application do not limit this.
[0102] The embodiment of the application obtains an initial optimization embedding vector by adding random noise to the first embedding vector; takes the initial optimization embedding vector as an optimization parameter, and obtains a current optimization gradient of the optimization parameter based on a loss function of gradient optimization; optimizes and learns the optimization parameter based on the optimization gradient and a preset learning rate to obtain an intermediate optimization embedding vector; and takes the intermediate optimization embedding vector when a gradient iteration termination condition is met as the second embedding vector. Since random noise is added to the first embedding vector, an initial gradient is provided for gradient optimization, ensuring that the gradient is non-zero and accelerating the optimization process. At the same time, by introducing randomness, it is avoided that the gradient optimization falls into a local optimal solution. By gradient optimization of the optimization parameter, the hard prompt is converted from a production process dependent on expert experience and manual trial and error to an automatic process driven by data and a loss function. By fine-tuning the embedding vector corresponding to the hard prompt through the gradient, the loss can be reduced to the greatest extent, and the task performance is improved. The intermediate optimization embedding vector is taken as the second embedding vector when the iteration termination condition is met, preventing model overfitting and improving the generalization ability.
[0103] Based on the first embodiment and / or the second embodiment of the application, in the third embodiment of the application, the same or similar contents as the above-mentioned embodiment one and / or embodiment two can be referred to the above introduction, and will not be described in detail hereinafter. On this basis, please refer to Figure 3 , Figure 3 The flowchart provided in the third embodiment of the hard prompt optimization method based on gradient optimization of the application is shown in the figure.
[0104] As shown in Figure 3 , in the embodiment of the application, the step of mapping the second embedding vector to the vocabulary table to obtain a discrete token sequence corresponding to the second embedding vector comprises:
[0105] Step S31, obtaining a preset projection function;
[0106] Step S32, mapping the second embedding vector to the nearest token index in the vocabulary table based on the preset projection function, to obtain a discrete token sequence.
[0107] It should be noted that the above-mentioned preset projection function can be a function capable of mapping any point in a continuous vector space back to the nearest point in a vocabulary table. In the embodiment of the application, the continuous second embedding vector can be mapped to the nearest discrete token in the vocabulary table by the preset projection function to obtain a discrete token sequence, and then the discrete token sequence is spliced to obtain the final optimized hard prompt. The specific projection process can be as follows:
[0108] ;
[0109] Among them, represents a preset projection function, token is a discrete word corresponding to the second embedding vector, and P2 is the second embedding vector.
[0110] It can be understood that the corresponding discrete word sequence can be determined based on the above-mentioned optimized embedding vector.
[0111] It should be noted that the preset projection function or projection method used by the projection can be selected based on actual application, such as the nearest neighbor method, a neural network model, etc., and the embodiments of the present application do not limit this.
[0112] The embodiments of the present application obtain a preset projection function; map the second embedding vector to the word index closest to the second embedding vector in the vocabulary based on the preset projection function, and obtain a discrete word sequence. Since the optimized first embedding vector is projected into the embedding vector closest to the first embedding vector, the high-performance solution found in the continuous space by gradient optimization is converted into natural language, and an optimized hard prompt with both soft prompt performance and hard prompt availability is obtained.
[0113] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the hard prompt optimization method based on gradient optimization of the present application. More forms of simple transformation based on this technical concept are within the protection scope of the present application.
[0114] The present application also provides a hard prompt optimization device based on gradient optimization, which is described with reference to Figure 4 , Figure 4 is a schematic diagram of the module structure of the hard prompt optimization device based on gradient optimization of the embodiments of the present application. The hard prompt optimization device based on gradient optimization comprises:
[0115] The word segmentation module 10 is configured to obtain an initial manually written hard prompt and determine a first embedding vector corresponding to the hard prompt.
[0116] The optimization module 20 is configured to perform gradient optimization on the first embedding vector as an optimization parameter to obtain a second embedding vector.
[0117] The projection module 30 is configured to map the second embedding vector to the vocabulary to obtain a discrete word sequence corresponding to the second embedding vector.
[0118] The decoding module 40 is configured to determine an optimized hard prompt based on the discrete word sequence.
[0119] The hard prompt optimization device based on gradient optimization provided in the application adopts the hard prompt optimization method based on gradient optimization in the above embodiment, and can solve the technical problem that the existing prompt word makes the cost of large language model training higher. Compared with the prior art, the hard prompt optimization device based on gradient optimization provided in the application has the same beneficial effects as the hard prompt optimization method based on gradient optimization provided in the above embodiment, and other technical features in the hard prompt optimization device based on gradient optimization are the same as the features disclosed in the above embodiment method, which will not be repeated here.
[0120] The application provides a hard prompt optimization device based on gradient optimization, which comprises at least one processor and a memory connected in communication with the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the hard prompt optimization method based on gradient optimization in the above embodiment one.
[0121] Reference will be made to the accompanying drawings Figure 5 which shows a structural schematic diagram of the hard prompt optimization device based on gradient optimization suitable for being used to implement the embodiments of the application. The hard prompt optimization device based on gradient optimization in the embodiments of the application can include but is not limited to mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (Personal Digital Assistant), PADs (Portable Application Description), PMPs (Portable Media Player), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), and the like, and fixed terminals such as digital TVs, desktop computers, and the like. Figure 5 The hard prompt optimization device based on gradient optimization shown is only an example, and should not bring any limitation to the functions and use range of the embodiments of the application.
[0122] As Figure 5As shown, the gradient-based hard prompt optimization device can include a processing device 1001 (e.g., a central processing unit, a graphics processing unit, etc.) that can perform various appropriate actions and processes according to programs stored in a read-only memory (ROM) 1002 or loaded from a storage device 1003 into a random access memory (RAM) 1004. Various programs and data required for the operation of the gradient-based hard prompt optimization device are also stored in the RAM 1004. The processing device 1001, the ROM 1002, and the RAM 1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; the storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the gradient-based hard prompt optimization device to communicate with other devices wirelessly or by wire to exchange data. Although the gradient-based hard prompt optimization device with various systems is shown in the figure, it should be understood that all the shown systems are not required to be implemented or possessed. More or fewer systems can be alternatively implemented or possessed.
[0123] In particular, according to embodiments of the present disclosure, the processes described above with reference to the flowcharts can be implemented as a computer software program. For example, embodiments of the present disclosure include a computer program product comprising a computer program carried on a computer readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network through a communication device, or installed from the storage device 1003, or installed from the ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the methods of the embodiments of the present disclosure are performed.
[0124] The gradient-based hard prompt optimization device provided by the present disclosure adopts the gradient-based hard prompt optimization method in the above-mentioned embodiments, and can solve the technical problem that the existing prompt words make the cost of training a large language model higher. Compared with the prior art, the gradient-based hard prompt optimization device provided by the present disclosure has the same beneficial effects as the gradient-based hard prompt optimization method provided by the above-mentioned embodiments, and other technical features in the gradient-based hard prompt optimization device are the same as the features disclosed in the previous embodiment method, which will not be repeated here.
[0125] It should be understood that various aspects disclosed herein can be implemented in hardware, software, firmware, or combinations thereof, to achieve the various aspects disclosed herein. In the description above, specific features, structures, materials or characteristics can be combined in any suitable manner without necessarily being limited to cases where such features, structures, materials or characteristics are combined.
[0126] The above description is provided as an enabling teaching of the application and is not intended to limit the scope of the application. Thus, any modification of the method of gradient-based hard prompt optimization described herein, which comes within the scope of the application claimed, is to be considered as within the scope of the application. Therefore, the scope of the application is defined by the appended claims.
[0127] The application provides a computer readable storage medium having stored thereon computer readable program instructions (i.e., computer programs) for performing the gradient-based hard prompt optimization method of the above-described embodiments.
[0128] The computer readable storage medium provided by the application may, for example, be a U disk, but is not limited to an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, system, or device, or any combination thereof. More specific examples of the computer readable storage medium can include, but are not limited to, an electrical connection having one or more conductive wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM) or a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present embodiment, the computer readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer readable storage medium can be transmitted in any suitable medium, including but not limited to electrical wire, optical cable, RF (Radio Frequency, RF), etc., or any suitable combination of the above.
[0129] The above computer readable storage medium can be included in the gradient-based hard prompt optimization device; or can exist separately and not be assembled into the gradient-based hard prompt optimization device.
[0130] The computer readable storage medium described above carries one or more programs, when the one or more programs are executed by the gradient-based hard prompt optimization device, the gradient-based hard prompt optimization device is caused to:
[0131] An initial manually written hard prompt is obtained, and a first embedding vector corresponding to the hard prompt in a vocabulary table is determined;
[0132] The first embedding vector is gradient-optimized as an optimization parameter to obtain a second embedding vector after optimization;
[0133] The second embedding vector is mapped to the vocabulary table to obtain a discrete token sequence corresponding to the second embedding vector;
[0134] An optimized hard prompt is determined based on the discrete token sequence.
[0135] Computer program code for carrying out operations of the present application can be written in one or more programming languages or combinations of languages including object oriented programming languages such as Java, Smalltalk, C++ or conventional procedural programming languages such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).
[0136] The flowcharts and block diagrams in the drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowcharts or block diagrams can represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and / or flowchart illustrations, and combinations thereof, can be implemented by a dedicated hardware-based system that performs the specified functions or operations, or combinations of hardware and software.
[0137] The modules involved in the embodiments of the present application can be implemented in the form of software or in the form of hardware. In some cases, the name of the module does not constitute a limitation on the unit itself.
[0138] The readable storage medium provided by the present application is a computer readable storage medium, which stores computer readable program instructions (i.e., computer programs) for executing the gradient optimization-based hard prompt optimization method described above, and can solve the technical problem that the existing prompt word makes the cost of training a large language model higher. Compared with the prior art, the computer readable storage medium provided by the present application has the same beneficial effects as the gradient optimization-based hard prompt optimization method provided by the above embodiments, and will not be described here.
[0139] The present application also provides a computer program product comprising a computer program, which, when executed by a processor, implements the steps of the gradient optimization-based hard prompt optimization method as described above.
[0140] The computer program product provided by the present application can solve the technical problem that the existing prompt word makes the cost of training a large language model higher. Compared with the prior art, the computer program product provided by the present application has the same beneficial effects as the gradient optimization-based hard prompt optimization method provided by the above embodiments, and will not be described here.
[0141] The above only describes some embodiments of the present application, and does not limit the protection scope of the present application, and any equivalent structural transformation made by using the content of the present application specification and drawings, or direct / indirect application in other related technical fields is included in the protection scope of the present application.
Claims
1. A hard cue optimization method based on gradient optimization, characterized in that, The method includes: Obtain the initial manually written hard cues and determine the first embedding vector of the hard cues in the vocabulary; The first embedding vector is used as an optimization parameter for gradient optimization to obtain the optimized second embedding vector. The second embedding vector is mapped to the vocabulary to obtain a discrete word sequence corresponding to the second embedding vector; The optimized hard cue is determined based on the discrete word sequence; The step of performing gradient optimization using the first embedding vector as optimization parameters to obtain the optimized second embedding vector includes: Random noise is added to the first embedding vector to obtain the initial optimized embedding vector; The initial optimized embedding vector is used as the optimization parameter, and the current optimized gradient of the optimization parameter is obtained based on the loss function of gradient optimization; Based on the optimization gradient and the preset learning rate, the optimization parameters are optimized and learned to obtain an intermediate optimized embedding vector; The intermediate optimized embedding vector that satisfies the gradient iteration termination condition is used as the second embedding vector. The loss function is: ； in, The loss function is based on gradient optimization. For sequence loss function, This is the regularization loss; To optimize parameters; Y is the input to the large language model; Y is the expected output of the large language model. These are the parameters of a large, frozen language model. Indicates hyperparameters, The distance function is used to represent... Distance to each embedding vector in the embedding matrix E.
2. The hard cue optimization method based on gradient optimization as described in claim 1, characterized in that, After the step of optimizing the optimization parameters based on the optimization gradient and the preset learning rate to obtain the intermediate optimized embedding vector, the method further includes: If the gradient iteration termination condition is not met, the intermediate optimized embedding vector is used as the optimization parameter, and the process returns to the step of obtaining the current optimized gradient of the optimization parameter using the gradient-based optimization loss function.
3. The hard cue optimization method based on gradient optimization as described in claim 1, characterized in that, The step of obtaining the current optimization gradient of the optimization parameters using the gradient-based optimization loss function includes: Obtain a preset training dataset, and randomly sample training data from the preset training dataset; The partial derivative of the loss function with respect to the optimization parameters is determined based on the sampled training data; The partial derivative is used as the current optimization gradient of the optimization parameters.
4. The hard cue optimization method based on gradient optimization as described in claim 1, characterized in that, The step of mapping the second embedding vector to the vocabulary to obtain a discrete word sequence corresponding to the second embedding vector includes: Get the preset projection function; Based on the preset projection function, the second embedding vector is mapped to the word index in the vocabulary that is closest to the second embedding vector, thus obtaining a discrete word sequence.
5. The hard cue optimization method based on gradient optimization as described in claim 1, characterized in that, The step of determining the first embedding vector corresponding to the hard cue in the vocabulary includes: The hard prompt is segmented by a word segmenter based on a large language model to obtain hard prompt word units; The first embedding vector corresponding to the hard cue word is determined in the vocabulary of the large language model.
6. A hard cue optimization device based on gradient optimization, characterized in that, The gradient-based hard cue optimization device includes: The word segmentation module is used to obtain the initially manually written hard prompts and determine the first embedding vector of the hard prompts in the vocabulary; An optimization module is used to perform gradient optimization using the first embedding vector as optimization parameters to obtain an optimized second embedding vector. The projection module is used to map the second embedding vector to the vocabulary to obtain a discrete word sequence corresponding to the second embedding vector. The decoding module is used to determine the optimized hard cue based on the discrete word sequence; The optimization module is further configured to add random noise to the first embedding vector to obtain an initial optimized embedding vector; use the initial optimized embedding vector as an optimization parameter and obtain the current optimization gradient of the optimization parameter based on the gradient optimization loss function; perform optimization learning on the optimization parameter based on the optimization gradient and a preset learning rate to obtain an intermediate optimized embedding vector; and use the intermediate optimized embedding vector that satisfies the gradient iteration termination condition as the second embedding vector. The loss function is: ； in, The loss function is based on gradient optimization. For sequence loss function, This is the regularization loss; To optimize parameters; Y is the input to the large language model; Y is the expected output of the large language model. These are the parameters of a large, frozen language model. Indicates hyperparameters, The distance function is used to represent... Distance to each embedding vector in the embedding matrix E.
7. An electronic device, characterized in that, The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the gradient-based hard hint optimization method as described in any one of claims 1 to 5.
8. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the hard hint optimization method based on gradient optimization as described in any one of claims 1 to 5.