Prompt word generation methods, devices, equipment, storage media, and program products
By using graph neural networks and self-attention mechanisms to determine the similarity of prompt words in a large language model, and updating the prompt words to improve their similarity, the conflict problem between prompt words is resolved, and the accuracy of the large language model's responses is improved.
Patent Information
- Application Number
- CN202410804605.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-06-20
- Publication Date
- 2026-11-17
- Estimated Expiration
- 2044-06-20
AI Technical Summary
Current large language models have limited ability to improve answer accuracy with prompts, and there is a lack of optimization methods to address the mutual influence and conflicts between prompts.
By acquiring multiple initial prompts for the target question, a target answer corresponding to each initial prompt is generated. Based on graph neural networks and self-attention mechanisms, the similarity information between prompts is determined, and the prompts are updated to improve the similarity, generating a second prompt with higher similarity.
It improves the accuracy of large language models in answering target questions, reduces the impact of conflicting prompts, and enhances the accuracy of answers.
Smart Images

Figure CN118820415B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of computer technology, and in particular to a method, apparatus, device, storage medium, and program product for generating prompt words. Background Technology
[0002] Currently, in large language models (LLMs), the accuracy of answers to user-input questions can be improved by matching prompt words to the user's input questions.
[0003] However, current large language models have limited ability to improve answer accuracy with prompt words. Therefore, there is a lack of methods to optimize prompt words and further enhance their ability to improve answer accuracy. Summary of the Invention
[0004] This application provides a method, apparatus, device, storage medium, and program product for generating prompt words, which solves the technical problem that prompt words have a low ability to improve the accuracy of answers in related technologies.
[0005] In a first aspect, this application provides a method for generating prompt words, comprising: obtaining a target question to be input into a first large language model and a plurality of initial prompt words corresponding to the target question; inputting the plurality of initial prompt words and the target question into the first large language model to generate a target answer corresponding to each initial prompt word; determining similarity information corresponding to a first prompt word based on the target answer corresponding to each initial prompt word; the similarity information is used to indicate the similarity between the first prompt word and other prompt words, wherein the first prompt word is any one of the plurality of initial prompt words, and the other prompt words are initial prompt words other than the first prompt word among the plurality of initial prompt words; updating the first prompt word based on the similarity information corresponding to the first prompt word to obtain a second prompt word.
[0006] In this application, since the similarity information corresponding to the first prompt word can characterize the degree of similarity between the first prompt word and other initial prompt words among multiple initial prompt words, updating the first prompt word through this similarity information can yield a second prompt word with a higher degree of similarity to the other initial prompt words. Thus, multiple prompt words corresponding to the target question can be obtained to improve the accuracy of the large language model's answer to the target question. Furthermore, the high similarity between these multiple prompt words can address the problem of reduced accuracy of the answer to the target question generated by the large language model due to negative impacts such as conflicts between prompt words.
[0007] Optionally, determining the similarity information corresponding to the first prompt word based on the target answer corresponding to each initial prompt word includes: determining a target similarity based on the target answer corresponding to each initial prompt word, wherein the target similarity is the similarity between the target answer corresponding to the first prompt word and the target answers corresponding to other prompt words; and using the target similarity as the similarity information of the first prompt word.
[0008] In this application, there is a correlation between the target answer corresponding to the first prompt word and the first prompt word. The similarity between the target answer corresponding to the first prompt word and the answers corresponding to other prompt words can reflect the degree of similarity between the first prompt word and other prompt words. Therefore, the target similarity can accurately represent the similarity information of the first prompt word. Thus, the target similarity is used as the similarity information of the first prompt word.
[0009] Optionally, determining the similarity information corresponding to the first prompt word based on the target answer corresponding to each initial prompt word includes: obtaining a first feature vector of the target answer corresponding to each initial prompt word; for each target answer, processing the first feature vector of the target answer based on a graph neural network and a self-attention mechanism to obtain a second feature vector of the target answer, thereby obtaining multiple second feature vectors; and determining the similarity information corresponding to the first prompt word based on the multiple second feature vectors.
[0010] In this application, because the first feature vector of the target answer is processed based on a graph neural network and a self-attention mechanism, the resulting second feature vector contains the correlation between the target answer corresponding to the first prompt word and the target answers corresponding to other prompt words. Therefore, based on multiple second feature vectors, the similarity information corresponding to the first prompt word can be accurately and reliably determined.
[0011] Optionally, the graph neural network includes nodes corresponding to each target answer, and each node corresponds to a first feature vector. The graph neural network and self-attention mechanism are used to process the first feature vector of the target answer to obtain a second feature vector of the target answer, including: determining an attention weight matrix containing the attention weights between each node and its neighboring nodes based on the first feature vector corresponding to each node in the graph neural network; and determining the second feature vector of the target answer based on the attention weight matrix and the first feature vector corresponding to each node.
[0012] In this application, the attention weight matrix of the attention weight between adjacent nodes contains the attention weight information between the first feature vectors of each node. This attention weight information can indicate the relationship between each node. Therefore, based on the attention weight matrix and the first feature vector corresponding to each node, the second feature vector of the target answer can be accurately determined.
[0013] Optionally, determining the similarity information corresponding to the first prompt word based on multiple second feature vectors includes: inputting multiple second feature vectors into a cross-entropy loss determination function to obtain a cross-entropy loss corresponding to the first prompt word, wherein the cross-entropy loss is used to indicate the similarity between the target answer corresponding to the first prompt word and the target answers corresponding to other prompt words; inputting multiple second feature vectors into a contrast loss determination function to obtain a contrast loss corresponding to the first prompt word, wherein the contrast loss is used to indicate the difference between the similarity between the target answer corresponding to the first prompt word and the target answers corresponding to other prompt words and the similarity threshold; and determining the similarity information corresponding to the first prompt word based on the cross-entropy loss and the contrast loss.
[0014] Optionally, updating the first prompt word based on the similarity information corresponding to the first prompt word to obtain the second prompt word includes: updating the first prompt word based on the cross-entropy loss and the contrast loss to obtain the third prompt word; determining the cross-entropy loss and the contrast loss of the third prompt word; and determining the third prompt word as the second prompt word based on the fact that both the cross-entropy loss and the contrast loss of the third prompt word are less than the loss value threshold.
[0015] In this application, both the cross-entropy loss and the contrast loss of the third prompt word are less than the loss threshold, indicating that the similarity between the third prompt word and other prompt words is high, and the difference between the similarity between the third prompt word and other prompt words and the similarity threshold is small. Therefore, identifying the third prompt word as the second prompt word can accurately and reliably obtain prompt words that enhance the ability to answer the target question.
[0016] Secondly, this application provides a prompt word generation device, comprising: an acquisition module, a generation module, and a determination module; the acquisition module is used to acquire a target question to be input into a first large language model and a plurality of initial prompt words corresponding to the target question; the generation module is used to input the plurality of initial prompt words and the target question into the first large language model to generate a target answer corresponding to each initial prompt word; the determination module is used to determine similarity information corresponding to a first prompt word based on the target answer corresponding to each initial prompt word; the similarity information is used to indicate the similarity between the first prompt word and other prompt words, wherein the first prompt word is any one of the plurality of initial prompt words, and the other prompt words are initial prompt words other than the first prompt word among the plurality of initial prompt words; the generation module is further used to update the first prompt word based on the similarity information corresponding to the first prompt word to obtain a second prompt word.
[0017] Optionally, the determining module is specifically used to determine the target similarity based on the target answer corresponding to each initial prompt word, wherein the target similarity is the similarity between the target answer corresponding to the first prompt word and the target answers corresponding to other prompt words; the determining module is also specifically used to use the target similarity as the similarity information of the first prompt word.
[0018] Optionally, the device further includes a processing module; the acquisition module is further configured to acquire a first feature vector of the target answer corresponding to each initial prompt word; the processing module is configured to process the first feature vector of the target answer based on a graph neural network and a self-attention mechanism for each target answer to obtain a second feature vector of the target answer, thereby acquiring multiple second feature vectors; the determination module is further configured to determine the similarity information corresponding to the first prompt word based on the multiple second feature vectors.
[0019] Optionally, the graph neural network includes a node corresponding to each target answer, and each node corresponds to a first feature vector; the determining module is further specifically used to determine an attention weight matrix containing the attention weights between each node and its neighboring nodes based on the first feature vector corresponding to each node in the graph neural network; the determining module is further specifically used to determine the second feature vector of the target answer based on the attention weight matrix and the first feature vector corresponding to each node.
[0020] Optionally, the processing module is further configured to input multiple second feature vectors into a cross-entropy loss determination function to obtain a cross-entropy loss corresponding to the first prompt word, wherein the cross-entropy loss is used to indicate the similarity between the target answer corresponding to the first prompt word and the target answers corresponding to the other prompt words; the processing module is further configured to input multiple second feature vectors into a contrast loss determination function to obtain a contrast loss corresponding to the first prompt word, wherein the contrast loss is used to indicate the difference between the similarity between the target answer corresponding to the first prompt word and the target answers corresponding to the other prompt words and the similarity threshold; the determination module is further configured to determine the similarity information corresponding to the first prompt word based on the cross-entropy loss and the contrast loss.
[0021] Optionally, the generation module is further configured to update the first prompt word based on the cross-entropy loss and the contrast loss to obtain a third prompt word; the determination module is further configured to determine the cross-entropy loss and the contrast loss of the third prompt word; the determination module is further configured to determine the third prompt word as the second prompt word based on the fact that both the cross-entropy loss and the contrast loss of the third prompt word are less than the loss value threshold.
[0022] Thirdly, this application provides an electronic device, including: a processor and a memory configured to store processor-executable instructions; wherein the processor is configured to execute the instructions to implement any of the optional prompt word generation methods in the first aspect described above.
[0023] Fourthly, this application provides a computer-readable storage medium storing instructions that, when executed by an electronic device, enable the electronic device to perform any of the optional prompt generation methods described in the first aspect.
[0024] Fifthly, this application provides a computer program product including computer program instructions, which, when executed by a processor, implement any of the optional prompt word generation methods described in the first aspect above.
[0025] The prompt word generation method, apparatus, device, storage medium, and program product provided in this application obtain a target question to be input into a first large language model and multiple initial prompt words corresponding to the target question; input the multiple initial prompt words and the target question into the first large language model to generate a target answer corresponding to each initial prompt word; determine similarity information corresponding to a first prompt word based on the target answer corresponding to each initial prompt word; the similarity information is used to indicate the similarity between the first prompt word and other prompt words, where the first prompt word is any one of the multiple initial prompt words, and the other prompt words are the initial prompt words other than the first prompt word; update the first prompt word based on the similarity information corresponding to the first prompt word to obtain a second prompt word with a higher similarity to the initial prompt words other than the first prompt word. In this application, since the similarity information corresponding to the first prompt word can characterize the degree of similarity between the first prompt word and the initial prompt words other than the first prompt word, updating the first prompt word through the similarity information can obtain a second prompt word with a higher degree of similarity to the initial prompt words other than the first prompt word. In this way, multiple prompt words can be obtained to improve the accuracy of the large language model's answer to the target question. Moreover, these multiple prompt words have a high degree of similarity with each other, which can at least solve the problem of reduced accuracy of the answer to the target question generated by the large language model due to negative impacts such as conflicts between prompt words. Attached Figure Description
[0026] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below.
[0027] Figure 1 A flowchart illustrating a prompt word generation method provided in an embodiment of this application;
[0028] Figure 2 A flowchart illustrating another prompt word generation method provided in an embodiment of this application;
[0029] Figure 3 A flowchart illustrating another prompt word generation method provided in an embodiment of this application;
[0030] Figure 4 A flowchart illustrating another prompt word generation method provided in an embodiment of this application;
[0031] Figure 5 A flowchart illustrating another prompt word generation method provided in an embodiment of this application;
[0032] Figure 6 A flowchart illustrating another prompt word generation method provided in an embodiment of this application;
[0033] Figure 7 This is a schematic diagram of the structure of a prompt word generation device provided in an embodiment of this application;
[0034] Figure 8 This is a schematic diagram of another prompt word generation device provided in an embodiment of this application. Detailed Implementation
[0035] The prompt word generation method, apparatus, device, storage medium, and program product provided in the embodiments of this application will now be described in detail with reference to the accompanying drawings.
[0036] The terms "first" and "second," etc., in the specification and drawings of this application are used to distinguish different objects, rather than to describe a specific order of objects. For example, "first HTTP request message" and "second HTTP request message," etc., are used to distinguish different HTTP request messages, rather than to describe a specific order of HTTP request messages.
[0037] Furthermore, the terms “comprising” and “having”, and any variations thereof, used in the description of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the steps or units listed, but may optionally include other steps or units not listed, or may optionally include other steps or units inherent to such processes, methods, products, or apparatus.
[0038] It should be noted that in the embodiments of this application, the words "exemplary" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design scheme described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of the words "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.
[0039] The term "and / or" as used in this application includes using either one of two methods or using both methods simultaneously.
[0040] In the description of this application, unless otherwise stated, "a plurality of" means two or more.
[0041] Based on the background description, in related technologies, the accuracy of answers to user-input questions can be improved by matching prompt words to the user's input questions. However, the ability of prompt words in current large language models to improve answer accuracy is relatively low. Therefore, there is a lack of a method to optimize prompt words and further enhance their ability to improve answer accuracy. Furthermore, related technologies do not consider the mutual influence and conflict between prompt words; that is, different prompt words may lead the large language model to generate different or opposite outputs, and these outputs may interfere with each other or be contradictory. Based on this, embodiments of this application provide a prompt word generation method, apparatus, device, storage medium, and program product. Since the similarity information corresponding to the first prompt word can characterize the degree of similarity between the first prompt word and other initial prompt words among a plurality of initial prompt words, updating the first prompt word through this similarity information can obtain a second prompt word with a higher degree of similarity to other initial prompt words among a plurality of initial prompt words. In this way, multiple prompt words can be obtained to improve the accuracy of the large language model's answer to the target question. Moreover, these multiple prompt words have a high degree of similarity with each other, which can at least solve the problem of reduced accuracy of the answer to the target question generated by the large language model due to negative impacts such as conflicts between prompt words.
[0042] For example, the electronic device executing the prompt word generation method provided in this application embodiment can be a mobile phone, tablet computer, desktop computer, laptop computer, handheld computer, notebook computer, ultra-mobile personal computer (UMPC), netbook, as well as cellular phone, personal digital assistant (PDA), augmented reality (AR) / virtual reality (VR) device. This application embodiment does not impose special limitations on the specific form of the electronic device. It can interact with the user through one or more methods such as keyboard, touchpad, touch screen, remote control, voice interaction, or handwriting device.
[0043] The following section provides a detailed explanation of the specific process for generating prompt words.
[0044] like Figure 1 As shown, the prompt word generation method provided in this application embodiment may include S101-S104:
[0045] S101. Obtain the target question of the first language model to be input and the multiple initial prompt words corresponding to the target question.
[0046] In some embodiments, the initial prompts corresponding to the target question may be obtained based on the knowledge domain corresponding to the target question. Optionally, they may be generated based on the knowledge domain corresponding to the target question or filtered based on an existing prompt database.
[0047] S102. Input multiple initial prompts and target questions into the first language model to generate the target answer corresponding to each initial prompt.
[0048] It should be understood that by inputting one of the multiple initial prompt words and the target question into the first language model, the target answer corresponding to that initial prompt word can be obtained, thus obtaining the target answer corresponding to each initial prompt word.
[0049] S103. Based on the target answer corresponding to each initial prompt word, determine the similarity information corresponding to the first prompt word.
[0050] The similarity information is used to indicate the similarity between the first prompt word and other prompt words. The first prompt word is any one of the multiple initial prompt words, and the other prompt words are the initial prompt words other than the first prompt word.
[0051] For example, suppose there are multiple prompt words, including prompt words A, B, and C, with A as the first prompt word and B and C as the other prompt words. In this case, the similarity information of the first prompt word includes the similarity between A and B, and the similarity between A and C.
[0052] In one possible implementation, S103 above, based on the target answer corresponding to each initial prompt word, determines the similarity information corresponding to the first prompt word, which is achieved in the following way: Based on the target answer corresponding to each initial prompt word, determine the target similarity. Use the target similarity as the similarity information of the first prompt word. Wherein, the target similarity is the similarity between the target answer corresponding to the first prompt word and the target answers corresponding to other prompt words.
[0053] It should be understood that there is a correlation between the target answer corresponding to the first prompt word and the first prompt word itself. The similarity between the target answer corresponding to the first prompt word and the answers corresponding to other prompt words can reflect the degree of similarity between the first prompt word and other prompt words. Therefore, the target similarity can accurately represent the similarity information of the first prompt word. Thus, the target similarity is used as the similarity information of the first prompt word.
[0054] In some embodiments, the similarity information corresponding to the first prompt word can also be the difference information of the first prompt word. That is, when the similarity information is a similarity percentage, the difference between 1 and the similarity information can be determined as the difference information.
[0055] S104. Based on the similarity information corresponding to the first prompt word, update the first prompt word to obtain the second prompt word.
[0056] It should be understood that the similarity information corresponding to the first prompt word includes the degree of similarity between the first prompt word and other prompt words, and the second prompt word is the prompt word obtained by updating the first prompt word according to the increasing trend of the degree of similarity between the first prompt word and other prompt words.
[0057] In one possible implementation, if the similarity information corresponding to the first prompt word includes a similarity threshold between the first prompt word and other prompt words, the electronic device updates the first prompt word based on the other prompt words to obtain a second prompt word. It should be understood that the similarity between the second prompt word and other prompt words is greater than the similarity in the similarity information corresponding to the first prompt word.
[0058] In this embodiment, since the similarity information corresponding to the first prompt word can characterize the degree of similarity between the first prompt word and other initial prompt words among the multiple initial prompt words, updating the first prompt word through this similarity information can yield a second prompt word with a higher degree of similarity to the other initial prompt words among the multiple initial prompt words. Thus, multiple prompt words corresponding to the target question can be obtained to improve the accuracy of the large language model's answer to the target question, and these multiple prompt words have a high degree of similarity to each other. This can at least solve the problem of reduced accuracy of the answer to the target question generated by the large language model due to negative impacts such as conflicts between prompt words.
[0059] Combination Figure 1 The illustrated embodiments, such as Figure 2 As shown, S103 above, based on the target answer corresponding to each initial prompt word, determines the similarity information corresponding to the first prompt word, which may include S201-S203:
[0060] S201. Obtain the first feature vector of the target answer corresponding to each initial prompt word.
[0061] In one possible implementation, the electronic device generates the first feature vector of the target answer based on a vectorized model.
[0062] It should be understood that the first feature vector is the target answer in digital form. After converting the target answer into the first feature vector, the target answer can be processed using mathematical tools, which can more efficiently complete the process of determining similarity information.
[0063] S202. For each target answer, the first feature vector of the target answer is processed based on the graph neural network and the self-attention mechanism to obtain the second feature vector of the target answer, so as to obtain multiple second feature vectors.
[0064] In one possible implementation, the graph neural network includes nodes corresponding to each target answer, and each node corresponds to a first feature vector. In step S302 above, the first feature vector of the target answer is processed based on the graph neural network and a self-attention mechanism to obtain the second feature vector of the target answer. This is achieved as follows: Based on the first feature vector corresponding to each node in the graph neural network, an attention weight matrix is determined, containing the attention weights between each node and its neighboring nodes. Based on the attention weight matrix and the first feature vector corresponding to each node, the second feature vector of the target answer is determined.
[0065] In some embodiments, the first feature vector, the attention weight of the first feature vector, and the second feature vector satisfy the following formulas (1), (2), and (3), respectively:
[0066]
[0067] in, It is the first feature vector of the target answer (called a node in the graph neural network, and will be referred to as a node thereafter) i at the l-th layer; σ is the set of neighboring nodes of node i; σ is the activation function, such as the ReLU activation function; W (I) , W q W k W v It is a trainable weight matrix; a ij It is the attention weight of node i to node j; o i It is the second feature vector of node i, which is calculated by the self-attention mechanism based on the feature vectors of node i and its neighboring nodes and the attention weights.
[0068] It should be understood that the attention weight matrix, which contains the attention weight information between the first feature vector of each node and its neighboring nodes, indicates the relationship between each node. Therefore, based on the attention weight matrix and the first feature vector corresponding to each node, the second feature vector of the target answer can be accurately determined.
[0069] S203. Based on multiple second feature vectors, determine the similarity information corresponding to the first prompt word.
[0070] In one possible implementation, the similarity between the second feature vector corresponding to the first prompt word and the second feature vector corresponding to other prompt words can be calculated, and this similarity can be determined as the similarity information corresponding to the first prompt word.
[0071] In this embodiment, since the first feature vector of the target answer is processed based on a graph neural network and a self-attention mechanism, the resulting second feature vector contains the correlation between the target answer corresponding to the first prompt word and the target answers corresponding to other prompt words. Therefore, based on multiple second feature vectors, the similarity information corresponding to the first prompt word can be accurately and reliably determined.
[0072] In some embodiments, combined with Figure 2 The illustrated embodiments, such as Figure 3 As shown, S203 above determines the similarity information corresponding to the first prompt word based on multiple second feature vectors, including S301-S303:
[0073] S301. Input multiple second feature vectors into the cross-entropy loss determination function to obtain the cross-entropy loss corresponding to the first prompt word.
[0074] The cross-entropy loss is used to indicate the similarity between the target answer corresponding to the first prompt word and the target answers corresponding to other prompt words.
[0075] S302. Input multiple second feature vectors into the contrast loss determination function to obtain the contrast loss corresponding to the first prompt word.
[0076] The contrast loss is used to indicate the similarity between the target answer corresponding to the first prompt word and the target answers corresponding to other prompt words, and the difference between the loss loss and the similarity threshold.
[0077] It should be noted that the execution order of S301 and S302 described above is not limited in the embodiments of this application. For example, S301 can be executed first and then S302, or S301 can be executed first and then S302, or S301 and S302 can be executed simultaneously.
[0078] S303. Based on cross-entropy loss and contrast loss, determine the similarity information corresponding to the first prompt word.
[0079] In one possible implementation, the cross-entropy loss and contrast loss can be directly determined as the similarity information corresponding to the first prompt word.
[0080] In one possible implementation, the similarity information corresponding to the first prompt word can include a judgment result indicating whether there is a conflict or not between the first prompt word and other prompt words. In this case, the classification ability of cross-entropy loss can be used to determine whether there is a conflict or not between the first prompt word and other prompt words, and the judgment result can be added to the similarity information.
[0081] In one possible implementation, combining Figure 3 ,like Figure 4 As shown, in S104 above, the first prompt word is updated based on the similarity information corresponding to the first prompt word to obtain the second prompt word, including S401-S403:
[0082] S401. Based on cross-entropy loss and contrast loss, update the first prompt word to obtain the third prompt word.
[0083] In one possible implementation, the first prompt word can be updated and the third prompt word determined based on the decreasing trend of cross-entropy loss and contrastive loss, using gradient descent.
[0084] S402. Determine the cross-entropy loss and contrast loss of the third cue word.
[0085] In one possible implementation, the process of determining the cross-entropy loss and contrast loss of the third cue word is analogous to the process of determining the cross-entropy loss and contrast loss of the first cue word.
[0086] S403. Since both the cross-entropy loss and the contrast loss based on the third prompt word are less than the loss threshold, the third prompt word is determined as the second prompt word.
[0087] It should be understood that both the cross-entropy loss and the contrast loss of the third prompt word are less than the loss threshold, indicating a high similarity between the third prompt word and other prompt words, and a small difference between the similarity between the third prompt word and other prompt words and the similarity threshold. Therefore, identifying the third prompt word as the second prompt word can accurately and reliably yield prompt words that enhance the ability to answer the target question.
[0088] In some embodiments, the similarity information of nodes can be a similarity matrix, and the dissimilarity information of nodes can be a dissimilarity matrix. The similarity matrix and dissimilarity matrix of nodes in graph neural networks and attention mechanisms satisfy the following formulas (4) and (5):
[0089] S ij =cosine(o i ,o j (4)
[0090] D ij =1-S ij (5)
[0091] Among them, S ij Let D be the similarity matrix between node i and node j, where cosine is the cosine similarity function. ij It is the difference matrix between node i and node j.
[0092] In some embodiments, the above method further includes data preprocessing. Data preprocessing includes cleaning, word segmentation, and annotation of raw data such as the target question.
[0093] In some embodiments, the target problem corresponds to a target knowledge domain, such as Figure 5 As shown, the prompt word generation method provided in this application embodiment further includes S501-S503:
[0094] S501. Obtain the first answer to the target question generated by the large language model based on the initial prompt words.
[0095] S502. Based on the quality score determination function determined by meta-learning technology, generate the quality score corresponding to the first answer.
[0096] The quality score corresponding to the first answer is used to indicate the quality of the first answer in the target knowledge domain.
[0097] In some embodiments, the quality score is also referred to as a reward signal.
[0098] S503. Based on the quality score corresponding to the first answer, update the initial prompt words to obtain the target prompt words.
[0099] In some embodiments, the large language model can generate a second answer to the target question based on the target prompt words, and the quality score of the second answer is higher than that of the first answer.
[0100] In some embodiments, a quality score generation function is generated based on a model-agnostic meta-learning algorithm in meta-learning.
[0101] It should be understood that the quality score corresponding to the first answer is used to indicate the quality of the first answer in the target knowledge domain. Therefore, updating the initial prompt words based on the quality score can yield target prompt words that make the answer to the target question more accurate in the target knowledge domain. This can improve the transferability and adaptability of prompt words in the large language model within the target knowledge domain.
[0102] In some embodiments, the initial prompt word is updated based on the quality score corresponding to the first answer, which is achieved by updating the initial prompt word based on the policy gradient algorithm in reinforcement learning and the quality score corresponding to the first answer to obtain the target prompt word.
[0103] Specifically, the initial prompt word is considered an action, the target knowledge domain is considered an environment, the first answer is considered a state, and the quality score is considered a reward. Thus, in the policy gradient algorithm, the initial prompt word can be updated based on the quality score. If the updated quality score is higher than a threshold, the updated initial prompt word corresponding to the updated quality score is determined as the target prompt word. This allows for accurate and reliable optimization of the prompt word.
[0104] In some embodiments, the action in the policy gradient algorithm can be obtained by parameterizing the prompt words based on the policy network; and the state in the policy gradient algorithm can be obtained by estimating the state value based on the value function network.
[0105] In some embodiments, the update of the initial prompt words based on the quality score can be viewed as an update of the policy network.
[0106] For example, the policy network can be updated using the REINFORCE algorithm, and the value function network can be updated using the advantage actor-critic algorithm. In this way, the initial prompt word can be updated reliably, and the target prompt word can be obtained.
[0107] For example, the policy network and the value function network satisfy the following equations (6) and (7), respectively:
[0108] π θ (a|s)=softmax(W a f θ (s)+b a (6)
[0109] V φ (s)=W v f φ (s)+b v (7)
[0110] Where θ and φ are the parameters of the policy network and value function network, respectively, a is the action (i.e., the prompt word), s is the state (first answer), and W... a W v These are trainable weights, b v b a For trainable bias, f θ and f φ It is a multi-layer perceptron with two shared parameters.
[0111] In some embodiments, such as Figure 6 As shown, the prompt word generation method provided in this application embodiment further includes S601-S603:
[0112] S601. Input the target question and the corresponding initial prompt words into the first and second language models to obtain the answers to the target question generated by the first and second language models.
[0113] The size of the largest language model is smaller than that of the second largest language model.
[0114] S602. Input the answer to the target question generated by the first language model, the true answer to the target question, and the answer to the target question generated by the second language model into the target loss function to obtain the quality score of the answer to the target question generated by the first language model.
[0115] In some embodiments, the answers other than the true answer are all probability distributions of category labels, and the true answer is the true category label corresponding to the target question.
[0116] In some embodiments, the answer to the target question generated by the second language model is called a soft label, and the true answer to the target question is called a hard label.
[0117] For example, the hard labels and soft labels mentioned above satisfy the following formulas (8) and (9), respectively:
[0118] y h =one-hot(y) (8)
[0119] y s =softmax(f ω (x)) (9)
[0120] Among them, y h It's a hard label, y s It's a soft tag, y is a true category tag, f ω ω is the output function of the second largest language model, x is the target problem (or input data).
[0121] In some embodiments, the quality score of the answer to the target question generated by the first largest language model, which generates the target loss function, includes the mean squared error loss and cross-entropy loss of the answer to the target question generated by the first largest language model.
[0122] S603. Based on the quality score of the answer to the target question generated by the first major language model, update the initial prompt words and obtain the target prompt words.
[0123] In this embodiment, the quality score of the answer to the target question generated by the first large language model is obtained by inputting the answer to the target question generated by the first large language model, the true answer to the target question, and the answer to the target question generated by the second large language model into the target loss function. Therefore, based on the quality score of the answer to the target question generated by the first large language model, the target prompt words obtained are more capable of generating more accurate answers to the target question in the first large language model. Thus, prompt words for the first large language model can be obtained based on a model larger than the first large language model, improving the compatibility and stability between the prompt words and the first large language model itself.
[0124] Furthermore, obtaining prompt words from a larger language model than the largest language model can be called knowledge distillation. This process can also transfer knowledge information from the larger model to the prompt words of the largest language model. This improvement allows the smaller, more energy-efficient language model to perform better. Therefore, in scenarios where large and small language models operate in parallel, assigning tasks originally handled by the large language model to the smaller model can reduce the computational overhead caused by the large language model's operation.
[0125] In some embodiments, the prompt word generation method provided in this application may further include: training multiple large language models based on multi-task learning.
[0126] This application embodiment can divide electronic devices and the like into functional modules according to the above method examples. For example, each function can be divided into its own functional module, or two or more functions can be integrated into one processing module. The integrated module can be implemented in hardware or as a software functional module. It should be noted that the module division in this application embodiment is illustrative and only represents one logical functional division; other division methods may be used in actual implementation.
[0127] When dividing each function into modules according to its corresponding function. Figure 7 A possible structural schematic diagram of the prompt word generation device involved in the above embodiments is shown, such as... Figure 7 As shown, the prompt word generation device 70 may include: an acquisition module 701, a generation module 702, and a determination module 703.
[0128] The acquisition module 701 is used to acquire the target question of the first large language model to be input and the multiple initial prompt words corresponding to the target question.
[0129] The generation module 702 is used to input multiple initial prompt words and target questions into the first language model and generate the target answer corresponding to each initial prompt word.
[0130] The determination module 703 is used to determine the similarity information corresponding to the first prompt word based on the target answer corresponding to each initial prompt word. The similarity information is used to indicate the similarity between the first prompt word and other prompt words. The first prompt word is any one of the multiple initial prompt words, and the other prompt words are the initial prompt words other than the first prompt word.
[0131] The generation module 702 is also used to update the first prompt word based on the similarity information corresponding to the first prompt word to obtain the second prompt word.
[0132] Optionally, the determining module 703 is specifically used to determine the target similarity based on the target answer corresponding to each initial prompt word, wherein the target similarity is the similarity between the target answer corresponding to the first prompt word and the target answers corresponding to other prompt words.
[0133] The determination module 703 is also specifically used to use the target similarity as the similarity information of the first prompt word.
[0134] Optionally, the device also includes a processing module 704.
[0135] The acquisition module 701 is also used to acquire the first feature vector of the target answer corresponding to each initial prompt word.
[0136] The processing module 704 is used to process the first feature vector of the target answer based on the graph neural network and the self-attention mechanism for each target answer to obtain the second feature vector of the target answer, so as to obtain multiple second feature vectors.
[0137] The determination module 703 is also specifically used to determine the similarity information corresponding to the first prompt word based on multiple second feature vectors.
[0138] Optionally, the graph neural network includes a node corresponding to each target answer, and each node corresponds to a first feature vector.
[0139] The determination module 703 is also specifically used to determine an attention weight matrix containing the attention weights between each node and its neighboring nodes, based on the first feature vector corresponding to each node in the graph neural network.
[0140] The determination module 703 is also specifically used to determine the second feature vector of the target answer based on the attention weight matrix and the first feature vector corresponding to each node.
[0141] Optionally, the processing module 704 is further configured to input multiple second feature vectors into the cross-entropy loss determination function to obtain the cross-entropy loss corresponding to the first prompt word. The cross-entropy loss is used to indicate the similarity between the target answer corresponding to the first prompt word and the target answers corresponding to other prompt words.
[0142] The processing module 704 is also used to input multiple second feature vectors into the contrast loss determination function to obtain the contrast loss corresponding to the first prompt word. The contrast loss is used to indicate the similarity between the target answer corresponding to the first prompt word and the target answer corresponding to other prompt words, and the difference between the first prompt word and the similarity threshold.
[0143] The determination module 703 is also specifically used to determine the similarity information corresponding to the first prompt word based on cross-entropy loss and contrast loss.
[0144] Optionally, the generation module 702 is further configured to update the first prompt word based on the cross-entropy loss and the contrast loss to obtain the third prompt word.
[0145] The determination module 703 is also used to determine the cross-entropy loss and the contrast loss of the third prompt word.
[0146] The determination module 703 is further configured to determine the third prompt word as the second prompt word based on the fact that both the cross-entropy loss and the contrast loss of the third prompt word are less than the loss value threshold.
[0147] When using integrated units, Figure 8 A schematic diagram of a possible structure of the prompt word generation device involved in the above embodiments is shown. For example... Figure 8 As shown, the prompt word generation device 80 may include a processing module 801 and a communication module 802. The processing module 801 can be used to control and manage the operation of the prompt word generation device 80. The communication module 802 can be used to support communication between the prompt word generation device 80 and other entities. Optionally, as shown... Figure 7 As shown, the prompt word generation device 80 may further include a storage module 803 for storing the program code and data of the prompt word generation device 80.
[0148] The processing module 801 can be a processor or a controller. The communication module 802 can be a transceiver, transceiver circuit, or communication interface, etc. The storage module 803 can be a memory.
[0149] In this configuration, when the processing module 801 is a processor, the communication module 802 is a transceiver, and the storage module 803 is a memory, the processor, transceiver, and memory can be connected via a bus. The bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc.
[0150] It should be understood that in the various embodiments of this application, the order of the above-mentioned processes does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.
[0151] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.
[0152] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.
[0153] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0154] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented using software programs, implementation can be, in whole or in part, in the form of a computer program product. This computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device containing one or more servers, data centers, etc., that can be integrated with the medium. The available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state disks (SSDs)).
[0155] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.
Claims
1. A method for generating prompt words, characterized in that, The method includes: Obtain the target question of the first large language model to be input and multiple initial prompt words corresponding to the target question; Input the multiple initial prompt words and the target question into the first large language model to generate the target answer corresponding to each initial prompt word; Based on the target answer corresponding to each initial prompt word, similarity information corresponding to the first prompt word is determined; the similarity information is used to indicate the similarity between the first prompt word and other prompt words, wherein the first prompt word is any one of the plurality of initial prompt words, and the other prompt words are initial prompt words other than the first prompt word among the plurality of initial prompt words; Based on the similarity information corresponding to the first prompt word, the first prompt word is updated to obtain the second prompt word, including: If the similarity information corresponding to the first prompt word includes a case where the similarity between the first prompt word and other prompt words is lower than a similarity threshold, the first prompt word is updated based on the other prompt words to obtain the second prompt word; Based on cross-entropy loss and contrast loss, the first prompt word is updated to obtain the third prompt word; the cross-entropy loss and contrast loss of the third prompt word are determined; based on the fact that both the cross-entropy loss and contrast loss of the third prompt word are less than the loss value threshold, the third prompt word is determined as the second prompt word.
2. The method according to claim 1, characterized in that, The step of determining the similarity information corresponding to the first prompt word based on the target answer corresponding to each initial prompt word includes: Based on the target answer corresponding to each initial prompt word, a target similarity is determined, wherein the target similarity is the similarity between the target answer corresponding to the first prompt word and the target answers corresponding to other prompt words; The target similarity is used as the similarity information of the first prompt word.
3. The method according to claim 1 or 2, characterized in that, The step of determining the similarity information corresponding to the first prompt word based on the target answer corresponding to each initial prompt word includes: Obtain the first feature vector of the target answer corresponding to each of the initial prompt words; For each target answer, the first feature vector of the target answer is processed based on a graph neural network and a self-attention mechanism to obtain the second feature vector of the target answer, thereby obtaining multiple second feature vectors; Based on multiple second feature vectors, the similarity information corresponding to the first prompt word is determined.
4. The method according to claim 3, characterized in that, The graph neural network includes nodes corresponding to each target answer, and each node corresponds to a first feature vector. The process of processing the first feature vector of the target answer based on the graph neural network and a self-attention mechanism to obtain the second feature vector of the target answer includes: Based on the first feature vector corresponding to each node in the graph neural network, an attention weight matrix is determined, which includes the attention weights between each node and its neighboring nodes. Based on the attention weight matrix and the first feature vector corresponding to each node, the second feature vector of the target answer is determined.
5. The method according to claim 3, characterized in that, The step of determining the similarity information corresponding to the first prompt word based on multiple second feature vectors includes: Multiple second feature vectors are input into the cross-entropy loss determination function to obtain the cross-entropy loss corresponding to the first prompt word. The cross-entropy loss is used to indicate the similarity between the target answer corresponding to the first prompt word and the target answers corresponding to the other prompt words. Multiple second feature vectors are input into a contrast loss determination function to obtain the contrast loss corresponding to the first prompt word. The contrast loss is used to indicate the similarity between the target answer corresponding to the first prompt word and the target answers corresponding to the other prompt words, and the difference between the loss and the similarity threshold. Based on the cross-entropy loss and the contrast loss, the similarity information corresponding to the first prompt word is determined.
6. A prompt word generation device, characterized in that, The device includes an acquisition module, a generation module, and a determination module; The acquisition module is used to acquire the target question of the first large language model to be input and multiple initial prompt words corresponding to the target question; The generation module is used to input the plurality of initial prompt words and the target question into the first large language model to generate the target answer corresponding to each initial prompt word; The determining module is used to determine the similarity information corresponding to the first prompt word based on the target answer corresponding to each of the initial prompt words; The similarity information is used to indicate the similarity between the first prompt word and other prompt words, wherein the first prompt word is any one of the plurality of initial prompt words, and the other prompt words are initial prompt words other than the first prompt word among the plurality of initial prompt words; The generation module is further configured to update the first prompt word based on the similarity information corresponding to the first prompt word, and obtain the second prompt word; The generation module is specifically used to: update the first prompt word based on the other prompt words when the similarity information corresponding to the first prompt word includes the fact that the similarity between the first prompt word and other prompt words is lower than the similarity threshold, thereby obtaining a second prompt word; Based on cross-entropy loss and contrast loss, the first prompt word is updated to obtain the third prompt word; Determine the cross-entropy loss and contrast loss of the third prompt word; based on the fact that both the cross-entropy loss and contrast loss of the third prompt word are less than the loss threshold, determine the third prompt word as the second prompt word.
7. An electronic device, characterized in that, The electronic device includes: processor; A memory configured to store processor-executable instructions; The processor is configured to execute the instructions to implement the prompt word generation method as described in any one of claims 1-5.
8. A computer-readable storage medium storing instructions thereon, characterized in that, When the instructions in the computer-readable storage medium are executed by an electronic device, the electronic device is able to perform the prompt word generation method as described in any one of claims 1-5.
9. A computer program product, characterized in that, The computer program product includes computing technology program instructions, which, when executed by a processor, implement the prompt word generation method as described in any one of claims 1-5.
Citation Information
Patent Citations
Question and answer interaction method and device, electronic equipment and storage medium
CN117332067A
Interactive questioning method, interactive questioning system, interactive questioning program, and recording medium with the same program recorded thereon
JP2007322836A
