Prompt template determination method and device, electronic equipment and storage medium

Abstract

The invention provides a prompt template determination method and device, electronic equipment and a storage medium. The method comprises the steps that multiple pieces of question and answer data are acquired; modifying a pre-constructed prompt template for a predetermined number of times to obtain a predetermined number of modified prompt templates; based on the multiple pieces of question and answer data, screening out multiple target prompt templates from the preset number of modification prompt templates; grouping the plurality of target prompt templates to obtain a plurality of groups of target prompt templates, and in response to determining that a random value corresponding to the group of target prompt templates is greater than a preset random value, performing cross exchange on prompt statements in the group of target prompt templates; based on the multiple pieces of question and answer data, a final prompt template is screened out from the multiple target prompt templates which are subjected to cross exchange, the technical problem that in the prior art, prompt of the prompt templates is not accurate is solved, and the accuracy of outputting answers is improved.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a method, device, electronic device and storage medium for determining a prompt template. Background Art

[0002] In the field of knowledge question answering, traditional question answering systems often rely on predefined question-answer pairs, which often fail to provide accurate and satisfactory answers when faced with diverse user questioning methods. With the development of artificial intelligence technology, especially the advancement of natural language processing technology, knowledge question answering models have begun to adopt machine learning algorithms to improve their understanding of user questions by training large amounts of text data. However, existing knowledge question answering models still have certain limitations when dealing with complex questions, long-tail questions, or questions that require multi-step reasoning. Solving these problems requires the model to better understand and process the user's question intentions, as well as to more accurately locate and retrieve relevant knowledge. If the model does not understand the question accurately, the accuracy of the output answer will be reduced. Summary of the invention

[0003] In view of this, the purpose of the present application is to propose a method, device, electronic device and storage medium for determining a prompt template to overcome all or part of the deficiencies in the prior art.

[0004] Based on the above-mentioned purpose, the present application provides a method for determining a prompt template, including: obtaining multiple question and answer data; modifying a pre-constructed prompt template a predetermined number of times to obtain the predetermined number of modified prompt templates; based on the multiple question and answer data, screening out multiple target prompt templates from the predetermined number of modified prompt templates; grouping the multiple target prompt templates to obtain multiple group target prompt templates, and in response to determining that the random value corresponding to the group target prompt template is greater than the predetermined random value, cross-interchanging the prompt sentences in the group target prompt template; based on the multiple question and answer data, screening out a final prompt template from the multiple target prompt templates that have been cross-interchanged.

[0005] Optionally, based on the multiple question and answer data, multiple target prompt templates are screened out from the predetermined number of modification prompt templates, including: for each question and answer data, the question and answer data is filled into the predetermined number of modification prompt templates to obtain multiple first filling data, and the multiple first filling data are input into the pre-trained question and answer model in sequence, and the first accuracy scores corresponding to the first filling data are output through the question and answer model; based on all the first accuracy scores, a first total score corresponding to the first filling data corresponding to the modification prompt template is calculated; all first total scores corresponding to the predetermined number of modification prompt templates are sorted in order from large to small, and the modification prompt templates corresponding to the first total scores sorted into the top N are determined as the multiple target prompt templates, wherein N is a positive integer.

[0006] Optionally, based on the multiple question and answer data, a final prompt template is screened out from the multiple target prompt templates that have been cross-interchanged, including: for each question and answer data, filling the question and answer data into the multiple target prompt templates to obtain multiple second filling data, inputting the multiple second filling data into a pre-trained question and answer model in sequence, and outputting second accuracy scores corresponding to the second filling data through the question and answer model; based on all second accuracy scores, calculating a second total score corresponding to the second filling data corresponding to the target prompt template; sorting all second total scores corresponding to the multiple target prompt templates in order from large to small, comparing the second total score sorted as one with the first total score sorted as one to obtain the maximum total score, and determining the target prompt template corresponding to the maximum total score as the final prompt template.

[0007] Optionally, the pre-constructed prompt template is modified a predetermined number of times to obtain the predetermined number of modified prompt templates, including: for each modification, randomly selecting a predetermined operation from multiple predetermined operations, using the predetermined operation to modify the prompt statement in the prompt template to obtain the modified prompt template.

[0008] Optionally, the predetermined operation is a deletion operation; and the using the predetermined operation to modify the prompt statement to obtain the modified prompt template includes: deleting the prompt statement in the prompt template to obtain the modified prompt template.

[0009] Optionally, the predetermined operation is a replacement operation using a pre-trained question-answering model; using the predetermined operation to modify the prompt statement to obtain the modified prompt template includes: inputting the prompt template into the question-answering model, replacing the prompt statement in the prompt template by using the question-answering model to obtain the modified prompt template.

[0010] Optionally, the predetermined operation is to perform a translation operation using a pre-trained question-answering model; and the method of modifying the prompt statement using the predetermined operation to obtain the modified prompt template includes: inputting the prompt template into the question-answering model, performing language judgment on the prompt statement in the prompt template through the question-answering model to obtain the language type of the prompt statement, and translating the prompt statement into a predetermined language type corresponding to the language type to obtain the modified prompt template.

[0011] Optionally, the predetermined operation is to perform an expansion operation using a pre-trained question-answering model; and using the predetermined operation to modify the prompt statement to obtain the modified prompt template includes: inputting the prompt template into the question-answering model, replacing or translating the prompt statement in the prompt template through the question-answering model, and merging the replaced or translated prompt statement with the prompt template to obtain the modified prompt template.

[0012] Based on the same inventive concept, the present application also provides a device for determining a prompt template, including: an acquisition module, configured to acquire multiple question and answer data; a modification module, configured to modify a pre-built prompt template a predetermined number of times to obtain the predetermined number of modified prompt templates; a first screening module, configured to screen out multiple target prompt templates from the predetermined number of modified prompt templates based on the multiple question and answer data; a grouping module, configured to group the multiple target prompt templates to obtain multiple group target prompt templates, and in response to determining that the random value corresponding to the group target prompt template is greater than the predetermined random value, cross-interchange the prompt statements in the group target prompt template; a second screening module, configured to screen out a final prompt template from the multiple target prompt templates that have been cross-interchanged based on the multiple question and answer data.

[0013] Based on the same inventive concept, the present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable by the processor, wherein the processor implements the method as described above when executing the computer program.

[0014] As can be seen from the above, the method, device, electronic device and storage medium for determining the prompt template provided by the present application include obtaining multiple question and answer data. The pre-constructed prompt template is modified a predetermined number of times to obtain the predetermined number of modified prompt templates, and multiple types of prompt templates are provided, so that the prompt template becomes relatively rich. Based on the multiple question and answer data, multiple target prompt templates are screened out from the predetermined number of modified prompt templates, further narrowing the screening range for the final prompt template and eliminating the randomness of the screening. The multiple target prompt templates are grouped to obtain multiple group target prompt templates. In response to determining that the random value corresponding to the group target prompt template is greater than the predetermined random value, the prompt statements in the group target prompt template are cross-interchanged to achieve the purpose of further enriching the types of prompt templates. Based on the multiple question and answer data, the final prompt template is screened out from the multiple target prompt templates that have been cross-interchanged to achieve the purpose of accurately determining the prompt template. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions in the present application or related technologies, the drawings required for use in the embodiments or related technical descriptions are briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0016] Figure 1 A flowchart of a method for determining a prompt template according to an embodiment of the present application;

[0017] Figure 2 A flowchart of a method for determining a prompt template according to another embodiment of the present application;

[0018] Figure 3 A schematic diagram of the structure of a device for determining a prompt template according to an embodiment of the present application;

[0019] Figure 4 This is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0020] In order to make the objectives, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in combination with specific embodiments and with reference to the accompanying drawings.

[0021] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present application should be the usual meanings understood by people with ordinary skills in the field to which the present application belongs. The "first", "second" and similar words used in the embodiments of the present application do not represent any order, quantity or importance, but are only used to distinguish different components. "Including" or "comprising" and similar words mean that the elements or objects appearing in front of the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0022] As described in the background technology section, in the field of knowledge question answering, traditional question answering systems often rely on predefined question-answer pairs, which often find it difficult to provide accurate and satisfactory answers when faced with diverse user questioning methods. With the development of artificial intelligence technology, especially the advancement of natural language processing technology, knowledge question answering models have begun to adopt machine learning algorithms to improve their ability to understand user questions by training large amounts of text data. However, existing knowledge question answering models still have certain limitations when dealing with complex problems, long-tail problems, or problems that require multi-step reasoning. Solving these problems requires the model to be able to better understand and process the user's question intentions, as well as to more accurately locate and retrieve relevant knowledge. If the model does not understand the problem accurately, the accuracy of the output answer will be reduced.

[0023] In the field of natural language processing, pre-trained language models have become a major innovation that has greatly promoted the development of the discipline. Although these models show great potential in learning scenarios with small amounts of data, they still face the problem of high computational costs when fine-tuning. Current prompt tuning techniques based on discrete Fourier transforms often require additional word embeddings or injection of hidden states, which may be difficult to operate in practical applications. In addition, these techniques usually also require the call of additional language models to adjust existing prompts, thereby adding more dependencies. For example, using training examples directly as pre-inputs to prompts, that is, the in-context learning method, is usually not ideal.

[0024] In view of this, the present application embodiment proposes a method for determining a prompt template, referring to Figure 1 , including the following steps:

[0025] Step 101, obtaining multiple question and answer data.

[0026] In this step, the basis for effective interaction with the question-answering model is clear and specific goals, which can significantly reduce the uncertainty and deviation of the answers output by the question-answering model and ensure that the output meets expectations. For example, when asking the question-answering model to generate descriptive text, it should be clearly stated whether it is describing a scene, a person, or an event, and provide relevant background information such as time, place, and subject. Therefore, concise prompt words are easy to understand and help the question-answering model to parse intent faster and more accurately. Avoid lengthy and complex expressions and use direct and easy-to-understand language. Conciseness is not only reflected in the text, but also in the amount of information provided. Excessive redundant information may interfere with the understanding of the question-answering model. Therefore, the prompt template used to prompt the answer is an indispensable part of the question-answering model.

[0027] In order to determine the precise prompt template, we first need to obtain multiple question-and-answer data. The characteristic of question-and-answer data is that questions and answers appear in pairs. In order to make the answers output by the question-and-answer model more accurate, it is also necessary to prompt the answers. Therefore, the question-and-answer data also includes prompt content, which is used to prompt the answers to the questions. Based on this feature, it is necessary to collect multiple question-and-answer data as training data sets and test data sets. Relying on crawler technology, network protocol data packets are continuously collected from the network, the data of the data packets are cleaned, and questions, prompt content and corresponding answers are extracted. Some question-and-answer pairs are modified, keeping the questions unchanged and changing the wrong answers. Exemplarily, the content of the data set contains 278 question-and-answer pairs, of which 77 are correctly answered and 201 are incorrectly answered. Randomly take 20 rows of data from the initial data set as the training set and 258 rows of data as the test set. Finally, the initial data set in json format is created. The format of question and answer data is divided into three columns. The first column is the question, the second column is the answer, and the third column is the label (the correct answer is marked with Y, and the incorrect answer is marked with N). In addition to the question, answer and label, the rest of the content is descriptive content. For example, "question":"What is 1+1?","anwser":"Equal to 1","label":"Y"; "question":"What is 1+1?","anwser":"Equal to 2","label":"N".

[0028] Step 102: modify the pre-built prompt template a predetermined number of times to obtain the predetermined number of modified prompt templates.

[0029] In this step, a prompt template to be optimized is constructed, and the prompt template includes three parts, namely, a description statement, a question block to be filled, and an answer block to be filled. The prompt template is divided into rows and the question block to be filled and the answer block to be filled are retained to obtain a text block in a sequence form, and the text block is saved in a list to obtain a prompt word template list. Since there is only one prompt template constructed in advance, the number is relatively small. In order to make the number of prompt templates relatively large and then select the optimal prompt template, the prompt template is modified a predetermined number of times to obtain a predetermined number of modified prompt templates. It should be noted that the predetermined number can be set using hyperparameters, and the set predetermined number meets the actual needs. Exemplarily, the predetermined number can be set to 64. By modifying the prompt template a predetermined number of times, a variety of types of prompt templates are provided, so that the prompt template becomes relatively rich.

[0030] Step 103: Based on the multiple question and answer data, multiple target prompt templates are selected from the predetermined number of modification prompt templates.

[0031] In this step, the number of modification prompt templates is relatively large. If the final prompt template is directly selected from the modification prompt template, there is a chance, so that the final prompt template determined is not accurate. Therefore, it is necessary to screen out multiple target prompt templates from a predetermined number of modification prompt templates based on multiple question and answer data, wherein the number of target prompt templates is less than the number of modification prompt templates, and illustratively, the number of target prompt templates is 8. By screening out multiple target prompt templates, the screening range for the final prompt template is further narrowed, and the chance of screening is eliminated.

[0032] Step 104, grouping the multiple target prompt templates to obtain multiple group target prompt templates, and in response to determining that the random value corresponding to the group target prompt template is greater than a predetermined random value, cross-exchanging the prompt sentences in the group target prompt template.

[0033] In this step, if the final prompt template is directly selected in the target prompt template, the type of the target prompt template is not rich enough, and there may be other optimal prompt templates. In order to eliminate the possibility of the above problem, multiple target prompt templates are grouped to obtain multiple groups of target prompt templates. Exemplarily, multiple target prompt templates are grouped in pairs, and 8 target prompt templates are divided into 4 groups. Assuming that it is divided into M groups, it is necessary to generate M random values. If a certain random value is greater than a predetermined random value (such as 0.75), then the prompt sentences in all the target prompt templates in this group are cross-interchanged to obtain a new group of prompt templates. Exemplarily, a numerical value between 0 and 1, i.e. a random value, is randomly generated. If the random value is greater than the predetermined random value 0.75, then no cross-interchange is performed. If the random value is less than or equal to the predetermined random value 0.75, then cross-interchange is performed. The specific process of cross-exchange is: assuming that there are n target prompt templates in a group, then randomly select two of the n target prompt templates, replace the prompt sentence in one of the target prompt templates with the prompt sentence in another target prompt template, and obtain a new target prompt template. In this way, n(n-1) / 2 new target prompt templates can be obtained. By cross-exchanging the prompt sentences in the target prompt templates of the group, the purpose of further enriching the types of prompt templates can be achieved.

[0034] Determine the length of the two target prompt templates. The target prompt template with the longer length is called A, and the target prompt template with the shorter length is called B. Set the initial index to 0, traverse the text blocks in A and B in turn, and compare whether the text content of the text block in A and the text block in B is the same under the same index. If they are the same, continue traversing. If they are different, add the different text block index numbers to the different text block lists. The traversal process ends when the text block in A is traversed. Randomly take an index number from the different text block lists as the exchange index number. If the exchange index number is less than or equal to the length of the text block in A and less than or equal to the length of the text block in B, the content of A and the content of B at the exchange index number are exchanged. If the exchange index number is greater than or equal to the length of the text block in B and less than or equal to the length of the text block in A, the content of the text block in A at the exchange index number is added to the end of the text block in B, and the content of the text block in A at the exchange index number becomes an empty string. For A, randomly generate a value between 0 and 1. If this value is greater than the random rate of 0.5, the random generation algorithm operation is not performed. If this value is less than or equal to 0.5, the random generation algorithm is used. For B, a value between 0 and 1 is randomly generated. If this value is greater than the random rate of 0.5, the random generation algorithm is not used. If this value is less than or equal to 0.5, the random generation algorithm is used.

[0035] Step 105: based on the plurality of question-answer data, a final prompt template is selected from the plurality of target prompt templates that have been cross-interchangeably selected.

[0036] In this step, if the target prompt template can still output the correct answer after cross-exchange, it reflects the accuracy and stability of the target prompt template after cross-exchange. Therefore, the final prompt template is selected from the multiple target prompt templates after cross-exchange to achieve the purpose of accurately determining the prompt template.

[0037] After the final prompt template is determined, it is imported into the question-answering model. By optimizing the use of prompt words, the accuracy of the question-answering model in identifying the user's question intention is improved, and the efficiency of knowledge retrieval and answer generation is improved. Through algorithm optimization, the most appropriate prompt words can be recommended to users based on the characteristics of the question-answering model and the knowledge base. These prompt words can effectively guide the question-answering model to perform deeper semantic understanding and knowledge retrieval. In addition, this method does not need to rely on additional application interfaces or manually designed prompts, which reduces the difficulty of implementation.

[0038] Through the above scheme, multiple question and answer data are obtained. The pre-constructed prompt template is modified a predetermined number of times to obtain the predetermined number of modified prompt templates, and multiple types of prompt templates are provided, so that the prompt template becomes relatively rich. Based on the multiple question and answer data, multiple target prompt templates are screened out from the predetermined number of modified prompt templates, further narrowing the screening range for the final prompt template and eliminating the randomness of the screening. The multiple target prompt templates are grouped to obtain multiple group target prompt templates. In response to determining that the random value corresponding to the group target prompt template is greater than the predetermined random value, the prompt statements in the group target prompt template are cross-interchanged, so as to achieve the purpose of further enriching the types of prompt templates. Based on the multiple question and answer data, the final prompt template is screened out from the multiple target prompt templates that have been cross-interchanged, so as to achieve the purpose of accurately determining the prompt template.

[0039] In some embodiments, based on the multiple question and answer data, multiple target prompt templates are screened out from the predetermined number of modification prompt templates, including: for each question and answer data, the question and answer data is filled into the predetermined number of modification prompt templates to obtain multiple first filling data, and the multiple first filling data are input into the pre-trained question and answer model in sequence, and the first accuracy scores corresponding to the first filling data are output through the question and answer model; based on all the first accuracy scores, a first total score corresponding to the first filling data corresponding to the modification prompt template is calculated; all first total scores corresponding to the predetermined number of modification prompt templates are sorted in order from large to small, and the modification prompt templates corresponding to the first total scores sorted into the top N are determined as the multiple target prompt templates, where N is a positive integer.

[0040] In this embodiment, all question-answering data and modification prompt templates are first combined, and the question-answering data are filled into the modification prompt template to obtain multiple first filling data, and then these first filling data are input into the question-answering model. Since the question-answering model has learned a large amount of question-answering data, it can output the probability of the answer to the question corresponding to the filling data, that is, the accuracy rate. This probability can be used as the accuracy score, that is, the higher the accuracy score, the closer the filling data is to the correct answer. Next, it is necessary to count the first accuracy score of each filling data filled into the modification prompt template, add them together as the first total score of the modification prompt template, and finally sort all the first total scores of the modification prompt templates from large to small, and select the modification prompt templates corresponding to the top N first total scores as the target prompt template. It should be noted here that the specific value of "N" can be adjusted according to the actual situation, for example, N is 8. The modification prompt template is assigned a score, so that the accuracy of the modification prompt template prompt is visualized, so as to achieve the purpose of accurately screening the target prompt template.

[0041] Exemplarily, 258 rows of question and answer data in the test set are filled into 64 modification prompt templates to generate 16512 complete random knowledge question and answer prompt words. Then, the complete random knowledge question and answer prompt words with a number of 16512 are sent to the question and answer model, and the question and answer model returns the answer label of each complete random knowledge question and answer prompt word. According to the label result of each complete random knowledge question and answer prompt word, the complete random knowledge question and answer prompt words generated by the 64 random knowledge question and answer prompt word templates are scored (the label result returned by the question and answer model is consistent with the label result in the test set, and 0 points are inconsistent). Sort the 64 modification prompt templates from high to low according to the scores, select the modification prompt templates with the top 8 scores, store them in a list, and obtain a better list of modification prompt templates.

[0042] In some embodiments, based on the multiple question and answer data, a final prompt template is selected from the multiple target prompt templates that have been cross-interchanged, including: for each question and answer data, the question and answer data is filled into the multiple target prompt templates to obtain multiple second filling data, and the multiple second filling data are input into the pre-trained question and answer model in sequence, and the second accuracy scores corresponding to the second filling data are output through the question and answer model; based on all the second accuracy scores, a second total score corresponding to the second filling data corresponding to the target prompt template is calculated; all the second total scores corresponding to the multiple target prompt templates are sorted from large to small, and the second total score ranked as one is compared with the first total score ranked as one to obtain the maximum total score, and the target prompt template corresponding to the maximum total score is determined as the final prompt template.

[0043] In this embodiment, firstly, all the question and answer data and the target prompt template are combined, and the question and answer data are filled into the target prompt template to obtain a plurality of second filling data, and then these second filling data are input into the question and answer model. Since the question and answer model has learned a large amount of question and answer data, it can output the probability of the answer to the question corresponding to the filling data, that is, the accuracy rate. This probability can be used as the accuracy score, that is, the higher the accuracy score, the closer the filling data is to the correct answer. Next, it is necessary to count the second accuracy score of each filling data filled into the target prompt template, add them together as the second total score of the target prompt template, and finally sort the second total scores of all the target prompt templates from large to small. Obtain the second total score ranked 1 and the first total score ranked 1, compare the first total score and the second total score, obtain the maximum total score, and determine the target prompt template corresponding to the maximum total score as the final prompt template. For example, when the maximum total score is the first total score, the target prompt template corresponding to the first total score is determined as the final prompt template; when the maximum total score is the second total score, the cross-interchange target prompt template corresponding to the second total score is determined as the final prompt template. The final prompt template has the highest total score, indicating that the accuracy of the answer output by the question-answering model using the final prompt template is the highest, thus achieving the purpose of accurately screening out the final prompt template.

[0044] Exemplarily, 258 rows of question and answer data in the test set are filled into 8 target prompt templates to generate 2064 complete random knowledge question and answer prompt words. Then, the 2064 complete new prompt words are sent to the question and answer model, and the question and answer model returns the answer label of each complete new prompt word. According to the label result of each complete new prompt word, the complete new prompt words generated by the 8 target prompt templates are scored (the label result returned by the question and answer model is consistent with the label result in the test set, and 0 points are inconsistent). Sort the 8 target prompt templates from high to low according to the scores, and select the target prompt template that ranks first. Get the second total score ranked 1 and the first total score ranked 1, compare the first total score and the second total score, get the maximum total score, and determine the target prompt template corresponding to the maximum total score as the final prompt template.

[0045] In some embodiments, the pre-built prompt template is modified a predetermined number of times to obtain the predetermined number of modified prompt templates, including: for each modification, randomly selecting a predetermined operation from multiple predetermined operations, and using the predetermined operation to modify the prompt statement in the prompt template to obtain the modified prompt template.

[0046] In this embodiment, the random algorithm randomly selects a predetermined operation from a plurality of predetermined operations according to probability to change the prompt statement in the prompt template. The predetermined operation can be set according to demand. For example, the plurality of predetermined operations include: deletion, replacement, exchange, translation, extension, etc. The selection probability of the plurality of predetermined operations can be set according to demand. For example, the selection probabilities of the plurality of predetermined operations are: 0.1, 0.3, 0.3, 0.2, 0.1 respectively. By modifying the prompt statement in the prompt template, the types of prompt templates are enriched.

[0047] It should be noted that, in order to ensure the existence of the prompt template, before modifying the prompt template, it can be first determined whether there is a prompt template in the prompt word template list, that is, calculate and determine whether the length of the prompt word template list is 0. If the length is 0, it is directly returned. If the length is not 0, it means that the prompt word template exists and the prompt template can be modified. The present embodiment can also be replaced by randomly modifying the prompt template, randomly selecting a text block in the prompt word template list, and determining whether the text block is a question to be filled block or an answer to be filled block. If it is a question to be filled block or an answer to be filled block, the predetermined operation is not performed and it is directly returned. If it is not a question to be filled block or an answer to be filled block, the predetermined operation is performed. It should be supplemented that, in the case where the number of randomly modified prompt templates is less than the number of preset prompt templates to be modified, the prompt template needs to be randomly modified again.

[0048] In some embodiments, the predetermined operation is a deletion operation; and the using the predetermined operation to modify the prompt statement to obtain the modified prompt template includes: deleting the prompt statement in the prompt template to obtain the modified prompt template.

[0049] In this embodiment, the prompt template includes a prompt sentence. In order to enrich the types of the prompt template, the prompt template may not include the prompt sentence. Therefore, the prompt sentence in the prompt template is deleted to obtain a modified prompt template, so that the modified prompt template is different from the prompt template, and the types of the prompt template are increased.

[0050] In some embodiments, the predetermined operation is a replacement operation using a pre-trained question-answering model; using the predetermined operation to modify the prompt statement to obtain the modified prompt template includes: inputting the prompt template into the question-answering model, replacing the prompt statement in the prompt template by using the question-answering model to obtain the modified prompt template.

[0051] In this embodiment, the prompt template includes a prompt sentence. In order to enrich the types of the prompt template, the prompt sentence in the prompt template can be replaced. The prompt template is input into the question-answer model, and the prompt sentence is replaced by the question-answer model to obtain a modified prompt template, thereby further increasing the types of the prompt template.

[0052] In some embodiments, the predetermined operation is to perform a translation operation using a pre-trained question-answering model; the method of modifying the prompt statement using the predetermined operation to obtain the modified prompt template includes: inputting the prompt template into the question-answering model, performing language judgment on the prompt statement in the prompt template through the question-answering model to obtain the language type of the prompt statement, translating the prompt statement into a predetermined language type corresponding to the language type, and obtaining the modified prompt template.

[0053] In the present embodiment, the prompt template includes a prompt statement. In order to enrich the types of the prompt template, the prompt statement in the prompt template can be translated. The prompt template is input into the question-answer model, the language type of the prompt statement is determined by the question-answer model, and the prompt statement is translated into a predetermined language type corresponding to the language type to obtain a modified prompt template. The predetermined language type corresponding to the language type can be set as required, and the language type is associated with the predetermined language type and stored in a pre-constructed data table. Exemplarily, when the language type of the prompt statement is Chinese, the prompt statement can be translated into English; when the language type of the prompt statement is English, the prompt statement can be translated into English. By translating the prompt statement, the types of prompt templates are enriched.

[0054] In some embodiments, the predetermined operation is to perform an expansion operation using a pre-trained question-answering model; using the predetermined operation to modify the prompt statement to obtain the modified prompt template includes: inputting the prompt template into the question-answering model, replacing or translating the prompt statement in the prompt template through the question-answering model, merging the replaced or translated prompt statement with the prompt template to obtain the modified prompt template.

[0055] In this embodiment, the prompt template includes a prompt statement. In order to enrich the types of the prompt template, the prompt statement in the prompt template can be expanded. The prompt template is input into the question-answering model, and the prompt statement in the prompt template is subjected to the aforementioned replacement operation or translation operation through the question-answering model. As for the specific operation to be performed, it can be randomly selected. Then, the replaced or translated prompt statement is merged with the prompt template through the question-answering model to obtain a modified prompt template. By translating the prompt statement, the types of the prompt template are expanded.

[0056] In another embodiment provided in the present application, Figure 2 As shown, first, it is necessary to search for the training question and answer data set and the test question and answer data set and create a prompt template. Set the hyperparameters, exemplarily, set the number of prompt templates obtained by the random generation algorithm. The set hyperparameters can be to set the number of randomly generated prompt word templates to 64, the random rate and the crossover rate to 0.5 and 0.75 respectively, and set the number of processes to 5. A predetermined number of prompt templates are obtained by the random generation algorithm. Through the question and answer model, multiple target prompt templates are selected from the predetermined number of prompt templates, and the total score corresponding to each target prompt template is determined. Multiple target prompt templates are grouped and cross-interchanged. Through the question and answer model, the target prompt template that has been cross-interchanged is selected, and the total score corresponding to each cross-interchanged target prompt template is determined. Determine the target prompt template with the largest total score in the two questions and answers respectively, compare the total scores of the two questions and answers, and determine the target prompt template corresponding to the maximum total score as the final prompt word template.

[0057] It should be noted that the method of the embodiment of the present application can be performed by a single device, such as a computer or server. The method of this embodiment can also be applied to a distributed scenario and completed by multiple devices cooperating with each other. In the case of such a distributed scenario, one of the multiple devices can only perform one or more steps in the method of the embodiment of the present application, and the multiple devices will interact with each other to complete the described method.

[0058] It should be noted that the above describes some embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the above embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0059] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, the present application also provides a device for determining a prompt template.

[0060] refer to Figure 3 , the device for determining the prompt template includes:

[0061] The acquisition module 10 is configured to acquire a plurality of question and answer data.

[0062] The modification module 20 is configured to modify the pre-built prompt template a predetermined number of times to obtain the predetermined number of modified prompt templates.

[0063] The first screening module 30 is configured to screen out a plurality of target prompt templates from the predetermined number of modification prompt templates based on the plurality of question and answer data.

[0064] The grouping module 40 is configured to group the multiple target prompt templates to obtain multiple group target prompt templates, and in response to determining that the random value corresponding to the group target prompt template is greater than a predetermined random value, cross-interchange the prompt sentences in the group target prompt template.

[0065] The second screening module 50 is configured to screen out a final prompt template from the multiple target prompt templates that have been cross-interchange based on the multiple question-answer data.

[0066] Through the above-mentioned device, multiple question and answer data are obtained. The pre-constructed prompt template is modified a predetermined number of times to obtain the predetermined number of modified prompt templates, and multiple types of prompt templates are provided, so that the prompt template becomes relatively rich. Based on the multiple question and answer data, multiple target prompt templates are screened out from the predetermined number of modified prompt templates, further narrowing the screening range for the final prompt template and eliminating the randomness of the screening. The multiple target prompt templates are grouped to obtain multiple group target prompt templates. In response to determining that the random value corresponding to the group target prompt template is greater than the predetermined random value, the prompt statements in the group target prompt template are cross-interchanged, so as to achieve the purpose of further enriching the types of prompt templates. Based on the multiple question and answer data, the final prompt template is screened out from the multiple target prompt templates that have been cross-interchanged, so as to achieve the purpose of accurately determining the prompt template.

[0067] In some embodiments, the first screening module 30 is further configured to fill the question and answer data into the predetermined number of modification prompt templates for each question and answer data to obtain multiple first filling data, input the multiple first filling data into the pre-trained question and answer model in sequence, and output the first accuracy score corresponding to the first filling data through the question and answer model; calculate the first total score corresponding to the first filling data corresponding to the modification prompt template based on all the first accuracy scores; sort all the first total scores corresponding to the predetermined number of modification prompt templates in order from large to small, and determine the modification prompt templates corresponding to the first total scores ranked in the top N as the multiple target prompt templates, where N is a positive integer.

[0068] In some embodiments, the second screening module 50 is further configured to fill the question and answer data into the multiple target prompt templates for each question and answer data to obtain multiple second filling data, input the multiple second filling data into the pre-trained question and answer model in sequence, and output the second accuracy score corresponding to the second filling data through the question and answer model; calculate the second total score corresponding to the second filling data corresponding to the target prompt template based on all the second accuracy scores; compare the second total score ranked as one with the first total score ranked as one to obtain the maximum total score, and determine the target prompt template corresponding to the maximum total score as the final prompt template.

[0069] In some embodiments, the modification module 20 is further configured to randomly select a predetermined operation from a plurality of predetermined operations for each modification, and use the predetermined operation to modify the prompt statement in the prompt template to obtain the modified prompt template.

[0070] In some embodiments, the modification module 20 is further configured such that the predetermined operation is a deletion operation; a deletion operation is performed on the prompt statement in the prompt template to obtain the modified prompt template.

[0071] In some embodiments, the modification module 20 is further configured such that the predetermined operation is to perform a replacement operation using a pre-trained question-answering model; the prompt template is input into the question-answering model, and the prompt statement in the prompt template is replaced by the question-answering model to obtain the modified prompt template.

[0072] In some embodiments, the modification module 20 is further configured such that the predetermined operation is to perform a translation operation using a pre-trained question-answering model; the prompt template is input into the question-answering model, a language judgment is performed on the prompt statement in the prompt template through the question-answering model to obtain a language type of the prompt statement, the prompt statement is translated into a predetermined language type corresponding to the language type, and the modified prompt template is obtained.

[0073] In some embodiments, the modification module 20 is further configured such that the predetermined operation is to perform an extended operation using a pre-trained question-answering model; the prompt template is input into the question-answering model, the prompt statement in the prompt template is replaced or translated through the question-answering model, and the replaced or translated prompt statement is merged with the prompt template to obtain the modified prompt template.

[0074] For the convenience of description, the above device is described in terms of functions divided into various modules. Of course, when implementing the present application, the functions of each module can be implemented in the same or multiple software and / or hardware.

[0075] The device of the above embodiment is used to implement the corresponding method for determining the prompt template in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here.

[0076] Based on the same inventive concept, corresponding to any of the above-mentioned embodiments and methods, the present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method for determining the prompt template as described in any of the above embodiments is implemented.

[0077] Figure 4A more specific schematic diagram of the hardware structure of an electronic device provided in this embodiment is shown, and the device may include: a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040 are connected to each other through the bus 1050 in the device.

[0078] The processor 1010 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0079] The memory 1020 may be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 may store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program codes are stored in the memory 1020 and are called and executed by the processor 1010.

[0080] The input / output interface 1030 is used to connect the input / output module to realize information input and output. The input / output module can be configured in the device as a component (not shown in the figure), or it can be externally connected to the device to provide corresponding functions. The input device may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device may include a display, a speaker, a vibrator, an indicator light, etc.

[0081] The communication interface 1040 is used to connect a communication module (not shown) to realize communication interaction between the device and other devices. The communication module can realize communication through a wired mode (such as USB, network cable, etc.) or a wireless mode (such as mobile network, WIFI, Bluetooth, etc.).

[0082] The bus 1050 includes a path that transmits information between the various components of the device (eg, the processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040).

[0083] It should be noted that, although the above device only shows the processor 1010, the memory 1020, the input / output interface 1030, the communication interface 1040 and the bus 1050, in the specific implementation process, the device may also include other components necessary for normal operation. In addition, it can be understood by those skilled in the art that the above device may also only include the components necessary for implementing the embodiments of the present specification, and does not necessarily include all the components shown in the figure.

[0084] The electronic device of the above embodiment is used to implement the corresponding method for determining the prompt template in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here.

[0085] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, the present application also provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the method for determining the prompt template as described in any of the above embodiments.

[0086] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0087] The computer instructions stored in the storage medium of the above embodiment are used to enable the computer to execute the method for determining the prompt template as described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0088] Based on the same concept, corresponding to any of the above-mentioned embodiment methods, the present application also provides a computer program product, including computer program instructions. When the computer program instructions are run on a computer, the computer executes the method for determining the prompt template as described in any of the above embodiments, which has the beneficial effects of the corresponding method embodiments and will not be repeated here.

[0089] It should be noted that the embodiments of the present application can be further described in the following manner:

[0090] It is understandable that before using the technical solutions of each embodiment of the present disclosure, the type, scope of use, usage scenarios, etc. of the personal information involved will be informed to the user in an appropriate manner, and the user's authorization will be obtained.

[0091] For example, in response to receiving an active request from a user, a prompt message is sent to the user to clearly remind the user that the operation requested to be performed will require obtaining and using the user's personal information. Thus, the user can independently choose whether to provide personal information to software or hardware such as an electronic device, application, server, or storage medium that performs the operation of the technical solution of the present disclosure according to the prompt message.

[0092] As an optional but non-limiting implementation, in response to receiving the user's active request, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. In addition, the pop-up window may also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0093] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation of the present disclosure. Other methods that meet relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0094] A person skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present application is limited to these examples. In line with the concept of the present application, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the present application as described above, which are not provided in detail for the sake of simplicity.

[0095] In addition, to simplify the description and discussion, and in order not to make the embodiments of the present application difficult to understand, the known power supply / ground connection with the integrated circuit (IC) chip and other components may or may not be shown in the provided drawings. In addition, the device can be shown in the form of a block diagram to avoid making the embodiments of the present application difficult to understand, and this also takes into account the fact that the details of the implementation of these block diagram devices are highly dependent on the platform to be implemented in the embodiments of the present application (that is, these details should be fully within the scope of understanding of those skilled in the art). In the case of elaborating specific details (e.g., circuits) to describe exemplary embodiments of the present application, it is obvious to those skilled in the art that the embodiments of the present application can be implemented without these specific details or when these specific details are changed. Therefore, these descriptions should be considered to be illustrative rather than restrictive.

[0096] Although the present application has been described in conjunction with specific embodiments of the present application, many replacements, modifications and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may use the embodiments discussed.

[0097] The embodiments of the present application are intended to cover all such substitutions, modifications and variations that fall within the broad scope of the present application. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present application should be included in the protection scope of the present application.

Claims

1. A method for determining a prompt template, characterized in that: include: Get multiple question and answer data; Modifying the pre-built prompt template a predetermined number of times to obtain the predetermined number of modified prompt templates; Based on the plurality of question and answer data, screening out a plurality of target prompt templates from the predetermined number of modification prompt templates; Grouping the plurality of target prompt templates to obtain a plurality of group target prompt templates, and in response to determining that the random value corresponding to the group target prompt template is greater than a predetermined random value, cross-exchanging prompt sentences in the group target prompt template; Based on the multiple question-answer data, a final prompt template is selected from the multiple target prompt templates that have been cross-interchangeable.

2. The method according to claim 1, characterized in that The step of selecting a plurality of target prompt templates from the predetermined number of modification prompt templates based on the plurality of question and answer data includes: For each question and answer data, fill the question and answer data into the predetermined number of modification prompt templates to obtain a plurality of first filling data, input the plurality of first filling data into the pre-trained question and answer model in sequence, and output a first accuracy score corresponding to the first filling data through the question and answer model; Based on all the first accuracy scores, calculating a first total score corresponding to the first filling data corresponding to the modification prompt template; All first total scores corresponding to the predetermined number of modification prompt templates are sorted in descending order, and the modification prompt templates corresponding to the first N first total scores are determined as the multiple target prompt templates, where N is a positive integer.

3. The method according to claim 2, characterized in that The step of selecting a final prompt template from the multiple target prompt templates that have been cross-exchanged based on the multiple question-answer data includes: For each question and answer data, fill the question and answer data into the multiple target prompt templates to obtain multiple second filling data, input the multiple second filling data into the pre-trained question and answer model in sequence, and output a second accuracy score corresponding to the second filling data through the question and answer model; Based on all the second accuracy scores, calculating a second total score corresponding to the second filling data corresponding to the target prompt template; All second total scores corresponding to the multiple target prompt templates are sorted in descending order, the second total score ranked as one is compared with the first total score ranked as one to obtain the maximum total score, and the target prompt template corresponding to the maximum total score is determined as the final prompt template.

4. The method according to claim 1, characterized in that: The step of modifying the pre-built prompt template a predetermined number of times to obtain the predetermined number of modified prompt templates includes: For each modification, a predetermined operation is randomly selected from a plurality of predetermined operations, and the prompt statement in the prompt template is modified by using the predetermined operation to obtain the modified prompt template.

5. The method according to claim 4, characterized in that The predetermined operation is a deletion operation; The step of modifying the prompt statement by using the predetermined operation to obtain the modified prompt template includes: The prompt statement in the prompt template is deleted to obtain the modified prompt template.

6. The method according to claim 4, characterized in that The predetermined operation is to perform a replacement operation using a pre-trained question-answering model; The step of modifying the prompt statement by using the predetermined operation to obtain the modified prompt template includes: The prompt template is input into the question-answer model, and the prompt sentence in the prompt template is replaced by the question-answer model to obtain the modified prompt template.

7. The method according to claim 4, characterized in that The predetermined operation is to perform a translation operation using a pre-trained question-answering model; The step of modifying the prompt statement by using the predetermined operation to obtain the modified prompt template includes: The prompt template is input into the question-answer model, the prompt sentence in the prompt template is language-judged through the question-answer model to obtain the language type of the prompt sentence, the prompt sentence is translated into a predetermined language type corresponding to the language type to obtain the modified prompt template.

8. The method according to claim 4, characterized in that The predetermined operation is to perform an expansion operation using a pre-trained question-answering model; The step of modifying the prompt statement by using the predetermined operation to obtain the modified prompt template includes: The prompt template is input into the question-answering model, the prompt sentence in the prompt template is replaced or translated through the question-answering model, and the replaced or translated prompt sentence is merged with the prompt template to obtain the modified prompt template.

9. A device for determining a prompt template, characterized in that: include: An acquisition module, configured to acquire a plurality of question and answer data; A modification module is configured to modify the pre-built prompt template a predetermined number of times to obtain the predetermined number of modified prompt templates; A first screening module is configured to screen out a plurality of target prompt templates from the predetermined number of modification prompt templates based on the plurality of question and answer data; a grouping module configured to group the plurality of target prompt templates to obtain a plurality of group target prompt templates, and in response to determining that the random value corresponding to the group target prompt template is greater than a predetermined random value, cross-exchange prompt sentences in the group target prompt template; The second screening module is configured to screen out a final prompt template from the multiple target prompt templates that have been cross-interchange based on the multiple question-answer data.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the program, the method according to any one of claims 1 to 7 is implemented.

Images