Prompt template processing method and device applied to language model and electronic equipment

Abstract

The present disclosure provides a processing method and device of a prompt template applied to a language model and an electronic device, the computer technical field, especially the deep learning technical field, the artificial intelligence technical field and the large language model technical field, which can be applied to intelligent search, intelligent question answering and other application scenarios. The specific implementation scheme is: in response to an update instruction representing an update of a prompt template, constructing prompt information according to the update instruction and the prompt template; processing the prompt information by using a language model to obtain template defect information; updating the prompt template according to the template defect information to obtain at least one candidate prompt template; and determining a target prompt template according to the at least one candidate prompt template and a template evaluation result of the at least one candidate prompt template.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to the fields of deep learning, artificial intelligence, and large language model technology, and can be applied to application scenarios such as intelligent search and intelligent question answering. Background Technology

[0002] In human-computer dialogue scenarios such as intelligent question answering, pre-trained large language models (LLMs) can be used to process user input information. This allows the large language models to understand user questions based on their powerful semantic understanding capabilities and generate the responses that users need, thus meeting their complex and diverse requirements. Summary of the Invention

[0003] This disclosure provides a method, apparatus, electronic device, and storage medium for processing prompt templates applied to a language model.

[0004] According to one aspect of this disclosure, a method for processing prompt templates applied to a language model is provided, comprising: responding to an update instruction that represents updating the prompt template; constructing prompt information based on the update instruction and the prompt template; processing the prompt information using a language model to obtain template defect information; updating the prompt template based on the template defect information to obtain at least one candidate prompt template; and determining a target prompt template based on the at least one candidate prompt template and a template evaluation result for the at least one candidate prompt template.

[0005] According to another aspect of this disclosure, a processing apparatus for a prompt template applied to a language model is provided, comprising: a prompt information construction module, configured to construct prompt information based on the update instruction and the prompt template in response to an update instruction representing an update of the prompt template; a template defect information acquisition module, configured to process the prompt information using a language model to obtain template defect information; a candidate prompt template acquisition module, configured to update the prompt template based on the template defect information to obtain at least one candidate prompt template; and a target prompt template determination module, configured to determine a target prompt template based on at least one candidate prompt template and a template evaluation result for at least one candidate prompt template.

[0006] According to another aspect of this disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform a method provided according to an embodiment of this disclosure.

[0007] According to another aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are used to cause the computer to perform a method provided according to an embodiment of this disclosure.

[0008] According to another aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the method provided according to embodiments of this disclosure.

[0009] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0010] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0011] Figure 1 The illustration schematically shows an exemplary system architecture of a processing method and apparatus for prompt templates that can be applied to a language model according to embodiments of the present disclosure;

[0012] Figure 2 A flowchart illustrating a method for processing a prompt template applied to a language model according to an embodiment of the present disclosure is shown schematically.

[0013] Figure 3 This diagram illustrates an application scenario for evaluating candidate suggestion templates according to embodiments of the present disclosure.

[0014] Figure 4 A flowchart illustrating a method for processing prompt templates applied to a language model according to another embodiment of the present disclosure is shown schematically.

[0015] Figure 5 A schematic diagram of a prompt template update platform according to an embodiment of the present disclosure is shown.

[0016] Figure 6 A block diagram schematically illustrates a processing apparatus for a prompt template applied to a language model according to an embodiment of the present disclosure; and

[0017] Figure 7 A block diagram of an electronic device suitable for implementing a processing method for prompt templates applied to a language model, according to an embodiment of the present disclosure, is shown schematically. Detailed Implementation

[0018] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0019] In the technical solution disclosed herein, the acquisition, storage, and application of user personal information comply with the provisions of relevant laws and regulations, necessary confidentiality measures have been taken, and there is no violation of public order and good morals.

[0020] With the rapid development of artificial intelligence technology, in applications such as intelligent question answering, intelligent search, and intelligent customer service, Large Language Models (LLMs) can be used to process user input and generate responses that match the user's input, thus meeting user needs promptly and accurately. Prompt engineering is a technique for developing or optimizing prompts (or prompt templates). The inventors discovered that users can determine prompts containing task information based on optimized prompt input. Under the control of these prompts, the LLM can more accurately understand the user's task requirements, thereby outputting more accurate responses. Therefore, optimizing and evaluating prompts can improve the response capabilities of LLMs.

[0021] Embodiments of this disclosure provide a method, apparatus, electronic device, and storage medium for processing prompt templates applied to a language model. The method for processing prompt templates applied to a language model includes: responding to an update instruction characterizing an updated prompt template; constructing prompt information based on the update instruction and the prompt template; processing the prompt information using a language model to obtain template defect information; updating the prompt template based on the template defect information to obtain at least one candidate prompt template; and determining a target prompt template based on the at least one candidate prompt template and a template evaluation result for the at least one candidate prompt template.

[0022] According to embodiments of this disclosure, for prompt templates applied to a large language model, the prompt information constructed based on update instructions and prompt templates is processed by the language model. Based on the semantic understanding and analysis capabilities of the language model, the template defects of the prompt template can be characterized more intuitively and accurately. Thus, the prompt template is updated based on the template defect information, allowing the obtained candidate prompt templates to more accurately overcome the template defects in the prompt information. By determining the target prompt template based on the template evaluation results of the candidate prompt templates, the obtained target prompt template can be updated more accurately to a prompt template suitable for controlling the large language model to provide precise answers, improving the optimization efficiency and accuracy of the prompt template. Furthermore, the user can generate input information to control the large language model to complete the answer task based on the updated target prompt template, helping the large language model improve text prediction accuracy and the matching degree between the answer content and the user input information.

[0023] It should be noted that the language model involved in this embodiment may include a large language model built based on deep learning algorithms, such as a large language model built based on the Transformer algorithm. The prompt template may include information in any format, such as characters, words, or fields, used to help the language model understand the semantic information of the user's needs and to control the language model in generating the response content. The user can generate prompt information by filling in the requirement content into the prompt template and inputting the prompt information into the language model, which can then output response content information that matches the user's needs.

[0024] Figure 1 The illustration schematically shows an exemplary system architecture of a processing method and apparatus for prompt templates that can be applied to a language model according to embodiments of the present disclosure.

[0025] It is important to note that Figure 1 The examples shown are merely examples of system architectures applicable to embodiments of this disclosure, intended to help those skilled in the art understand the technical content of this disclosure. They do not imply that embodiments of this disclosure cannot be used in other devices, systems, environments, or scenarios. For instance, in another embodiment, an exemplary system architecture applicable to the processing method and apparatus for prompt templates applied to a language model may include a terminal device. However, the terminal device can implement the processing method and apparatus for prompt templates applied to a language model provided in the embodiments of this disclosure without interacting with a server.

[0026] like Figure 1As shown, the system architecture 100 according to this embodiment may include terminal devices 101, 102, and 103, a network 104, and a server 105. The network 104 serves as a medium for providing a communication link between the terminal devices 101, 102, and 103 and the server 105. The network 104 may include various connection types, such as wired and / or wireless communication links, etc.

[0027] Users can use terminal devices 101, 102, and 103 to interact with server 105 via network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, and 103, such as knowledge reading applications, web browser applications, search applications, instant messaging tools, email clients, and / or social platform software, etc. (for example only).

[0028] Terminal devices 101, 102, and 103 can be various electronic devices with displays and web browsing capabilities, including but not limited to smartphones, tablets, laptops, and desktop computers.

[0029] Server 105 can be a server that provides various services, such as a backend management server that supports the content browsed by users using terminal devices 101, 102, and 103 (for example only). The backend management server can analyze and process data such as received user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal devices.

[0030] It should be noted that the method for processing prompt templates applied to a language model provided in this embodiment can generally be executed by server 105. Correspondingly, the device for processing prompt templates applied to a language model provided in this embodiment can generally be located in server 105. The method for processing prompt templates applied to a language model provided in this embodiment can also be executed by a server or server cluster that is different from server 105 and capable of communicating with terminal devices 101, 102, 103 and / or server 105. Correspondingly, the device for processing prompt templates applied to a language model provided in this embodiment can also be located in a server or server cluster that is different from server 105 and capable of communicating with terminal devices 101, 102, 103 and / or server 105.

[0031] Alternatively, the method for processing prompt templates applied to a language model provided in this embodiment of the present disclosure can also be executed by terminal devices 101, 102, or 103. Accordingly, the device for processing prompt templates applied to a language model provided in this embodiment of the present disclosure can also be located in terminal devices 101, 102, or 103.

[0032] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0033] Figure 2 A flowchart illustrating a method for processing a prompt template applied to a language model according to an embodiment of the present disclosure is shown.

[0034] like Figure 2 As shown, the method for processing prompt templates applied to the language model includes operations S210 to S240.

[0035] In operation S210, in response to the update instruction representing the update prompt template, a prompt message is constructed based on the update instruction and the prompt template.

[0036] When operating S220, the prompt information is processed using a language model to obtain template defect information.

[0037] In operation S230, the prompt template is updated based on the template defect information to obtain at least one candidate prompt template.

[0038] In operation S240, a target prompt template is determined based on at least one candidate prompt template and the template evaluation result for the at least one candidate prompt template.

[0039] According to embodiments of this disclosure, the update instruction may include a task instruction to update the prompt information. The update instruction may be determined based on the user's update operation, or it may be determined based on preset update conditions, such as generating the update instruction based on the evaluation result of the prompt information. Embodiments of this disclosure do not limit the method of obtaining the update instruction.

[0040] According to embodiments of this disclosure, the update instruction may include an update strategy for updating the prompt information. Constructing the prompt information based on the update instruction and the prompt template may include obtaining the prompt information based on the update strategy and the prompt template, thereby obtaining semantic attributes that control the language model to understand the update strategy and the prompt template, and generating template defect information representing the defect attributes of the prompt template. The template defect information can be represented in any form, such as based on text description, or based on other forms such as defect type and defect identifier. Embodiments of this disclosure do not limit the form in which the template defect information is represented.

[0041] According to embodiments of this disclosure, the prompt template is updated based on the template defect information. The template defect information and prompt template can be processed based on a pre-trained neural network model, such as a language model, to generate one or more candidate prompt templates.

[0042] According to embodiments of this disclosure, the language model may include a pre-trained large language model. The language model can understand task requirements based on prompt markers (such as characters, words, and fields) used to indicate tasks in the prompt information, so as to control the language model to perform rewriting operations such as deletion, replacement, and addition on the prompt template based on the prompt markers, and obtain updated candidate prompt templates.

[0043] According to embodiments of this disclosure, the template evaluation result may include evaluation information in any form, such as evaluation score, evaluation level, and evaluation description, for the candidate prompt template. The candidate prompt template can be evaluated in any manner to obtain the template evaluation result. For example, the template evaluation result can be obtained based on the user's actions, or it can be obtained by processing the candidate prompt template using a pre-trained neural network model.

[0044] According to embodiments of this disclosure, determining a target prompt template based on at least one candidate prompt template and a template evaluation result for the at least one candidate prompt template may include determining the target prompt template from at least one candidate prompt template based on the template evaluation result. Alternatively, it may include processing the candidate prompt templates and the template evaluation result using a pre-trained deep learning model to determine the target prompt template from at least one candidate prompt template. Alternatively, the candidate prompt templates may be rewritten based on the template evaluation result to obtain the target prompt template.

[0045] According to embodiments of this disclosure, processing prompt information using a language model to obtain template defect information may include: processing the prompt information using a language model to generate defect description text characterizing the defect attributes of the prompt template; and determining the template defect information based on the defect description words in the defect description text.

[0046] According to embodiments of this disclosure, the prompt information may include a prompt template and a sequence of prompt tags to help the language model understand the intent of defect attribute detection. The prompt tag sequence can be used to help the language model understand the permission attribute detection task type of the prompt information, thereby allowing the language model to input the prompt information and output defect description text that can accurately represent the defect attributes of the prompt template. The defect description text may include defect description words that represent defect attributes, such as "unclear expression," etc.

[0047] It should be noted that a sequence of prompt markers can include multiple prompt markers, and a prompt marker can include the smallest unit of the prompt information, such as a word, phrase, or field.

[0048] For example, the prompt message could be a paragraph enclosed in " / / ":

[0049] The current prompt template is:

[0050] "{prompt}"

[0051] What's wrong with this prompt template? Please provide examples of the problem, one for each example. <start>and <end>pack. / /

[0052] In the prompt message, {prompt} can represent the prompt template to be updated, and each word or field in the prompt message can be represented as a prompt tag.

[0053] According to embodiments of this disclosure, the defect description text can characterize the defect attributes of the prompt template based on natural language. For example, it can characterize the defect attributes based on examples, thereby clearly representing the defects of the prompt template and improving the interpretability of the defect attributes. For another example, the defect description text can include text from examples. Furthermore, the defect description text can also characterize the defect attributes based on natural language descriptions of defect attribute types.

[0054] According to embodiments of this disclosure, determining template defect information based on defect description words in defect description text may include: constructing cause requirement prompt information based on defect description words; and processing cause requirement prompt information using a language model to obtain defect cause description text, wherein the defect cause description text includes defect cause information, and the template defect information includes defect cause information.

[0055] According to embodiments of this disclosure, the cause requirement prompt information can be used to control the language model to understand the task requirement for the cause of the defect in the prompt template, and to prompt the language model to detect the cause of the defect. The cause requirement prompt information may include defect description words and a sequence of cause prompt tags, so that the language model can generate defect cause description text describing the cause of the defect in the prompt template by processing the cause requirement prompt information, thereby improving the defect cause detection accuracy and the interpretability of the defect cause, and providing a basis for subsequently generating updated candidate prompt templates based on the defect cause information. In this way, the candidate prompt templates can overcome the defect through interpretable defect cause information, thereby improving the quality of the candidate prompt templates.

[0056] According to embodiments of this disclosure, constructing cause requirement prompt information based on defect description words may include: constructing cause requirement prompt information based on prompt template, defect description words, and cause prompt tag sequence; wherein, the cause prompt tag sequence is suitable for controlling the language model to understand cause requirements.

[0057] According to embodiments of this disclosure, the constructed prompt information including the prompt template, defect description text, and cause prompt tag sequence can be used to...

[0058] For example, the reason / requirement prompt can be a paragraph enclosed in " / / " as follows:

[0059] I'm trying to write a zero-shot classifier.

[0060] The current hint is:

[0061] "{prompt}"

[0062] However, it fails in the following example:

[0063] {errors}

[0064] Please provide {nums} reasons why this hint might cause errors in these examples.

[0065] Every reason is based on <start>and <end>pack / /

[0066] In this cause-response requirement prompt message, {prompt} can represent the prompt template to be updated, {errors} can represent the defect description, and the words or fields in the cause-response requirement prompt message can represent cause-response prompt tags. It can be understood that {prompt}, {errors}, and {nums} can be variables in the cause-response requirement prompt message.

[0067] It should be noted that in the description text of the defect reasons output by the language model, <start>and <end>The fields between can be defect cause information, which can be represented based on natural language descriptions or based on identifiers that characterize the cause of the defect.

[0068] According to embodiments of this disclosure, by constructing cause requirement prompt information based on the prompt template, defect description words, and cause prompt mark sequence, the language model can more accurately detect the cause of the defect in the prompt template under the condition of fully understanding the prompt template, thereby improving the accuracy of defect cause information.

[0069] According to embodiments of this disclosure, updating the prompt template based on template defect information to obtain at least one candidate prompt template may include: constructing template update prompt information based on defect cause information, template defect information, and the prompt template to be updated; inputting the template update prompt information into a language model; and outputting at least one candidate prompt template.

[0070] For example, a template update notification could be a paragraph enclosed in " / / ":

[0071] I'm trying to write a zero-shot classifier.

[0072] The current hint is:

[0073] "{prompt}"

[0074] However, it fails in the following example:

[0075] {errors}

[0076] Based on these examples, the problem with this suggestion is:

[0077] {gradients}

[0078] Based on the information above, I wrote {steps} different improvement suggestions. Each suggestion uses... <start>and <end>Package. This {steps} new tip is: / /

[0079] In this template update prompt message, {prompt} can represent the prompt template to be updated, {errors} can represent the defect description, and {gradients} can represent the defect cause information. The words or fields in the template update prompt message can be represented as template update prompt tags. It can be understood that {prompt}, {errors}, {gradients}, and {steps} can be variables in the template update prompt message.

[0080] According to embodiments of this disclosure, the prompt marker in the prompt message and the reason prompt marker in the reason requirement prompt message can be reused as the template update prompt marker in the template update prompt message, thereby saving the computational overhead and computation time of constructing the template update prompt marker and improving the generation efficiency of candidate prompt templates.

[0081] According to embodiments of this disclosure, the method for processing prompt templates applied to a language model may further include: processing preset evaluation rules and candidate prompt templates using a language model to obtain evaluation steps corresponding to the evaluation rules; and evaluating the candidate prompt templates according to the evaluation steps to obtain template evaluation results for the candidate prompt templates.

[0082] According to embodiments of this disclosure, preset evaluation rules can characterize the quality type related to the candidate prompt template. Evaluation rules may include, for example, general evaluation rules, explicit evaluation rules, etc. Embodiments of this disclosure do not limit the specific quality type represented by the evaluation rules, and those skilled in the art can select according to actual needs.

[0083] According to embodiments of this disclosure, evaluation step prompt information can be constructed based on evaluation rules and candidate prompt templates. The evaluation step prompt information is input into a language model, and predicted text describing the evaluation rules can be output, thereby obtaining the evaluation steps from the predicted text.

[0084] According to embodiments of this disclosure, evaluation step prompts corresponding to evaluation rules can be represented based on paragraphs enclosed in " / / ".

[0085] / / Please generate evaluation steps for the Prompt content according to general evaluation rules:

[0086] Evaluation rules:

[0087] {{General Evaluation Rules}}

[0088] Prompt:

[0089] {{Prompt}} / /

[0090] In this evaluation step prompt information, {{Prompt}} can represent a candidate prompt template to be evaluated, and {{General Evaluation Rule}} can represent an evaluation rule. By constructing evaluation step prompt information that includes evaluation rules and candidate prompt templates, the language model can be controlled to understand the quality type being evaluated for the candidate prompt template. This allows for a more accurate determination of the evaluation steps applicable to evaluating the candidate prompt template, providing a more accurate implementation method for subsequent precise evaluation of the candidate prompt template, thereby improving the evaluation accuracy of the candidate prompt template.

[0091] According to embodiments of this disclosure, the evaluation rules include multiple rules.

[0092] According to embodiments of this disclosure, evaluating candidate suggestion templates according to evaluation steps to obtain template evaluation results for candidate suggestion templates may include: processing evaluation steps and candidate suggestion templates using a language model, and generating candidate prediction text based on candidate suggestion templates to obtain template evaluation sub-results corresponding to evaluation rules; and determining template evaluation results for candidate suggestion templates based on template evaluation sub-results corresponding to multiple evaluation rules.

[0093] Figure 3 The illustration schematically depicts an application scenario diagram of evaluating candidate suggestion templates according to an embodiment of the present disclosure.

[0094] like Figure 3 As shown, this application scenario can include a client and a server. The client can respond to the user's operation and determine the evaluation step prompt information 310 based on the candidate prompt template 311 and the evaluation rule 312. Inputting the evaluation step prompt information 310 into the language model 320 can output the evaluation step 321. The evaluation step 321 can be returned to the client, and a template evaluation prompt information 330 can be generated based on the template evaluation prompt information 310 and the evaluation step 321.

[0095] like Figure 3 As shown, processing the evaluation step and candidate prompt template using a language model, and generating candidate prediction text based on the candidate prompt template, may include inputting template evaluation prompt information 330 into the language model 320. The language model 320 can generate candidate prediction text based on the candidate prompt template 311 contained in the template evaluation prompt information 330. Then, the language model 320 can process the candidate prediction text, candidate template 311, and evaluation step 321 to generate a template evaluation sub-result 322 corresponding to the evaluation rule 312. The template evaluation sub-result 322 may include an evaluation score, or it may also include descriptive text describing the degree of difference between the candidate prompt template 311 and the evaluation rule 312. This embodiment does not limit the specific type of the template evaluation sub-result 322.

[0096] It should be understood that the candidate prediction text can be the text generated after the candidate prompt template is understood by the control language model 320 to predict the task type. The quality of the candidate prediction text can characterize the quality level of the candidate prompt template in relation to the card evaluation rule 312. Furthermore, by using the language model to process the evaluation steps and candidate prompt templates, as well as the candidate prediction text generated based on the candidate prompt templates, the output evaluation steps can be adapted to the candidate prompt templates and evaluation rules, thereby improving the accuracy of the candidate template evaluation results.

[0097] For example, the evaluation step prompt information may also include task variables that populate the candidate prompt template. By populating the candidate prompt template with task variables, candidate prompt information can be obtained, which can then be input into the language model to output candidate predicted text.

[0098] According to embodiments of this disclosure, the preset evaluation rules can be determined based on evaluation indicators corresponding to at least one of the following quality types: clarity, consistency, usability, innovativeness, and universality. For example, the evaluation rules may include five, characterized by clarity indicators, consistency indicators, usability indicators, innovativeness indicators, and universality indicators, respectively. Clarity indicators, consistency indicators, usability indicators, innovativeness indicators, and universality indicators can be described based on the content in Table 1 below.

[0099] Table 1

[0100]

[0101] As shown in Table 1, different weights can be set for different evaluation indicators based on the content shown in Table 1. When the template evaluation sub-results corresponding to multiple evaluation rules are evaluation scores, the weights corresponding to the evaluation indicators and the template evaluation sub-results can be weighted and summed to obtain the evaluation score (template evaluation result) corresponding to the candidate prompt template. For example, the template evaluation result can be determined based on the following formula (1).

[0102]

[0103] In formula (1), p(s) i ) represents the weight corresponding to the i-th evaluation index, s i represents the template evaluation sub-result corresponding to the i-th evaluation indicator, score represents the template evaluation result, and n represents the number of evaluation indicators.

[0104] According to embodiments of this disclosure, the template evaluation sub-result can be characterized based on indicator evaluation scores of 1 to 5, and the template evaluation sub-result can further include the indicator score prediction probability corresponding to each of the indicator evaluation scores 1 to 5, and the descriptive text corresponding to each indicator score prediction probability. By weighted summing of the indicator score prediction probability and the indicator evaluation score, the target indicator score representing the candidate prompt target's relevance to the evaluation indicator can be determined. Based on the target indicator scores corresponding to each of the multiple evaluation indicators, and the weights corresponding to each of the multiple evaluation indicators, the evaluation score representing the template evaluation result can be determined.

[0105] For example, a language model can input the following template evaluation prompt to determine the template evaluation sub-outcome. The template evaluation prompt is a paragraph enclosed by the / / symbol:

[0106] / / Your task is to score the input and output according to the evaluation steps, using the evaluation metrics dimension {evaluation metrics}, with scores ranging from 1 to 5, where 1 is the lowest and 5 is the highest, and calculate the probability of each possible score.

[0107] Please output only the JSON result, in the following format:

[0108] {′prob1′: {{p11}}, ′prob2′: {{p12}}, ′prob3′: {{p13}}, ′prob4′: {{p14}}, ′prob5′: {{p15}}, ′desc′: {{desc}}}

[0109] Please ensure you read and understand these instructions carefully. Please keep this document open while reviewing it and refer to it when needed.

[0110] Evaluation criteria:

[0111] {{Evaluation Indicators and Standards}}

[0112] Evaluation steps:

[0113] {{Evaluation Steps}}

[0114] Input:

[0115] {{Prompt}}

[0116] Output:

[0117] {{Output}} / /

[0118] Where {{p11}}, {{p12}}, {{p13}}, {{p14}}, and {{p15}} represent the predicted probabilities of the evaluation index for scores 1, 2, 3, 4, and 5, respectively, with the sum of the predicted probabilities being 1. {{desc}} can represent descriptive text explaining the predicted probabilities for determining the index score, thus improving the interpretability of the candidate suggestion template evaluation through descriptive text and predicted probabilities. Furthermore, the template evaluation sub-results can be displayed on user-related clients to help users further improve the candidate suggestion templates.

[0119] According to embodiments of this disclosure, determining a target prompt template based on at least one candidate prompt template and a template evaluation result for the at least one candidate prompt template may include: determining an intermediate prompt template from at least one candidate prompt template based on the template evaluation result; generating template modification prompt information based on the modification text corresponding to the modification instruction in response to a modification instruction for the intermediate prompt template; and processing the template modification prompt information according to a language model to obtain the target prompt template.

[0120] According to embodiments of this disclosure, candidate prompt templates whose evaluation scores, as represented by the template evaluation results, are greater than a preset score threshold can be determined as intermediate prompt templates. Alternatively, when there are multiple candidate prompt templates, the candidate prompt template with the highest evaluation score, as represented by the template evaluation results, can be determined as an intermediate prompt template, thereby allowing the selection of an intermediate prompt template with a relatively high quality level.

[0121] According to embodiments of this disclosure, the modification instruction can be determined based on user-inputted modification text, which may include text input through the interactive interface after an intermediate prompt template is displayed. Alternatively, the modification instruction may be generated through preset configuration options before generating the intermediate prompt template. Embodiments of this disclosure do not limit the specific method of obtaining the modification instruction.

[0122] According to embodiments of this disclosure, the modified text may include modification suggestions for characterizing modifications to the intermediate prompt template. By using a language model to process the template modification prompt information generated based on the modified text, the language model can be controlled to fully understand the modification method of the intermediate prompt template, thereby enabling precise modification of the intermediate prompt template according to the modification instructions and generating a target prompt template that matches the modification instructions.

[0123] According to embodiments of this disclosure, the modified text can represent a suggestion to modify the attributes of the intermediate prompt template, such as its length, word accuracy, and application scenario.

[0124] For example, template modification prompts could be based on paragraphs enclosed by the symbol " / / ":

[0125] The current prompt template is:

[0126] {prompt}

[0127] Improve the above prompt template based on the following suggestions:

[0128] {suggest}

[0129] The improved prompt is: / /

[0130] Here, {suggest} can represent a suggestion for modification corresponding to the modified text.

[0131] For example, template modification prompts can also be based on paragraphs enclosed by the symbol " / / ":

[0132] The current prompt is:

[0133] {prompt}

[0134] Please shorten the suggestion while preserving its semantics. The shortened suggestion is: / /

[0135] The phrase "Please shorten the prompt while preserving the semantics" can represent a modification suggestion corresponding to the modified text.

[0136] According to embodiments of this disclosure, determining an intermediate prompt template from at least one candidate prompt template based on the template evaluation results may further include: determining candidate prompt templates obtained in each of the N stages based on the template evaluation results corresponding to the prompt templates; and determining at least one intermediate prompt template from the candidate prompt templates corresponding to each of the N stages.

[0137] According to an embodiment of this disclosure, the nth candidate prompt template corresponding to the nth stage is generated in the following manner, the manner comprising: constructing n-1 candidate prompt information according to an update instruction and at least one n-1 candidate prompt template, wherein the n-1 candidate prompt template is determined based on the n-1 template evaluation result determined in the n-1 stage, n>1, and n is an integer, and the first template evaluation result is the template evaluation result corresponding to the prompt template; processing the n-1 candidate prompt information using a language model to obtain n-1 candidate template defect information corresponding to the n-1 candidate prompt template; and updating the n-1 candidate prompt template according to the n-1 candidate template defect information to obtain the nth candidate prompt template.

[0138] For example, in the first stage, the first template evaluation result is the template evaluation result corresponding to the prompt template to be updated. The first candidate prompt template can be a candidate prompt template obtained based on the prompt template processing method provided in the above embodiments of this disclosure. In the second stage, the first candidate prompt template obtained in the first stage can be processed based on the prompt template processing method provided in the above embodiments to update and optimize the first candidate prompt template, thereby obtaining the second candidate prompt template.

[0139] It should be understood that, when N=n=3, for the third stage, the second candidate prompt template obtained in the second stage can be processed based on the prompt template processing method provided in the above embodiments to update and optimize the second candidate prompt template, thereby obtaining the third candidate prompt template. Thus, candidate prompt templates corresponding to each of the three stages can be obtained.

[0140] According to embodiments of this disclosure, determining at least one intermediate prompt template from candidate prompt templates corresponding to each of the N stages may include determining the intermediate prompt template from at least one nth candidate prompt template obtained from the nth stage. For example, the nth candidate prompt template with the highest evaluation score represented by the evaluation result of the nth template may be determined as the intermediate prompt template.

[0141] According to embodiments of this disclosure, determining at least one intermediate prompt template from the candidate prompt templates corresponding to each of the N stages may further include: determining the candidate prompt template with the highest evaluation score represented by the template evaluation result from the candidate prompt templates corresponding to each of the N stages as the intermediate prompt template, thereby increasing the selection range of intermediate prompt templates.

[0142] According to embodiments of this disclosure, new candidate prompt templates are generated iteratively through multiple stages, which can achieve multi-level iterative optimization of the prompt templates to be updated. This enables the intermediate prompt templates to control the large language model for text prediction with higher quality, thereby improving the quality of the answers in question-and-answer scenarios.

[0143] Figure 4 A flowchart illustrating a method for processing a prompt template applied to a language model according to another embodiment of the present disclosure is shown.

[0144] like Figure 4 As shown, the method for processing prompt templates applied to a language model in this embodiment may include operations S401 to S410.

[0145] After the start operation is executed, operation S401 can be performed. In operation S401, a prompt message containing the prompt template to be updated and the update task attributes can be constructed based on the update instruction and the prompt template.

[0146] In operation S402, the prompt information can be processed using a language model to obtain the defect description text used to describe the defect attributes of the prompt template.

[0147] In operation S403, based on the defect description words in the defect description text, construct the cause requirement prompt information corresponding to the prompt template.

[0148] In operation S404, the language model is used to process the cause and requirement prompt information corresponding to the prompt template, and the defect cause description text corresponding to the prompt template is obtained.

[0149] When operating S405, a template update prompt message is generated based on the defect cause information and template defect information.

[0150] In operation S406, the language model is used to process the template update prompt information to obtain the candidate prompt template generated based on the prompt template, namely the first candidate prompt template.

[0151] In operation S407, it is determined whether the candidate suggestion templates meet the requirements. For example, the evaluation results of each candidate suggestion template can be compared with a preset evaluation score threshold to determine whether the candidate suggestion templates meet the requirements. Alternatively, a judgment can be made based on a preset number of iterations. If the preset number of iterations has not been reached, the judgment result of operation S407 is negative. If the preset number of iterations has been reached, the judgment result of operation S407 is positive.

[0152] If the result of operation S407 is negative, operation S401 to operation S407 can be executed repeatedly for each stage's corresponding candidate prompt template until the result of operation S407 is positive, at which point operation S408 is executed.

[0153] In operation S408, an intermediate prompt template is determined from the candidate prompt templates obtained from N stages.

[0154] In operation S409, a template modification prompt message is constructed based on the modification text representing the modification suggestion for the intermediate prompt template and the intermediate prompt template.

[0155] In operation S409, the language model is used to process the template modification prompt information to obtain the target prompt template. This allows for multiple rounds of iterative updates to the prompt template, resulting in a target prompt template suitable for controlling the language model to complete the prediction task.

[0156] Figure 5 A schematic diagram of a prompt template update platform according to an embodiment of the present disclosure is shown.

[0157] like Figure 5 As shown, the prompt template update platform 500 may include an optimization method selector 510, a prompt template optimizer 520, and a comparison evaluator 530. Prompt information 501, containing prompt template 5011 and task variables 5012, can be transmitted to the optimization method selector 510, and based on the user's selection, the prompt information 501 can be sent to the prompt template optimizer 520 from any one of the online optimization module, batch optimization module, and API (Application Programming Interface) optimization module. The online optimization module enables online updating and optimization of prompt templates, facilitating the rapid return of the optimized target prompt template. The batch optimization module is suitable for batch optimization of a large number of prompt templates to improve optimization efficiency. The API optimization module allows users to remotely call the prompt template update platform 500 by customizing API interface services for relevant clients to optimize prompt templates.

[0158] like Figure 5 As shown, the prompt template optimizer 520 can be used to execute the processing method for prompt templates applied to a language model according to the embodiments of this disclosure. The prompt template optimizer 520 may include an optimization configuration rule module, an optimization strategy module, and a pre-trained large language model. The optimization configuration rule module can configure optimization parameter information such as optimization iteration rounds, improvement suggestions, and quality score settings based on the iteration round configuration unit, improvement suggestion configuration unit, and quality score configuration unit, respectively, according to user operations or instructions. This allows for accurate configuration of the user's requirements for optimizing the prompt template, facilitating the optimization strategy execution module to accurately optimize and update the prompt template based on the configured parameters. The optimization strategy execution module can invoke the large language model based on the processing method provided in the embodiments of this disclosure to obtain the target prompt template 5021.

[0159] like Figure 5 As shown, target prompt template 5021 and corresponding task variable 5022 can be used to construct target prompt information 502. By processing target prompt information 502 using a large language model, target inference result 502' can be obtained. Target inference result 502' can include the predicted text output after inputting target prompt information 502 into the large language model. For example, prompt information 501 can also be input into the large language model to output inference result 501'. The quality of inference result 501' and target inference result 502' is evaluated by a comparison evaluator 530, thereby replacing the prompt template with a target prompt template that meets the preset conditions, thus satisfying the user's update needs for prompt templates.

[0160] Figure 6 A block diagram of a processing apparatus for a prompt template applied to a language model according to an embodiment of the present disclosure is shown schematically.

[0161] like Figure 6 As shown, the processing device 600 for prompt templates applied to a language model includes: a prompt information construction module 610, a template defect information acquisition module 620, a candidate prompt template acquisition module 630, and a target prompt template determination module 640.

[0162] The prompt message construction module 610 is used to construct prompt messages in response to an update command representing an update prompt template, based on the update command and the prompt template.

[0163] The template defect information acquisition module 620 is used to process the prompt information using a language model to obtain template defect information.

[0164] The candidate prompt template acquisition module 630 is used to update the prompt template according to the template defect information to obtain at least one candidate prompt template.

[0165] The target prompt template determination module 640 is used to determine a target prompt template based on at least one candidate prompt template and the template evaluation result for the at least one candidate prompt template.

[0166] According to embodiments of this disclosure, the template defect information acquisition module includes: a defect description text generation submodule and a template defect information determination submodule.

[0167] The defect description text generation submodule is used to process the prompt information using a language model and generate defect description text that represents the defect attributes of the prompt template.

[0168] The template defect information determination submodule is used to determine template defect information based on the defect description words in the defect description text.

[0169] According to embodiments of this disclosure, the template defect information determination submodule includes: a cause requirement prompt information construction unit and a defect cause description text acquisition unit.

[0170] The cause and requirement prompt information construction unit is used to construct cause and requirement prompt information based on the defect description.

[0171] The defect cause description text acquisition unit is used to process the cause requirement prompt information using a language model to obtain the defect cause description text, wherein the defect cause description text includes defect cause information, and the template defect information includes defect cause information.

[0172] According to embodiments of this disclosure, the cause requirement prompt information construction unit includes: a cause requirement prompt information construction subunit.

[0173] The cause and requirement prompt information construction subunit is used to construct cause and requirement prompt information based on the prompt template, defect description words and cause prompt mark sequence; wherein, the cause prompt mark sequence is suitable for the control language model to understand cause and requirement.

[0174] According to embodiments of this disclosure, the processing apparatus for prompt templates applied to a language model further includes: an evaluation step acquisition module and a template evaluation result acquisition module.

[0175] The evaluation step acquisition module is used to process the preset evaluation rules and candidate prompt templates using a language model to obtain the evaluation steps corresponding to the evaluation rules.

[0176] The template evaluation result acquisition module is used to evaluate the candidate prompt templates according to the evaluation steps and obtain the template evaluation results for the candidate prompt templates.

[0177] According to embodiments of this disclosure, the evaluation rules include multiple rules.

[0178] According to embodiments of this disclosure, the template evaluation result acquisition module includes: a template evaluation sub-result acquisition sub-module and a template evaluation result acquisition sub-module.

[0179] The template evaluation sub-result acquisition sub-module is used to process the evaluation steps and candidate prompt templates using a language model, as well as the candidate prediction text generated based on the candidate prompt templates, to obtain the template evaluation sub-results corresponding to the evaluation rules.

[0180] The template evaluation result acquisition submodule is used to determine the template evaluation result for the candidate prompt template based on the template evaluation sub-results corresponding to each of the multiple evaluation rules.

[0181] According to embodiments of this disclosure, the target prompt template determination module includes: an intermediate prompt template determination submodule, a template modification prompt information generation submodule, and a target prompt template acquisition submodule.

[0182] The intermediate prompt template determination submodule is used to determine an intermediate prompt template from at least one candidate prompt template based on the template evaluation results.

[0183] The template modification prompt message generation submodule is used to respond to modification instructions for intermediate prompt templates and generate template modification prompt messages based on the modification text corresponding to the modification instructions.

[0184] The target prompt template acquisition submodule is used to process the template and modify the prompt information according to the language model to obtain the target prompt template.

[0185] According to embodiments of this disclosure, the intermediate prompt template determination submodule includes: a candidate prompt template determination unit and an intermediate prompt template determination unit.

[0186] The candidate prompt template determination unit is used to determine the candidate prompt templates obtained in each of the N stages based on the template evaluation results corresponding to the prompt template. The nth candidate prompt template corresponding to the nth stage is generated as follows: constructing the (n-1)th candidate prompt information based on the update instruction and at least one (n-1)th candidate prompt template, wherein the (n-1)th candidate prompt template is determined based on the (n-1)th template evaluation result determined in the (n-1)th stage, where n > 1 and n is an integer, and the first template evaluation result is the template evaluation result corresponding to the prompt template; processing the (n-1)th candidate prompt information using a language model to obtain the (n-1)th candidate template defect information corresponding to the (n-1)th candidate prompt template; and updating the (n-1)th candidate prompt template based on the (n-1)th candidate template defect information to obtain the nth candidate prompt template.

[0187] The intermediate prompt template determination unit is used to determine at least one intermediate prompt template from the candidate prompt templates corresponding to each of the N stages.

[0188] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0189] According to an embodiment of the present disclosure, an electronic device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform a method provided according to an embodiment of the present disclosure.

[0190] According to embodiments of the present disclosure, a non-transitory computer-readable storage medium stores computer instructions, wherein the computer instructions are used to cause a computer to perform a method provided according to embodiments of the present disclosure.

[0191] According to an embodiment of the present disclosure, a computer program product includes a computer program that, when executed by a processor, implements the method provided according to an embodiment of the present disclosure.

[0192] Figure 7 The block diagram illustrates an electronic device suitable for implementing a processing method for prompt templates applied to a language model, according to embodiments of the present disclosure. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0193] like Figure 7 As shown, device 700 includes a computing unit 701, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 702 or a computer program loaded from storage unit 708 into random access memory (RAM) 703. RAM 703 may also store various programs and data required for the operation of device 700. The computing unit 701, ROM 702, and RAM 703 are interconnected via bus 704. Input / output (I / O) interface 705 is also connected to bus 704.

[0194] Multiple components in device 700 are connected to I / O interface 705, including: input unit 706, such as keyboard, mouse, etc.; output unit 707, such as various types of monitors, speakers, etc.; storage unit 708, such as disk, optical disk, etc.; and communication unit 709, such as network card, modem, wireless transceiver, etc. Communication unit 709 allows device 700 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0195] The computing unit 701 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 701 performs the various methods and processes described above, such as the processing method for prompt templates applied to a language model. For example, in some embodiments, the processing method for prompt templates applied to a language model can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 708. In some embodiments, part or all of the computer program can be loaded and / or installed on device 700 via ROM 702 and / or communication unit 709. When the computer program is loaded into RAM 703 and executed by the computing unit 701, one or more steps of the processing method for prompt templates applied to a language model described above can be performed. Alternatively, in other embodiments, the computing unit 701 may be configured by any other suitable means (e.g., by means of firmware) to perform a processing method for a prompt template applied to a language model.

[0196] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0197] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0198] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0199] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0200] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0201] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, distributed system servers, or servers incorporating blockchain technology.

[0202] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0203] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.< / end> < / start> < / end> < / start> < / end> < / start> < / end> < / start>

Claims

1. A method for processing prompt templates applied to a language model, comprising: In response to an update instruction representing an update prompt template, a prompt message is constructed based on the update instruction and the prompt template; The prompt information is processed using a language model to obtain template defect information; The prompt template is updated based on the template defect information to obtain at least one candidate prompt template; Based on the template evaluation results for the candidate prompt templates, an intermediate prompt template is determined from at least one of the candidate prompt templates; In response to a modification instruction for the intermediate prompt template, a template modification prompt message is generated based on the modification text corresponding to the modification instruction; The template modification prompt information is processed according to the language model to obtain the target prompt template; The intermediate prompt template is determined based on the following operations: Based on the template evaluation results corresponding to the prompt template, candidate prompt templates are determined for each of the N stages. The nth candidate prompt template corresponding to the nth stage is generated in the following manner: Based on the update instruction and at least one (n-1)th candidate prompt template, construct (n-1)th candidate prompt information, wherein the (n-1)th candidate prompt template is determined based on the (n-1)th template evaluation result determined in the (n-1)th stage, n>1, and n is an integer, and the first template evaluation result is the template evaluation result corresponding to the prompt template; The language model is used to process the (n-1)th candidate prompt information to obtain the (n-1)th candidate template defect information corresponding to the (n-1)th candidate prompt template; and The (n-1)th candidate prompt template is updated based on the defect information of the (n-1)th candidate template to obtain the nth candidate prompt template; and At least one intermediate prompt template is determined from the candidate prompt templates corresponding to each of the N stages.

2. The method according to claim 1, wherein, The process of using a language model to process the prompt information to obtain template defect information includes: The language model is used to process the prompt information to generate a defect description text that represents the defect attributes of the prompt template; The template defect information is determined based on the defect description words in the defect description text.

3. The method according to claim 2, wherein, The step of determining the template defect information based on the defect description words in the defect description text includes: Based on the defect description, construct a cause and requirement suggestion message; and The language model is used to process the cause requirement prompt information to obtain a defect cause description text, wherein the defect cause description text includes defect cause information, and the template defect information includes the defect cause information.

4. The method according to claim 3, wherein, The step of constructing the cause requirement prompt information based on the defect description includes: Based on the prompt template, the defect description word, and the cause prompt tag sequence, the cause requirement prompt information is constructed; wherein, the cause prompt tag sequence is used to control the language model to understand the cause requirement.

5. The method according to claim 1, further comprising: The language model is used to process the preset evaluation rules and the candidate prompt templates to obtain the evaluation steps corresponding to the evaluation rules; The candidate suggestion template is evaluated according to the evaluation steps to obtain the template evaluation result for the candidate suggestion template.

6. The method according to claim 5, wherein, The evaluation rules include multiple rules; The step of evaluating the candidate suggestion template according to the evaluation steps to obtain the template evaluation result for the candidate suggestion template includes: The language model is used to process the evaluation step and the candidate suggestion template, as well as the candidate predicted text generated based on the candidate suggestion template, to obtain a template evaluation sub-result corresponding to the evaluation rule; and Based on the template evaluation sub-results corresponding to each of the multiple evaluation rules, a template evaluation result is determined for the candidate prompt template.

7. A processing apparatus for prompt templates applied to a language model, comprising: The prompt information construction module is used to construct prompt information in response to an update instruction representing an update prompt template, based on the update instruction and the prompt template; The template defect information acquisition module is used to process the prompt information using a language model to obtain template defect information. A candidate prompt template obtaining module is used to update the prompt template according to the template defect information to obtain at least one candidate prompt template; as well as A target prompt template determination module is used to determine a target prompt template based on at least one of the candidate prompt templates and a template evaluation result for the at least one of the candidate prompt templates; The target prompt template determination module includes: An intermediate prompt template determination submodule is used to determine an intermediate prompt template from at least one of the candidate prompt templates based on the template evaluation results; A template modification prompt information generation submodule is used to respond to a modification instruction for the intermediate prompt template and generate template modification prompt information based on the modification text corresponding to the modification instruction; and The target prompt template acquisition submodule is used to process the template modification prompt information according to the language model to obtain the target prompt template; The intermediate prompt template determination submodule includes: The candidate prompt template determination unit is used to determine candidate prompt templates for each of the N stages based on the template evaluation results corresponding to the prompt template, wherein the nth candidate prompt template corresponding to the nth stage is generated in the following manner: Based on the update instruction and at least one (n-1)th candidate prompt template, construct (n-1)th candidate prompt information, wherein the (n-1)th candidate prompt template is determined based on the (n-1)th template evaluation result determined in the (n-1)th stage, n>1, and n is an integer, and the first template evaluation result is the template evaluation result corresponding to the prompt template; The language model is used to process the (n-1)th candidate prompt information to obtain the (n-1)th candidate template defect information corresponding to the (n-1)th candidate prompt template; and The (n-1)th candidate prompt template is updated based on the defect information of the (n-1)th candidate template to obtain the nth candidate prompt template; and The intermediate prompt template determination unit is used to determine at least one intermediate prompt template from the candidate prompt templates corresponding to each of the N stages.

8. The apparatus according to claim 7, wherein, The template defect information acquisition module includes: The defect description text generation submodule is used to process the prompt information using the language model and generate defect description text that characterizes the defect attributes of the prompt template. The template defect information determination submodule is used to determine the template defect information based on the defect description words in the defect description text.

9. The apparatus according to claim 8, wherein, The template defect information determination submodule includes: A cause-requirement prompt information construction unit is used to construct cause-requirement prompt information based on the defect description words; and The defect cause description text acquisition unit is used to process the cause requirement prompt information using the language model to obtain defect cause description text, wherein the defect cause description text includes defect cause information, and the template defect information includes the defect cause information.

10. The apparatus according to claim 9, wherein, The reason requirement prompt information construction unit includes: The cause and requirement prompt information construction subunit is used to construct the cause and requirement prompt information based on the prompt template, the defect description word, and the cause prompt mark sequence; wherein, the cause prompt mark sequence is used to control the language model to understand the cause and requirement.

11. The apparatus according to claim 7, further comprising: The evaluation step acquisition module is used to process the preset evaluation rules and the candidate prompt templates using the language model to obtain the evaluation steps corresponding to the evaluation rules; The template evaluation result acquisition module is used to evaluate the candidate prompt template according to the evaluation steps to obtain the template evaluation result for the candidate prompt template.

12. The apparatus according to claim 11, wherein, The evaluation rules include multiple rules; The template evaluation result acquisition module includes: The template evaluation sub-result acquisition submodule is used to process the evaluation step and the candidate prompt template using the language model, and the candidate predicted text generated based on the candidate prompt template, to obtain the template evaluation sub-result corresponding to the evaluation rule; and The template evaluation result acquisition submodule is used to determine the template evaluation result for the candidate prompt template based on the template evaluation sub-results corresponding to each of the multiple evaluation rules.

13. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1 to 6.

14. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1 to 6.

15. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1 to 6.

Images