Prompt word template processing method, device and platform and related equipment

Abstract

The invention provides a cue word template processing method, and the method comprises the steps: obtaining a plurality of cue word templates, a data set and gradient flow information, the gradient flow information being used for indicating the priority degree of selection of each evaluation algorithm in a plurality of evaluation algorithms; according to the gradient flow information, a first evaluation algorithm matched with the data set is screened out from multiple evaluation algorithms, and the priority degree of the first evaluation algorithm is higher than that of other evaluation algorithms matched with the data set; and evaluating the plurality of cue word templates according to a first evaluation algorithm and the data set, and determining a first cue word template with an optimal evaluation result. Thus, the first evaluation algorithm can be automatically screened out to evaluate the quality of the multiple cue word templates, a user does not need to manually select the evaluation algorithm, the requirement for the technical threshold of the user can be effectively lowered, and the accuracy of determining the high-quality cue word templates can also be improved. In addition, the invention also provides a corresponding processing device, a platform and related equipment.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a prompt word template processing method, device, platform and related equipment. Background Art

[0002] A prompt word template refers to a template text including variables and fixed text content. By inputting the specific value of the variable, a prompt word can be dynamically created. Among them, a prompt word is a complete text content of a paragraph (or a sentence), which can be used as an input for a model such as a large language model (LLM), so that the model can output the corresponding inference result for the prompt word. In actual application, by adjusting the variable value in the prompt word template, multiple different prompt words can be dynamically generated.

[0003] Without adjusting the model's capabilities, tuning the prompt words is an effective way to obtain high-quality model output. For example, in the intelligent answering scenario, by optimizing the problem description (i.e., optimizing the prompt words), the model can output a better answer (i.e., the model's output). High-quality prompt words can be obtained by designing high-quality prompt word templates. Among them, a high-quality prompt word template refers to a prompt word generated based on a prompt word template for different groups of variable inputs. After being input into the model, it can obtain a high-quality model output, that is, the prompt word template has a high degree of versatility.

[0004] At present, the prompt word template can be automatically evaluated by using an artificially selected evaluation algorithm, and a relatively high-quality prompt word template can be selected from multiple prompt word templates designed by users (such as business personnel). In actual application, different evaluation algorithms have large differences in the evaluation results of prompt word templates. For example, assuming that the expected model output result is "positive", and the actual output result of the model is "positive emotion", when the evaluation algorithm such as precise matching is adopted, the actual output result of the model does not match the expected model output result, then the quality of the prompt word template is considered to be low; when the evaluation algorithm such as fuzzy matching is adopted, the actual output result of the model matches the expected model output result, then the quality of the prompt word template can be considered to be high. Therefore, for prompt word templates in different scenarios, users need to understand all evaluation algorithms and select the evaluation algorithm suitable for the scenario to evaluate the quality of the prompt word template, which will increase the user's technical threshold and reduce the accuracy of determining high-quality prompt word templates. Summary of the invention

[0005] In view of this, the embodiment of the present application provides a prompt word template processing method to reduce the technical threshold of users and improve the accuracy of determining high-quality prompt word templates. The present application also provides a corresponding processing device, a prompt word template processing platform, a computing device cluster, a computer-readable storage medium, and a computer program product.

[0006] In a first aspect, an embodiment of the present application provides a prompt word template processing method, which can be executed by a corresponding processing device. Specifically, the processing device obtains multiple prompt word templates and data sets, the variables in the multiple prompt word templates can be the same or different, and the obtained data sets include at least one set of variable values, each set of variable values ​​is used to indicate the values ​​of the variables in the multiple prompt word templates; and the processing device also obtains gradient process information, the gradient process information is used to indicate the priority of each evaluation algorithm selected from multiple evaluation algorithms, such as the priority can be indicated by information such as priority level or score, so that the processing device can screen out a first evaluation algorithm that matches the data set from multiple evaluation algorithms according to the gradient process information, and the priority of the first evaluation algorithm is higher than the priority of other evaluation algorithms that match the data set in the multiple evaluation algorithms; in this way, the processing device evaluates the multiple prompt word templates according to the first evaluation algorithm and the data set, and obtains evaluation results corresponding to the multiple prompt word templates respectively; and determines a first prompt word template from the multiple prompt word templates according to the evaluation results corresponding to the multiple prompt word templates respectively, and the evaluation result corresponding to the determined first prompt word template is better than the evaluation results corresponding to other prompt word templates in the multiple prompt word templates.

[0007] Since in the process of processing the prompt word template, the processing device can automatically select the first evaluation algorithm from the multiple evaluation algorithms according to the selected priorities corresponding to the multiple evaluation algorithms indicated in the gradient process information to evaluate the quality of the multiple prompt word templates, there is no need for the user to manually select the evaluation algorithm for the multiple prompt word templates, which can effectively reduce the technical threshold requirements for evaluating high-quality prompt word templates for users, that is, reduce the difficulty of evaluating high-quality prompt word templates. Moreover, the first evaluation algorithm automatically selected by the processing device matches the data set, which can ensure that the accuracy of evaluating the prompt word template using the first evaluation algorithm can reach a high level, thereby improving the accuracy of determining high-quality prompt word templates.

[0008] In one possible implementation, the gradient process information acquired by the processing device includes multiple gradients, each gradient includes an identifier of at least one evaluation algorithm, and the priority of the evaluation algorithms belonging to different gradients is different; wherein, the priority of different evaluation algorithms belonging to the same gradient is the same, or, the priority of different evaluation algorithms belonging to the same gradient is different. In this way, the processing device can preferentially filter out the evaluation algorithm that matches the data set from the gradient with a higher priority, and if there is no suitable evaluation algorithm in the gradient, filter it from the gradient with a lower priority. In this way, by setting the evaluation algorithm with greater versatility in the gradient with a higher priority, the efficiency of the processing device in determining the appropriate evaluation algorithm can be improved.

[0009] In a possible implementation, the gradient process information is specifically a tree diagram, which includes multiple nodes and multiple directed edges, wherein each of the multiple nodes is used to indicate an evaluation algorithm, and each of the multiple directed edges is used to indicate the priority relationship between different evaluation algorithms. For example, when the directed edge between node A and node B is from node A to node B, it can be determined that the priority of the evaluation algorithm indicated by node A is higher than the priority of the evaluation algorithm indicated by node B. In this way, the processing device uses a tree diagram to manage multiple evaluation algorithms, which can reduce management difficulty and improve management efficiency.

[0010] In a possible implementation, the multiple gradients in the gradient process information include a first gradient and a second gradient (and may include more gradients), wherein the priority of the evaluation algorithm belonging to the first gradient is higher than the priority of the evaluation algorithm belonging to the second gradient; then, when the processing device selects the first evaluation algorithm matching the data set from the multiple evaluation algorithms according to the gradient process information, it may specifically first detect whether the data set includes at least one set of expected output results corresponding to the variable values; when the data set does not include the expected output results, the first evaluation algorithm is determined from at least one evaluation algorithm belonging to the first gradient; and when the data set includes the expected output results, the first evaluation algorithm is determined from at least one evaluation algorithm belonging to the second gradient according to the characteristics of the expected output results. In this way, the processing device may select an evaluation algorithm matching the expected output results according to whether there are expected output results in the data set, thereby ensuring that the accuracy of evaluating the prompt word template using the evaluation algorithm can reach a high level, thereby improving the accuracy of determining a high-quality prompt word template.

[0011] In a possible implementation, the data set acquired by the processing device includes multiple groups of variable values. When the expected output result does not have the characteristics of the preset type, the processing device can also use the first group of variable values ​​in the multiple groups of variable values ​​to pre-evaluate multiple prompt word templates according to at least one evaluation algorithm belonging to the second gradient, and obtain the pre-evaluation result corresponding to each prompt word template in the multiple prompt word templates, so as to determine the first evaluation algorithm from at least one evaluation algorithm according to the pre-evaluation result corresponding to each prompt word template. In this way, the processing device can determine the appropriate evaluation algorithm by pre-evaluation, so as to improve the accuracy of the high-quality prompt word template finally determined.

[0012] In a possible implementation, when the processing device selects the first evaluation algorithm that matches the data set from multiple evaluation algorithms according to the gradient process information, it can specifically be in response to the user's selection operation on the automatic setting control, and selects the first evaluation algorithm that matches the data set from multiple evaluation algorithms according to the gradient process information. In this way, the processing device can automatically select the first evaluation algorithm that matches the data set under the user's instruction to determine a high-quality prompt word template, so that the user can intervene and control the process of determining the high-quality prompt word template, thereby improving the user's experience.

[0013] In a possible implementation, when multiple evaluation algorithms do not match the data set, the processing device may also obtain a second evaluation algorithm customized by the user; and determine a second prompt word template from multiple prompt word templates based on the second evaluation algorithm and the data set, wherein the evaluation result corresponding to the second prompt word template is better than the evaluation results corresponding to other prompt word templates in the multiple prompt word templates. In this way, the processing device may support the user to implement quality evaluation of multiple prompt word templates based on the customized evaluation algorithm, thereby improving the flexibility of the user in determining high-quality prompt word templates and improving the user's usage experience.

[0014] In a possible implementation, the processing device may also output the gradient flow information when the user does not perform the selection operation for the automatic setting control, and in response to the user's selection operation for multiple evaluation algorithms in the gradient flow information, determine the selected evaluation algorithm from the multiple evaluation algorithms as the third evaluation algorithm, so that the processing device determines the third prompt word template from the multiple prompt word templates according to the third evaluation algorithm and the data set, and the evaluation result corresponding to the third prompt word template is better than the evaluation results corresponding to other prompt word templates in the multiple prompt word templates. In this way, the processing device can intelligently guide the user to select a suitable evaluation algorithm to perform quality evaluation on the prompt word template by presenting the gradient flow information corresponding to multiple evaluation algorithms to the user, so as to assist the user in determining the appropriate evaluation algorithm and improve the user's use experience.

[0015] In a possible implementation, the processing device may also adjust the priority of the evaluation algorithm selected in the gradient process information in response to the modification operation on the gradient process information; or, add a new evaluation algorithm and the priority of the new evaluation algorithm in the gradient process information. In this way, the dynamic update of the gradient process information can be achieved, which enables the gradient process information to determine the appropriate evaluation algorithm for the user for a long time, thereby improving the practicality of the gradient process information.

[0016] In a second aspect, the present application provides a processing device, which includes: a configuration module for obtaining multiple prompt word templates; a data set management module for obtaining a data set, the data set including at least one set of variable values, each set of variable values ​​being used to indicate the values ​​of variables in multiple prompt word templates; an algorithm management module for obtaining gradient process information, the gradient process information being used to indicate the priority of each evaluation algorithm selected from multiple evaluation algorithms; a screening module for screening out a first evaluation algorithm that matches the data set from multiple evaluation algorithms based on the gradient process information, the priority of the first evaluation algorithm being higher than the priority of other evaluation algorithms that match the data set in the multiple evaluation algorithms; an execution module for evaluating multiple prompt word templates based on the first evaluation algorithm and the data set to obtain evaluation results corresponding to the multiple prompt word templates respectively; and determining a first prompt word template from the multiple prompt word templates based on the evaluation results corresponding to the multiple prompt word templates respectively, the evaluation result corresponding to the first prompt word template being better than the evaluation results corresponding to other prompt word templates in the multiple prompt word templates.

[0017] In one possible implementation, the gradient process information includes multiple gradients, each gradient includes an identifier of at least one evaluation algorithm, and there are differences in the priority of evaluation algorithms belonging to different gradients; wherein, the priorities of different evaluation algorithms belonging to the same gradient are the same, or, there are differences in the priority of different evaluation algorithms belonging to the same gradient.

[0018] In a possible implementation, the gradient process information is a tree diagram, which includes multiple nodes and multiple directed edges, each of the multiple nodes is used to indicate an evaluation algorithm, and each of the multiple directed edges is used to indicate a priority relationship between different evaluation algorithms.

[0019] In one possible implementation, the multiple gradients include a first gradient and a second gradient, and the priority of the evaluation algorithm belonging to the first gradient is higher than the priority of the evaluation algorithm belonging to the second gradient; the screening module is specifically used to: detect whether the data set includes at least one set of expected output results corresponding to variable values; when the data set does not include the expected output results, determine the first evaluation algorithm from at least one evaluation algorithm belonging to the first gradient; when the data set includes the expected output results, determine the first evaluation algorithm from at least one evaluation algorithm belonging to the second gradient based on the characteristics of the expected output results.

[0020] In a possible implementation, the data set includes multiple groups of variable values, and the screening module is also used to: when the expected output result does not have a feature of a preset type, use a first group of variable values ​​in the multiple groups of variable values ​​to pre-evaluate multiple prompt word templates according to at least one evaluation algorithm belonging to the second gradient to obtain a pre-evaluation result corresponding to each prompt word template in the multiple prompt word templates; and determine a first evaluation algorithm from at least one evaluation algorithm according to the pre-evaluation result corresponding to each prompt word template.

[0021] In a possible implementation, the screening module is specifically configured to: in response to a user's selection operation for automatically setting the control, screen out a first evaluation algorithm that matches the data set from a plurality of evaluation algorithms according to the gradient process information.

[0022] In a possible implementation, the execution module is further used to: when multiple evaluation algorithms do not match the data set, obtain a second evaluation algorithm defined by the user; determine a second prompt word template from multiple prompt word templates based on the second evaluation algorithm and the data set, and the evaluation result corresponding to the second prompt word template is better than the evaluation results corresponding to other prompt word templates in the multiple prompt word templates.

[0023] In a possible implementation, the configuration module is further used to output gradient process information when the user does not perform a selection operation for automatically setting the control; in response to the user's selection operation for multiple evaluation algorithms in the gradient process information, determine the selected evaluation algorithm from the multiple evaluation algorithms as a third evaluation algorithm; the execution module is further used to determine a third prompt word template from multiple prompt word templates based on the third evaluation algorithm and the data set, and the evaluation result corresponding to the third prompt word template is better than the evaluation results corresponding to other prompt word templates in the multiple prompt word templates.

[0024] In a possible implementation, the algorithm management module is further used to: in response to a modification operation on the gradient process information, adjust the priority of the evaluation algorithm selected in the gradient process information; or, add a new evaluation algorithm and the priority of the new evaluation algorithm in the gradient process information.

[0025] It is worth noting that the processing device provided in the second aspect corresponds to the prompt word template processing method provided in the first aspect. Therefore, the technical effects of the second aspect and any implementation method of the second aspect can refer to the technical effects of the first aspect or the corresponding implementation method of the first aspect.

[0026] In a third aspect, the present application provides a prompt word template processing platform, which includes a design device and a processing device in the above-mentioned first aspect or any possible implementation of the first aspect; wherein the design device is used to generate multiple prompt word templates.

[0027] In a fourth aspect, the present application provides a computing device cluster, wherein the computing device includes at least one computing device, and the at least one computing device includes at least one processor and at least one memory; the at least one memory is used to store instructions, and the at least one processor executes the instructions stored in the at least one memory, so that the computing device cluster executes the prompt word template processing method in the above-mentioned first aspect or any possible implementation of the first aspect. It should be noted that the memory can be integrated into the processor or can be independent of the processor. The at least one computing device may also include a bus. The processor is connected to the memory via a bus. The memory may include a readable memory and a random access memory.

[0028] In a fifth aspect, the present application provides a computer-readable storage medium, wherein instructions are stored in the computer-readable storage medium, and when the computer-readable storage medium is executed on at least one computing device, the at least one computing device executes the method described in the first aspect or any one of the implementations of the first aspect.

[0029] In a sixth aspect, the present application provides a computer program product comprising instructions, which, when executed on at least one computing device, enables the at least one computing device to execute the method described in the first aspect or any one of the implementations of the first aspect.

[0030] Based on the implementations provided in the above aspects, the present application can also be further combined to provide more implementations. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present application. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0032] Figure 1 A schematic diagram of the structure of an exemplary prompt word template processing platform provided in this application;

[0033] Figure 2 A flowchart of a prompt word template processing method provided in this application;

[0034] Figure 3 A schematic diagram of a process of generating a prompt word based on a prompt word template and applying the prompt word provided in the present application;

[0035] Figure 4 A schematic diagram of the gradient flow information in the form of a tree diagram provided for this application;

[0036] Figure 5 A schematic diagram of the structure of a computing device provided for this application;

[0037] Figure 6 A schematic diagram of the structure of a computing device cluster provided in this application. DETAILED DESCRIPTION

[0038] The scheme in the embodiments provided in this application will be described below in conjunction with the drawings in this application.

[0039] The terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances. This is just a way of distinguishing objects with the same attributes when describing the embodiments of this application.

[0040] See also Figure 1 , shows a schematic diagram of the structure of an exemplary prompt word template processing platform provided by the present application. Figure 1 As shown, the prompt word template processing platform 10 includes a processing device 100 and a design device 200, and the prompt word template processing platform 100 can provide a client 300 to interact with the user through the client 300, such as receiving information provided by the user through the client 300, and feeding back the processing result to the user through the client 300. The processing device 100, the design device 200 and the client 300 can communicate through the network, so as to realize the cloud service of providing the user with processing prompt word templates.

[0041] The client 300 may be an application program running on a user-side device, or may be a web browser provided externally by the prompt word template processing platform 10 . Figure 1 It is only shown that the prompt word template processing platform 10 includes one client. In actual application, the number of clients included in the prompt word template processing platform 10 can be any number, and there is no limitation on this.

[0042] The design device 200 is used to generate multiple prompt word templates for the same scene. For example, a user can design multiple prompt word templates on the prompt word template processing platform 10 of the design device 200 for an emotion recognition scene, and each prompt word template can be used to generate prompt words related to emotion recognition. Usually, the number of prompt word templates required for a single scene is small, such as only one prompt word template, so the design device 200 can provide the generated multiple prompt word templates to the processing device 100, so that the processing device 100 can determine the prompt word template with the best quality from the multiple prompt word templates as the only prompt word template in the scene.

[0043] The processing device 100 is used to perform quality evaluation on the multiple prompt words using a suitable evaluation algorithm according to the received multiple prompt word templates, so as to determine the prompt word template with the best quality from the multiple prompt word templates. In actual application, there may be multiple scenarios for an application developed by a user, and the user can design multiple prompt word templates for each scenario, so that the processing device 100 can determine the prompt word template with the best quality in each scenario through quality evaluation.

[0044] In the process of determining a high-quality prompt word template, if the user specifies an evaluation algorithm on the prompt word template processing platform 10, since not all evaluation algorithms are suitable for evaluating the quality of multiple prompt word templates in a certain scenario, the user needs to know all evaluation algorithms in advance and select an evaluation algorithm suitable for the scenario to perform quality evaluation on multiple prompt word templates, which will increase the user's technical threshold and increase the user's learning cost for the evaluation algorithm. In actual application scenarios, the user who designs the prompt word template is usually a business person who processes natural language text, and is usually not familiar with the underlying design logic of the evaluation algorithm, which makes it difficult for the user to select a suitable evaluation algorithm. Moreover, when the user selects an evaluation algorithm to perform quality evaluation on the prompt word template without sufficient understanding of the evaluation algorithm, it is easy to cause the evaluation algorithm selected by the user to be unsuitable for the prompt word template in the scenario, thereby resulting in the low accuracy of the high-quality prompt word template determined by the manually selected evaluation algorithm.

[0045] Based on this, in the prompt word template processing platform 10 provided in the present application, the processing device 100 automatically determines a suitable evaluation algorithm, thereby reducing the technical threshold of the user and improving the accuracy of determining a high-quality prompt word template. In specific implementation, the processing device 100 may specifically include a configuration module 101, a data set management module 102, an algorithm management module 103, a screening module 104, an execution module 105, etc. In the process of determining a high-quality prompt word template, the configuration module 101 obtains multiple prompt word templates to be processed, the variables in the multiple prompt word templates may be the same (or the variables in the multiple prompt word templates may be different), and there are differences in the fixed text content other than the variables. In addition, the data set management module 102 will obtain a data set and provide it to the configuration module 101, the data set includes at least one set of variable values, wherein each set of variable values ​​is used to indicate the value of the variable in the multiple prompt word templates. The algorithm management module 103 will obtain gradient process information and provide it to the configuration module 101, the gradient process information is used to indicate the priority of each evaluation algorithm selected in a plurality of evaluation algorithms, and the priority of different evaluation algorithms selected may be different. Then, the configuration module 101 provides the multiple prompt word templates, the data set and the gradient process information to the screening module 104, and the screening module 104 screens out the evaluation algorithm that matches the data set from the multiple evaluation algorithms according to the gradient process information, and the priority of the screened evaluation algorithm is higher than the priority of the remaining evaluation algorithms that match the data set, and then the screening module 104 feeds back the evaluation algorithm to the configuration module 101. In this way, the configuration module 101 instructs the execution module 105 to evaluate the multiple prompt word templates according to the screened evaluation algorithm and data set, obtain the evaluation results corresponding to the multiple prompt word templates, and determine the first prompt word template from the multiple prompt word templates according to the evaluation results corresponding to the multiple prompt word templates, and the evaluation result corresponding to the determined first prompt word template is better than the evaluation results corresponding to the other prompt word templates.

[0046] Since in the process of processing the prompt word template, the processing device 100 can automatically select a suitable evaluation algorithm (that is, an evaluation algorithm that matches the data set) from the multiple evaluation algorithms according to the selected priorities corresponding to the multiple evaluation algorithms indicated in the gradient process information to evaluate the quality of the multiple prompt word templates, there is no need for the user to manually select the evaluation algorithm for the multiple prompt word templates, which can effectively reduce the technical threshold requirements for evaluating high-quality prompt word templates for users, that is, reduce the difficulty of evaluating high-quality prompt word templates. Moreover, the evaluation algorithm automatically selected by the processing device 100 matches the data set, that is, it can adapt to the evaluation process of the prompt word template using the data set, which can ensure that the accuracy of evaluating the prompt word template using the evaluation algorithm can reach a high level, thereby improving the accuracy of the high-quality prompt word template finally determined.

[0047] For example, the above Figure 1 The processing device 100 and the design device 200 can be implemented by software or hardware. The processing device 100 is taken as an example for detailed description.

[0048] As an example of a software functional unit, the processing device 100 may include code running on a computing instance. The computing instance may include at least one of a host, a virtual machine, and a container. Furthermore, the computing instance may be one or more. For example, the processing device 100 may include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the code may be distributed in the same region or in different regions. Furthermore, the multiple hosts / virtual machines / containers used to run the code may be distributed in the same AZ or in different AZs, and each AZ includes a data center or multiple data centers with close geographical locations. Generally, a region may include multiple AZs.

[0049] Similarly, multiple hosts / virtual machines / containers used to run the code can be distributed in the same virtual private cloud (VPC) or in multiple VPCs. Usually, a VPC is set up in a region. For cross-region communication between two VPCs in the same region and between VPCs in different regions, a communication gateway needs to be set up in each VPC to achieve interconnection between VPCs through the communication gateway.

[0050] The processing device 100 is taken as an example of a hardware functional unit, and the processing device 100 may include at least one computing device, such as a server, etc. Alternatively, the processing device 100 may be implemented using a device including an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), a data processing unit (DPU) or any combination thereof.

[0051] It is worth noting that the above Figure 1 The prompt word template processing platform 10 shown is only used as an exemplary description. In actual application, the prompt word template processing platform 10 may also have other possible structures. For example, in other possible implementations, the prompt word template processing platform 10 may also include other devices to support the prompt word template processing platform 10 to have more functions, etc. For example, the prompt word template processing platform 10 may also include an output device so as to output information to the outside by using the output device, specifically, the gradient process information and the prompt word template data finally determined may be output to the client 300 so that the client 300 presents it to the user.

[0052] in addition, Figure 1 The functional module division in the processing device 100 is not limited to Figure 1 In other possible implementations, Figure 1 Some modules in the system can be combined into one module, for example, the configuration module 101 and the execution module 105 can also be combined into one module, then the configuration module 101 can use the selected evaluation algorithm to evaluate multiple prompt word templates and determine the best prompt word template. Or, Figure 1 Some modules in the processing device 100 may also be split into more modules. The present application does not limit the division of the functional modules in the processing device 100.

[0053] Next, various non-limiting specific implementations of the prompt word template processing process are described in detail.

[0054] See also Figure 2 , is a flow chart of a method for processing a prompt word template in an embodiment of the present application. The method can be applied to the above Figure 1 The prompt word template processing platform 10 shown in FIG. 1 may also be applied to other applicable platforms. Figure 1 The prompt word template processing platform 10 shown is used as an example for explanation.

[0055] Figure 2 The prompt word template processing method shown may specifically include:

[0056] S201: The processing device 100 obtains a plurality of prompt word templates provided by the design device 200.

[0057] The prompt word template may include one or more variables and fixed text content. The variables in multiple prompt word templates may be the same, or the variables in multiple prompt word templates may be different. For example, the prompt word template provided by the design device may be as follows: Figure 3As shown, the prompt word template may include two variables, namely variable I and variable II, and "xxx" in the prompt word template is used to represent fixed text content. When the variables in the prompt word template are assigned specific values, a complete text content can be generated based on the prompt word template, that is, a prompt word is obtained. For example, the prompt word template may be "Today's weather is {{variable I}}, and the tourists' mood has also become {{variable II}}". When the value of variable I is "from rain to sunny" and the value of variable II is "beautiful", the prompt word generated based on the prompt word template may be "Today's weather has changed from rain to sunny, and the tourists' mood will become beautiful". When the variables in the prompt word template are assigned different values, different prompt words can be generated based on the prompt word template. For example, Figure 3 As shown, when the value of variable I is "A" and the value of variable II is "B", the prompt word "xxxAxxxBxx" can be generated; when the value of variable I is "C" and the value of variable II is "D", the prompt word "xxxCxxxDxx" can be generated.

[0058] In this embodiment, the design device 200 can create multiple prompt word templates with the same variables for the same scene.

[0059] As an implementation example, the user can use the design device 200 to create multiple prompt word templates. In specific implementation, the design device 200 can create a design task and present a design interface to the user through the client 300, so that the user can design one or more prompt word templates on the design interface. The user can define the application scenario to which the prompt word template to be designed belongs on the design interface, and write the fixed text content in the prompt word template, and define at least one variable and the location of the variable in the prompt word template. In this way, the design device 200 can create a corresponding prompt word template according to the operation performed by the user on the design interface. Furthermore, the user can also modify the fixed text content in the prompt word template on the design interface, so that the design device 200 can generate multiple prompt word templates according to the fixed text content modified by the user, and the multiple prompt word templates have the same variables and different fixed text contents.

[0060] In addition to the above implementation examples, in actual application, the user can also use the client 300 to directly provide the design device 200 with multiple prompt word templates that have been created, and this embodiment does not limit this.

[0061] Since not all prompt word templates among the multiple prompt word templates created (or received) by the design device 200 have a high quality, the design device 200 may send the multiple prompt word templates to the processing device 100 to request the processing device 100 to select high-quality prompt word templates therefrom.

[0062] In this embodiment, the operations performed by the processing device 100 may specifically be performed by the configuration module 101, the data set management module 102, the algorithm management module 103, the screening module 104, and the execution module 105 included in the processing device 100, which is described in detail below.

[0063] S202: The configuration module 101 instructs the data set management module 102 to feed back the data set required for evaluating the prompt word template.

[0064] In this embodiment, the data set management module 102 is responsible for unified management of the data set. The data set includes at least one set of variable values, each set of variable values ​​is used to indicate the values ​​of the variables in the multiple prompt word templates. In actual application, the data set may also include the variables in the prompt word template.

[0065] For example, taking the prompt word template including variable I and variable II as an example, the structure of the data set managed by the data set management module 102 can be shown in Table 1 below.

[0066] Table 1

[0067]

[0068] Furthermore, if Figure 3 As shown, the prompt words generated based on the prompt word template are usually used as the input required for model reasoning, and the prompt words generated based on the high-quality prompt word template will make the model output have a higher quality, specifically, the actual output result of the model matches the expected model output result (hereinafter referred to as the expected output result). Therefore, the dataset managed by the dataset management module 102 can also include the expected output result, so that when the matching degree between the expected output result and the actual output result of the model is high, it can be determined that the quality of the model output is high. At this time, the structure of the dataset managed by the dataset management module 102 can be shown in Table 2 below.

[0069] Table 2

[0070]

[0071] Exemplarily, the model for inference based on the prompt words may be, for example, a sentiment recognition model, an intelligent response model, a large language model, etc., or may be other types of models, which are not limited in this embodiment.

[0072] In a possible implementation, when the user designs or provides multiple prompt word templates on the prompt word template processing platform 10 through the client 300, one or more groups of variable values ​​can also be defined on the prompt word template processing platform 10 for the variables in the prompt word template, so as to construct a data set. In specific implementation, the data set management module 102 can present a configuration interface for the data set to the user through the client 300, so that the user can define at least one group of variable values ​​for the variables in the prompt word template on the configuration interface, and each group of variable values ​​includes the values ​​of all variables in the prompt word template. In this way, the data set management module 102 can generate a corresponding data set according to the user's operation on the configuration interface, and can be saved locally. Alternatively, the user can send the generated data set to the data set management module 102 through the client 300, etc., and this embodiment does not limit this. Exemplarily, the prompt word template and the data set defined by the user on the prompt word template processing platform 10 can be mapped through the identifier of the design task created by the design device 200, so that the prompt word template processing platform 10 can use the user-defined data set to perform quality assessment on the prompt word template defined by the user according to the identifier.

[0073] After receiving multiple prompt word templates sent by the design device 200, the configuration module 101 can request a data set from the data set management module 102, so that the data set management module 102 can parse the identifier of the design task from the received request and traverse the data set with the identifier from the locally stored data set. Then, the data set management module 102 can feed back the traversed data set to the configuration module 101 to meet the demand for the data set when processing the prompt word template.

[0074] S203: The configuration module 101 instructs the algorithm management module 103 to feed back gradient process information, where the gradient process information is used to indicate the priority of each evaluation algorithm selected from the multiple evaluation algorithms.

[0075] In this embodiment, in the process of evaluating the quality of the prompt word template, it is necessary to use a suitable evaluation algorithm for evaluation. Since not all prompt word templates in actual application scenarios are applicable to one evaluation algorithm, the configuration module 101 can obtain multiple possible evaluation algorithms and select a suitable evaluation algorithm from the multiple evaluation algorithms to evaluate the quality of the prompt word template.

[0076] The algorithm management module 103 is responsible for managing various evaluation algorithms.

[0077] The evaluation algorithms managed by the algorithm management module 103 may be provided by the user. For example, the algorithm management module 103 may present an algorithm configuration interface to the user so that the user may provide multiple evaluation algorithms to the algorithm management module 103 on the algorithm configuration interface. Exemplarily, the evaluation algorithms provided by the user may be, for example, a model evaluation algorithm, a model evaluation-benchmark comparison (battle) algorithm, a rule evaluation algorithm, a rule evaluation-similarity algorithm, etc., or may be other types of algorithms.

[0078] The model evaluation algorithm refers to the use of model B (i.e., other models) to directly score the results output by model A after the prompt word is input into model A, so as to evaluate whether the template that generates the prompt word is a high-quality prompt word template.

[0079] The model evaluation-benchmark comparison algorithm means that after the prompt word is input into model A, model B is used to compare the difference between the output of model A and the benchmark data and give a score to evaluate the quality of the template that generates the prompt word.

[0080] The rule evaluation algorithm refers to the method of using rule matching to verify whether the result output by model A is one of the multiple expected results after the prompt word is input into model A, so as to evaluate the quality of the template that generates the prompt word. For example, the multiple expected results are "negative", "positive", and "normal". Based on the rule evaluation algorithm, it can be verified whether the result output by model A is one of "negative", "positive", and "normal". If so, it can be determined that the quality of the template that generates the prompt word is high, otherwise, it is determined that the quality of the prompt word template is low.

[0081] The rule evaluation-similarity algorithm refers to scoring by calculating the data similarity between the result output by model A and the expected result after the prompt word is input into model A. Generally speaking, the higher the data similarity, the higher the quality of the template generating the prompt word; conversely, the lower the data similarity, the lower the quality of the template generating the prompt word.

[0082] Furthermore, the algorithm management module 103 can also support users to customize new evaluation algorithms on the algorithm configuration interface, such as defining the name of the evaluation algorithm, applicable application scenarios, extracted / selected features, code implementation of the evaluation algorithm, input parameters, and output parameters. In actual application, users can also provide an introduction to the evaluation algorithm to enrich the information of the customized evaluation algorithm; and when the evaluation algorithm is selected by pre-evaluation, the customized evaluation algorithm can also include a pre-evaluation ratio, a scoring threshold value corresponding to the evaluation algorithm, etc., wherein the pre-evaluation ratio can be used to limit the amount of part of the data in the data set when the quality of the prompt word template is pre-evaluated, and the scoring threshold value can be used to limit the minimum score required to evaluate the quality of the prompt word template using the evaluation algorithm.

[0083] In this way, the algorithm management module 103 can obtain a variety of evaluation algorithms according to the user's operation on the algorithm configuration interface, and can save them locally.

[0084] In other implementations, the evaluation algorithm managed by the algorithm management module 103 may also be pre-configured in the algorithm management module 103 by a technician, and this embodiment does not limit this.

[0085] In this embodiment, when managing multiple evaluation algorithms, the algorithm management module 103 can add a priority mark to the evaluation algorithm, and the mark is used to indicate the priority of the evaluation algorithm selected for evaluating the quality of the prompt word template. Accordingly, when selecting the evaluation algorithm later, the evaluation algorithm with a higher priority can be selected to verify whether it is suitable for quality evaluation of the prompt word template.

[0086] Then, the algorithm management module 103 can generate gradient process information according to the identification of the priority of each evaluation algorithm, and the gradient process information includes multiple gradients, each gradient includes the identification of at least one evaluation algorithm, and the priority of the evaluation algorithms belonging to different gradients is different. For example, it is assumed that the gradient process information includes 3 gradients, namely the first gradient, the second gradient and the third gradient. Among them, the evaluation algorithm belonging to the first gradient has the highest priority, and the evaluation algorithm belonging to the third gradient has the lowest priority. In this way, the evaluation algorithm belonging to the first gradient can be preferentially selected to determine whether it is used for quality evaluation of the prompt word template; and when the evaluation algorithm is not suitable for evaluating the quality of the prompt word template, the evaluation algorithm belonging to the second gradient is selected for judgment, and so on.

[0087] The priorities of different evaluation algorithms belonging to the same gradient may be the same, or the priorities of different evaluation algorithms belonging to the same gradient may be different.

[0088] Exemplarily, the generated gradient process information can be a tree diagram, which includes multiple nodes and multiple directed edges. Each node is used to indicate an evaluation algorithm, and each directed edge is used to indicate the priority relationship between different nodes. For example, when the direction of the directed edge between node A and node B is "node A->node B", the priority of the evaluation algorithm indicated by node A (as a parent node) is higher than the priority of the evaluation algorithm indicated by node B (as a child node). When multiple child nodes are included under the same parent node, the priority of the evaluation algorithms indicated by the multiple child nodes is reduced in order from left to right, or the priority of the evaluation algorithms indicated by the multiple child nodes is the same. In actual application, the algorithm management module 103 uses a tree diagram to manage multiple evaluation algorithms, which can reduce management difficulty and improve management efficiency.

[0089] For example, assuming that the evaluation algorithms in the prompt word template processing platform 10 include a model evaluation algorithm, a model evaluation-benchmark comparison algorithm, a rule evaluation algorithm, and a rule evaluation-similarity algorithm, the algorithm management module 103 may generate the following: Figure 4 The tree diagram shown. Node A in the first gradient is used to indicate the model evaluation algorithm, node B in the second gradient is used to indicate the rule evaluation algorithm, node C in the third gradient is used to indicate the model evaluation-benchmark comparison algorithm, and node D in the third gradient is used to indicate the rule evaluation-similarity algorithm, and the priority of the evaluation algorithms indicated by nodes C and D can be the same or different.

[0090] After receiving the multiple prompt word templates sent by the design device 200, the configuration module 101 may send a request for an evaluation algorithm to the algorithm management module 103. Thus, the algorithm management module 103 may respond to the request and send the locally stored gradient flow information (and multiple evaluation algorithms) to the processing device 100, so as to subsequently select appropriate evaluation algorithms for the multiple prompt word templates based on the gradient flow information.

[0091] S204: the configuration module 101 sends the plurality of prompt word templates, the data set, and the gradient process information to the screening module 104 to instruct the screening module 104 to determine a suitable evaluation algorithm for the plurality of prompt word templates.

[0092] S205: The screening module 104 obtains multiple evaluation algorithms, and screens out a first evaluation algorithm that matches the data set from the multiple evaluation algorithms according to the gradient process information, wherein the priority of the first evaluation algorithm is higher than the priority of other evaluation algorithms that match the data set from the multiple evaluation algorithms.

[0093] Exemplarily, the evaluation algorithm may be managed by the algorithm management module 103, so the screening module 104 may obtain a plurality of evaluation algorithms by accessing the algorithm management module 103. Alternatively, when the algorithm management module 103 feeds back the gradient process information to the configuration module 101, the plurality of evaluation algorithms may be sent to the configuration module 101 together, so that when the configuration module 101 instructs the screening module 104 to determine a suitable evaluation algorithm for the plurality of prompt word templates, the plurality of evaluation algorithms may be sent to the screening module 104.

[0094] Since the gradient process information indicates the different priorities of different evaluation algorithms, the screening module 104 can test whether the evaluation algorithm with the highest priority is suitable for evaluating the quality of the prompt word template according to the priority of the evaluation algorithm. If it is determined that the evaluation algorithm is suitable for evaluating the quality of the prompt word template, the screening module 104 can directly determine the evaluation algorithm as the first evaluation algorithm, and then directly use the first evaluation algorithm to evaluate the quality of the prompt word template, without having to judge the remaining evaluation algorithms. If it is determined that the evaluation algorithm is not suitable for evaluating the quality of the prompt word template, the screening module 104 can continue to select the evaluation algorithm with the highest priority from the remaining multiple evaluation algorithms for judgment. Repeat the above process until an evaluation algorithm suitable for evaluating the quality of the prompt word template is determined, or the judgment of the last evaluation algorithm is completed.

[0095] In a possible implementation, the screening module 104 determines an evaluation algorithm that matches the data set among the multiple evaluation algorithms as a first evaluation algorithm that is ultimately used to evaluate the quality of the prompt word template.

[0096] In specific implementation, the gradient process information includes multiple gradients, taking the gradient process information including the first gradient and the second gradient as an example, and the priority of the evaluation algorithm belonging to the first gradient is higher than the priority of the evaluation algorithm belonging to the second gradient. Then, the screening module 104 can first detect whether each group of variable values ​​in the data set has the expected output result corresponding thereto. When the data set does not include the expected output result, the screening module 104 can determine the first evaluation algorithm from at least one evaluation algorithm belonging to the first gradient. At this time, the determined first evaluation algorithm is specifically an algorithm that can evaluate the quality of the prompt word template without the aid of the expected output result, such as the above-mentioned model evaluation algorithm. When the first gradient includes multiple model evaluation algorithms, the screening module 104 can select the model evaluation algorithm with the highest ranking as the above-mentioned first evaluation algorithm according to the arrangement order of the multiple model evaluation algorithms in the gradient process information; or the screening module 104 can randomly (or in other ways) select a model evaluation algorithm from the multiple model evaluation algorithms as the above-mentioned first evaluation algorithm.

[0097] When the data set includes an expected output result, the screening module 104 can determine a first evaluation algorithm from at least one evaluation algorithm belonging to the second gradient according to the characteristics of the expected output result. Exemplarily, the characteristics of the expected output result can be, for example, that the expected output result includes multiple fixed values. For example, the data set includes 100 groups of variable values, and the expected output result corresponding to each group of variable values ​​in the 100 groups of variable values ​​is one of the three values ​​of "positive", "negative" and "normal". At this time, the screening module 104 can be the first evaluation algorithm determined from at least one evaluation algorithm belonging to the second gradient, which can specifically be the above-mentioned rule evaluation algorithm, etc. It can be understood that when the expected output result expected in the data set has other types of characteristics, the first evaluation algorithm determined by the screening module 104 from at least one evaluation algorithm belonging to the second gradient can also be other types of algorithms.

[0098] In addition, when the data set includes an expected output result, but the expected output result does not have characteristics of a preset type, the screening module 104 can determine the first evaluation algorithm from the evaluation algorithms belonging to the second gradient, or from the remaining evaluation algorithms (such as the evaluation algorithms belonging to the third gradient, etc.).

[0099] Exemplarily, for each evaluation algorithm belonging to the second gradient, the screening module 104 can pre-evaluate multiple prompt word templates to obtain the pre-evaluation results corresponding to each prompt word template. For example, the screening module 104 can select part of the variable values ​​from the data set, and for each prompt word template, the screening module 104 can generate prompt words according to the part of the variable values ​​and the prompt word template, and input the prompt words into the model to obtain the actual output result of the model. Then, the screening module 104 can determine the matching degree between the actual output result and the expected output result in the data set, so as to score the prompt word template according to the matching degree. In this way, for each prompt word template, the score determined based on the evaluation algorithm can be obtained. Finally, the screening module 104 can determine the prompt word template with the highest score according to the scoring results of multiple prompt word templates corresponding to each evaluation algorithm, and determine the evaluation algorithm adopted for scoring the prompt word template as the first evaluation algorithm. In actual application, the screening module 104 can also determine the first evaluation algorithm by other pre-evaluation methods, and this embodiment does not limit this.

[0100] After determining the first evaluation algorithm, the screening module 104 may feed back indication information of the first evaluation algorithm to the configuration module 101 . The indication information may be, for example, the name of the first evaluation algorithm.

[0101] S206: The configuration module 101 sends the plurality of prompt word templates, the data set, and the first evaluation algorithm to the execution module 105 to instruct the execution module 105 to evaluate the quality of the plurality of prompt word templates.

[0102] In other embodiments, the configuration module 101 may also send the identifier of the first evaluation algorithm to the execution module 105 , so that the execution module 105 may obtain the first evaluation algorithm from a plurality of evaluation algorithms locally stored in the algorithm management module 103 according to the identifier.

[0103] S207: The execution module 105 evaluates the plurality of prompt word templates according to the first evaluation algorithm and the data set, and obtains evaluation results corresponding to the plurality of prompt word templates.

[0104] S208: The execution module 105 determines a first prompt word template from the plurality of prompt word templates according to the evaluation results respectively corresponding to the plurality of prompt word templates, wherein the evaluation result of the first prompt word template is better than the evaluation results corresponding to the other prompt word templates.

[0105] In a possible implementation, for each prompt word template, the execution module 105 can use multiple groups of variable values ​​in the data set to assign values ​​to the variables in the prompt word template to generate multiple corresponding prompt words. Then, the execution module 105 can input the multiple prompt words into the model respectively, and the model can infer each prompt word one by one to obtain the actual output result corresponding to each prompt word, so that the execution module 105 can use the evaluation algorithm to compare the actual output result with the expected output result. In this way, the execution module 105 can obtain the comparison result corresponding to each prompt word, and perform quality evaluation on the prompt word template according to the comparison result corresponding to each prompt word. For example, when the difference between the actual output result corresponding to all prompt words and the expected output result is small, it indicates that the versatility of the prompt word template is high, that is, the quality of the prompt word template is high, then, the execution module 105 determines that the prompt word template used to generate the prompt word has a high score; otherwise, the prompt word template has a low score. Among them, the score of the prompt word template is the evaluation result of the execution module 105 for the prompt word template.

[0106] Referring to the above embodiment, the execution module 105 can use the first evaluation algorithm to determine the score corresponding to each prompt word template, so that the execution module 105 can determine the prompt word template with the highest score from multiple prompt word templates, which is referred to as the first prompt word template for the convenience of distinction and description. In this way, the determined first prompt word template is the prompt word template with the highest quality among the multiple prompt word templates provided by the user or the design device 200.

[0107] It can be understood that in the above implementation, the evaluation result of the quality of each prompt word template by the execution module 105 is specifically a score. In other possible implementations, the evaluation result may also be other information that can be used to characterize the low quality of the prompt word template, which is not limited in this embodiment.

[0108] After determining the first prompt word template, the execution module 105 may feed back indication information of the first prompt word template to the configuration module 101 . The indication information may be, for example, the name of the first prompt word template.

[0109] S209: The configuration module 101 outputs a first prompt word template.

[0110] In specific implementation, the configuration module 101 may send the first prompt word to the client 300, so as to present the first prompt word template to the user through the client 300. In this way, the user can know which prompt word template has the best quality among the multiple prompt word templates currently designed.

[0111] In this way, in the process of determining a high-quality prompt word template, the processing device 100 can automatically determine a suitable evaluation algorithm without the need for manual specification by the user, which can effectively reduce the technical threshold requirements for evaluating the quality of the prompt word template for the user, that is, reduce the difficulty of evaluating a high-quality prompt word template. Moreover, the first evaluation algorithm automatically screened by the processing device 100 matches the data set, which can ensure that the accuracy of evaluating the prompt word template using the first evaluation algorithm can reach a high level, thereby improving the accuracy of determining a high-quality prompt word template. For example, when the expected output result in the data set is a fixed value, the processing device 100 can determine a high-quality prompt word template through an accurate matching evaluation algorithm; and when the expected output result in the data set is not a fixed value, the processing device 100 can determine a high-quality prompt word template through a fuzzy matching evaluation algorithm, thereby improving the accuracy of determining a high-quality prompt word template.

[0112] In addition, the configuration module 101 may also output intermediate data generated in the process of determining the first prompt word template. For example, when the configuration module 101 presents the first prompt word template through the client 300, it may also present the results of evaluating each prompt word template using the first evaluation algorithm, etc., so as to increase the user's confidence that the first prompt word template is a high-quality prompt word template and improve the user experience.

[0113] In actual application, the screening module 104 may also determine that the multiple evaluation algorithms in the gradient process information do not match the data set. For example, when the highest score of the prompt word template determined based on part of the data in the data set is greater than or equal to a preset score threshold, the screening module 104 will determine the evaluation algorithm used to score the prompt word template as the first evaluation algorithm. When the highest score of the prompt word template is less than the score threshold, the screening module 104 can determine that the multiple evaluation algorithms recorded in the gradient process information are not suitable for quality evaluation of multiple prompt word templates.

[0114] If the configuration module 101 receives the feedback of the screening module 104 and there is no suitable evaluation algorithm, it can be notified to the user through the client 300, and the user is prompted to define the evaluation algorithm. In this way, the user can define a new evaluation algorithm on the algorithm configuration interface presented by the client 300, which is referred to as the first evaluation algorithm for the convenience of distinction and description. Then, the configuration module 101 can obtain the second evaluation algorithm fed back by the client 300, and provide the second evaluation algorithm, multiple prompt word templates and data sets to the execution module 105, and the execution module 105 determines the second prompt word template from multiple prompt word templates according to the second evaluation algorithm and data sets, and the evaluation result corresponding to the determined second prompt word template is better than the evaluation result corresponding to other prompt word templates. Wherein, the execution module 105 uses the second evaluation algorithm to determine the specific implementation process of the second prompt word template, which can refer to the description of the relevant part of the above-mentioned determination of the first prompt word template by the first evaluation algorithm, and will not be repeated here.

[0115] As an implementation example, before performing quality assessment on multiple prompt word templates, the configuration module 101 may also send the acquired gradient process information to the client 300 so that the client 300 can present it to the user. Exemplarily, the client 300 may present a selection interface to the user, and prompt the user to select an assessment algorithm on the selection interface. The selection interface may be configured with a control to assist in setting the process. If the user does not click on the control, the selection interface may present multiple assessment algorithms in parallel in the form of a list, so that the user can select an assessment algorithm from the multiple assessment algorithms to assess the quality of the prompt word template.

[0116] When the user clicks on the control, the gradient process information can be further presented on the selection interface, such as a pop-up window showing a tree diagram corresponding to the gradient process information, so as to guide the user to select an evaluation algorithm using the gradient process information. For example, under the guidance of the gradient process information, the user can select the evaluation algorithm with the highest priority to evaluate the quality of the prompt word template, or guide the user to select the evaluation algorithm according to the priority level indicated by the gradient process information, so as to reduce the user's learning cost for the evaluation algorithm, and also reduce the technical threshold for the user to select the evaluation algorithm. Accordingly, the processing device 100 can respond to the user's selection operation for multiple evaluation algorithms in the gradient process information, and determine the selected evaluation algorithm, which is referred to as the third evaluation algorithm below for the convenience of distinction and description. In addition, the processing device 100 can determine the third prompt word template from multiple prompt word templates according to the third evaluation algorithm and the data set, and the evaluation result corresponding to the third prompt word template is better than the evaluation result corresponding to other prompt word templates. Among them, the specific implementation process of the processing device 100 determining the third prompt word template using the third evaluation algorithm can be referred to the above-mentioned related description, which will not be repeated here.

[0117] Furthermore, the selection interface may also be provided with an automatic setting control. When the user clicks the control, the processing device 100 may perform the above Figure 2 The process shown automatically selects a first evaluation algorithm from multiple evaluation algorithms to evaluate the quality of the prompt word template, thereby further lowering the technical threshold for users to determine the evaluation algorithm.

[0118] In addition, the processing device 100 can also support dynamic modification of locally stored gradient process information. For example, the algorithm management module 103 can present an operation and maintenance interface for the gradient process information to the administrator (such as an algorithm personnel, etc.) through the client 300, so that the administrator can visually edit the gradient process information on the operation and maintenance interface, such as adjusting the priority information of some evaluation algorithms (such as adjusting the position of some nodes in the tree diagram, etc.), adding a new evaluation algorithm and the priority of the evaluation algorithm in the gradient process information, etc. Correspondingly, the algorithm management module 103 can adjust the priority of the evaluation algorithm selected in the gradient process information according to the modification operation performed by the administrator on the operation and maintenance interface for the gradient process information, or add a new evaluation algorithm and the priority of the newly added evaluation algorithm in the gradient process information, or delete some evaluation algorithms and their priorities in the gradient process information, etc., so as to realize the dynamic update of the gradient process information.

[0119] It is worth noting that the above Figure 2The method steps and the execution order between them in the illustrated embodiment are only used as an implementation example and are not intended to be limiting. In other possible embodiments, different method steps may have different execution orders. For example, step S202 and step S203 may be executed in parallel, or step S203 may be executed first and then step S202 may be acquired. Alternatively, in other possible embodiments, there may be more or fewer method steps. Furthermore, the data transmission method between different modules in the processing device 100 is not limited to the same method. Figure 2 For example, in other embodiments, the configuration module 101 may also only send multiple prompt word templates and gradient process information to the screening module 104, and the screening module 104 obtains the data set from the data set management module 102, etc. This embodiment does not limit this.

[0120] Above Figure 2 In the illustrated embodiment, the processing device 100 involved in the prompt word template processing process may be software configured on a computing device or a computing device cluster, and by running the software on the computing device or the computing device cluster, the computing device or the computing device cluster may implement the functions of the processing device 100. Below, based on the perspective of hardware device implementation, the processing device 100 involved in the prompt word template processing process is described in detail.

[0121] Figure 5 The schematic diagram of the structure of a computing device is shown. The processing device 100 can be deployed on the computing device. The computing device can be a computing device in a cloud environment (such as a server) or a computing device in an edge environment, etc., which can be used to implement the above Figure 2 The functionality of the processing device 100 in the illustrated embodiment.

[0122] like Figure 5 As shown, computing device 500 includes processor 510, memory 520, communication interface 530 and bus 540. Processor 510, memory 520 and communication interface 530 communicate through bus 540. Bus 540 can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 The communication interface 530 is used to communicate with the outside, for example, to receive multiple prompt word templates, data sets, and customized evaluation algorithms provided by users.

[0123] Among them, the processor 510 can be a central processing unit (CPU), an application specific integrated circuit (ASIC), a graphics processing unit (GPU) or one or more integrated circuits. The processor 510 can also be an integrated circuit chip with signal processing capabilities. In the implementation process, the functions of the processing device 100 can be completed by the hardware integrated logic circuit or software instructions in the processor 510. The processor 510 can also be a general processor, a digital signal process (DSP), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, and can implement or execute the methods, steps and logic block diagrams disclosed in the embodiments of the present application. Among them, the general processor can be a microprocessor or the processor can also be any conventional processor, etc., and the method disclosed in the embodiments of the present application can be directly embodied as a hardware decoding processor to execute, or it can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 520 , and the processor 510 reads the information in the memory 520 and completes part or all of the functions in the processing device 100 in combination with its hardware.

[0124] The memory 520 may include a volatile memory, such as a random access memory (RAM), or a non-volatile memory, such as a read-only memory (ROM), a flash memory, a HDD, or a SSD.

[0125] The memory 520 stores executable codes, and the processor 510 executes the executable codes to execute the method executed by the aforementioned processing device 100 .

[0126] Specifically, in implementing Figure 2 In the case of the embodiment shown, and Figure 2 In the case where the processing device 100 described in the illustrated embodiment is implemented by software, the execution Figure 2The software or program code required for the functions of the processing device 100 is stored in the memory 520. The interaction between the processing device 100 and other devices is realized through the communication interface 530. The processor is used to execute the instructions in the memory 520 to implement the method executed by the processing device 100.

[0127] Figure 6 A schematic diagram of the structure of a computing device cluster is shown. Figure 6 The computing device cluster 60 shown includes multiple computing devices, and the processing device 100 can be deployed on multiple computing devices in the computing device cluster 60 in a distributed manner. Figure 6 As shown, the computing device cluster 60 includes multiple computing devices 600 , each computing device 600 includes a memory 620 , a processor 610 , a communication interface 630 and a bus 640 , wherein the memory 620 , the processor 610 , and the communication interface 630 are communicatively connected to each other via the bus 640 .

[0128] The processor 610 may be a CPU, a GPU, an ASIC, or one or more integrated circuits. The processor 610 may also be an integrated circuit chip with signal processing capabilities. In the implementation process, some functions of the processing device 100 may be completed by the hardware integrated logic circuit or software instructions in the processor 610. The processor 610 may also be a DSP, an FPGA, a general-purpose processor, other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, and may implement or execute some methods, steps, and logic block diagrams disclosed in the embodiments of the present application. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc., and the steps of the method disclosed in the embodiments of the present application may be directly embodied as a hardware decoding processor for execution, or may be executed by a combination of hardware and software modules in the decoding processor. The software module may be located in a mature storage medium in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 620. In each computing device 600, the processor 610 reads the information in the memory 620, and in combination with its hardware, some functions of the processing device 100 may be completed.

[0129] The memory 620 may include ROM, RAM, static storage device, dynamic storage device, hard disk (such as SSD, HDD), etc. The memory 620 may store program codes, for example, part or all of the program codes for implementing the configuration module 101, part or all of the program codes for implementing the data set management module 102, part or all of the program codes for implementing the algorithm management module 103, part or all of the program codes for implementing the screening module 104, part or all of the program codes for implementing the execution module 105, etc. For each computing device 600, when the program codes stored in the memory 620 are executed by the processor 610, the processor 610 executes part of the method executed by the processing device 100 based on the communication interface 630, such as a part of the computing devices 600 may be used to execute the methods executed by the above-mentioned configuration module 101, the data set management module 102, and the algorithm management module 103, and another part of the computing devices 600 is used to execute the methods executed by the above-mentioned screening module 104 and the execution module 105. The memory 620 may also store data, for example, intermediate data or result data generated during the execution of the processor 610, for example, the first evaluation algorithm, the first prompt word template, and evaluation results corresponding to the plurality of prompt word templates.

[0130] The communication interface 603 in each computing device 600 is used for external communication, such as interacting with other computing devices 600 .

[0131] The bus 640 may be a peripheral component interconnect standard bus or an extended industry standard architecture bus. For ease of representation, Figure 6 The bus 640 in each computing device 600 is represented by only one thick line, but this does not mean that there is only one bus or one type of bus.

[0132] The plurality of computing devices 600 establish communication paths through a communication network to implement the functions of the processing apparatus 100. Any computing device may be a computing device (eg, a server) in a cloud environment, or a computing device in an edge environment.

[0133] In addition, an embodiment of the present application also provides a computer-readable storage medium, which stores instructions. When the computer-readable storage medium is run on one or more computing devices, the one or more computing devices execute the method executed by the processing device 100 of the above embodiment.

[0134] In addition, the embodiment of the present application also provides a computer program product, when the computer program product is executed by one or more computing devices, the one or more computing devices execute any of the aforementioned prompt word template processing methods. The computer program product can be a software installation package, and when any of the aforementioned prompt word template processing methods is needed, the computer program product can be downloaded and executed on a computer.

[0135] It should also be noted that the embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment. In addition, in the embodiment drawings provided by the present application, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines.

[0136] Through the description of the above implementation mode, the technicians in the field can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course, it can also be implemented by special hardware including special integrated circuits, special CPUs, special memories, special components, etc. In general, all functions completed by computer programs can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be various, such as analog circuits, digital circuits or special circuits. However, for the present application, software program implementation is a better implementation mode in more cases. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer floppy disk, a U disk, a mobile hard disk, a ROM, a RAM, a disk or an optical disk, etc., including a number of instructions to enable a computer device (which can be a personal computer, a training device, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0137] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0138] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, a computer, a training device, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, training device, or data center. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device, a data center, etc. that includes one or more available media integrations. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)), etc.

Claims

1. A prompt word template processing method, characterized in that: The method comprises: Acquire a plurality of prompt word templates and a data set, wherein the data set includes at least one set of variable values, and each set of variable values ​​is used to indicate the values ​​of the variables in the plurality of prompt word templates; Acquire gradient process information, where the gradient process information is used to indicate a priority of each evaluation algorithm selected among multiple evaluation algorithms; According to the gradient process information, a first evaluation algorithm matching the data set is selected from the multiple evaluation algorithms, wherein a priority of the first evaluation algorithm is higher than a priority of other evaluation algorithms matching the data set from the multiple evaluation algorithms; According to the first evaluation algorithm and the data set, the plurality of prompt word templates are evaluated to obtain evaluation results respectively corresponding to the plurality of prompt word templates; According to the evaluation results respectively corresponding to the multiple prompt word templates, a first prompt word template is determined from the multiple prompt word templates, wherein the evaluation result corresponding to the first prompt word template is better than the evaluation results corresponding to other prompt word templates in the multiple prompt word templates.

2. The method according to claim 1, characterized in that The gradient process information includes multiple gradients, each gradient includes an identifier of at least one evaluation algorithm, and the priority of evaluation algorithms belonging to different gradients is different; Among them, different evaluation algorithms belonging to the same gradient have the same priority, or different evaluation algorithms belonging to the same gradient have different priority.

3. The method according to claim 2, characterized in that The gradient process information is a tree diagram, which includes multiple nodes and multiple directed edges. Each of the multiple nodes is used to indicate an evaluation algorithm, and each of the multiple directed edges is used to indicate a priority relationship between different evaluation algorithms.

4. The method according to claim 2 or 3, characterized in that: The plurality of gradients include a first gradient and a second gradient, and a priority of an evaluation algorithm belonging to the first gradient is higher than a priority of an evaluation algorithm belonging to the second gradient; The step of selecting a first evaluation algorithm matching the data set from the plurality of evaluation algorithms according to the gradient process information includes: Detecting whether the data set includes the expected output result corresponding to the at least one set of variable values; When the data set does not include the expected output result, determining the first evaluation algorithm from at least one evaluation algorithm belonging to the first gradient; When the data set includes the expected output result, the first evaluation algorithm is determined from at least one evaluation algorithm belonging to the second gradient according to the characteristics of the expected output result.

5. The method according to claim 4, characterized in that The data set includes a plurality of sets of variable values, and the method further includes: When the expected output result does not have a feature of a preset type, using a first group of variable values ​​among the multiple groups of variable values, pre-evaluating the multiple prompt word templates according to the at least one evaluation algorithm belonging to the second gradient, to obtain a pre-evaluation result corresponding to each prompt word template among the multiple prompt word templates; The first evaluation algorithm is determined from the at least one evaluation algorithm according to the pre-evaluation result corresponding to each prompt word template.

6. The method according to any one of claims 1 to 5, characterized in that: The step of selecting a first evaluation algorithm matching the data set from the plurality of evaluation algorithms according to the gradient process information includes: In response to a user's selection operation for automatically setting a control, a first evaluation algorithm matching the data set is screened out from the multiple evaluation algorithms according to the gradient process information.

7. The method according to claim 6, characterized in that The method further comprises: When the multiple evaluation algorithms do not match the data set, obtaining the user-defined second evaluation algorithm; According to the second evaluation algorithm and the data set, a second prompt word template is determined from the multiple prompt word templates, and an evaluation result corresponding to the second prompt word template is better than the evaluation results corresponding to other prompt word templates in the multiple prompt word templates.

8. The method according to claim 6 or 7, characterized in that: The method further comprises: When the user does not perform a selection operation on the automatic setting control, outputting the gradient process information; In response to a user's selection operation on multiple evaluation algorithms in the gradient process information, determining a selected evaluation algorithm from the multiple evaluation algorithms as a third evaluation algorithm; A third prompt word template is determined from the plurality of prompt word templates according to the third evaluation algorithm and the data set, wherein an evaluation result corresponding to the third prompt word template is better than evaluation results corresponding to other prompt word templates in the plurality of prompt word templates.

9. The method according to any one of claims 1 to 8, characterized in that: The method further comprises: In response to a modification operation on the gradient process information, a priority of an evaluation algorithm in the gradient process information is adjusted, or an evaluation algorithm and a priority of the newly added evaluation algorithm are added to the gradient process information.

10. A processing device, characterized in that: The processing device comprises: A configuration module is used to obtain multiple prompt word templates; A data set management module, used to obtain a data set, wherein the data set includes at least one set of variable values, each set of variable values ​​is used to indicate the values ​​of the variables in the plurality of prompt word templates; An algorithm management module, used to obtain gradient process information, wherein the gradient process information is used to indicate the priority of each evaluation algorithm selected among multiple evaluation algorithms; A screening module, configured to screen out a first evaluation algorithm that matches the data set from the multiple evaluation algorithms according to the gradient process information, wherein the priority of the first evaluation algorithm is higher than the priority of other evaluation algorithms that match the data set from the multiple evaluation algorithms; an execution module, configured to evaluate the plurality of prompt word templates according to the first evaluation algorithm and the data set to obtain evaluation results respectively corresponding to the plurality of prompt word templates; and to determine a first prompt word template from the plurality of prompt word templates according to the evaluation results respectively corresponding to the plurality of prompt word templates, wherein the evaluation result corresponding to the first prompt word template is better than the evaluation results corresponding to other prompt word templates among the plurality of prompt word templates.

11. The processing device according to claim 10, characterized in that The gradient process information includes multiple gradients, each gradient includes an identifier of at least one evaluation algorithm, and the priority of evaluation algorithms belonging to different gradients is different; Among them, different evaluation algorithms belonging to the same gradient have the same priority, or different evaluation algorithms belonging to the same gradient have different priority.

12. The processing device according to claim 11, characterized in that The gradient process information is a tree diagram, which includes multiple nodes and multiple directed edges. Each of the multiple nodes is used to indicate an evaluation algorithm, and each of the multiple directed edges is used to indicate a priority relationship between different evaluation algorithms.

13. The processing device according to claim 11 or 12, characterized in that: The plurality of gradients include a first gradient and a second gradient, and a priority of an evaluation algorithm belonging to the first gradient is higher than a priority of an evaluation algorithm belonging to the second gradient; The screening module is specifically used for: Detecting whether the data set includes the expected output result corresponding to the at least one set of variable values; When the data set does not include the expected output result, determining the first evaluation algorithm from at least one evaluation algorithm belonging to the first gradient; When the data set includes the expected output result, the first evaluation algorithm is determined from at least one evaluation algorithm belonging to the second gradient according to the characteristics of the expected output result.

14. The processing device according to claim 13, characterized in that The data set includes multiple sets of variable values, and the screening module is further used for: When the expected output result does not have a feature of a preset type, using a first group of variable values ​​among the multiple groups of variable values, pre-evaluating the multiple prompt word templates according to the at least one evaluation algorithm belonging to the second gradient, to obtain a pre-evaluation result corresponding to each prompt word template among the multiple prompt word templates; The first evaluation algorithm is determined from the at least one evaluation algorithm according to the pre-evaluation result corresponding to each prompt word template.

15. The processing device according to any one of claims 10 to 14, characterized in that The screening module is specifically used for: In response to a user's selection operation for automatically setting a control, a first evaluation algorithm matching the data set is screened out from the multiple evaluation algorithms according to the gradient process information.

16. The processing device according to claim 15, characterized in that The execution module is further used for: When the multiple evaluation algorithms do not match the data set, obtaining the user-defined second evaluation algorithm; According to the second evaluation algorithm and the data set, a second prompt word template is determined from the multiple prompt word templates, and an evaluation result corresponding to the second prompt word template is better than the evaluation results corresponding to other prompt word templates in the multiple prompt word templates.

17. The processing device according to claim 15 or 16, characterized in that The configuration module is further configured to output the gradient process information when the user does not perform a selection operation on the automatic setting control; in response to a user's selection operation on multiple evaluation algorithms in the gradient process information, determine a selected evaluation algorithm from the multiple evaluation algorithms as a third evaluation algorithm; The execution module is further used to determine a third prompt word template from the multiple prompt word templates according to the third evaluation algorithm and the data set, and the evaluation result corresponding to the third prompt word template is better than the evaluation results corresponding to other prompt word templates in the multiple prompt word templates.

18. The processing device according to any one of claims 10 to 17, characterized in that The algorithm management module is also used for: In response to a modification operation on the gradient process information, adjusting a priority level of selection of an evaluation algorithm in the gradient process information; Alternatively, a new evaluation algorithm and a priority of the new evaluation algorithm are added to the gradient process information.

19. A prompt word template processing platform, characterized in that: The prompt word template processing platform comprises a design device and a processing device as claimed in any one of claims 10 to 18; Wherein, the design device is used to generate the multiple prompt word templates.

20. A computing device cluster, characterized in that: comprising at least one computing device, each computing device comprising a processor and a memory; The processor is configured to execute instructions stored in the memory, so that the computing device cluster executes the method according to any one of claims 1 to 9.

21. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, which, when executed on at least one computing device, enable the at least one computing device to perform the method according to any one of claims 1 to 9.

Images