Target cue word information determination method and device and storage medium

Abstract

The embodiment of the invention provides a target cue word information determination method and device and a storage medium, and relates to the technical field of large models. The method comprises the steps of obtaining a first preset number of evaluation set data and a second preset number of cue word information; inputting each piece of to-be-input data into a preset model, and obtaining a corresponding output result based on the preset model and each piece of cue word information; wherein the number of the output results is equal to a first preset number; on the basis of each piece of cue word information, comparing the expected data with the corresponding output result to obtain a comparison result; and according to a comparison result, determining target prompt word information in the second preset number of prompt word information. By adopting the technical scheme, manual evaluation is not needed, and the efficiency and the evaluation accuracy are improved.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the field of large model technologies, and in particular, to a method, device, and storage medium for determining target prompt information. Background Art

[0002] Currently, in the decoration business scenario, large models are often used to analyze the room measurement reports. However, due to different prompt information in the large models, the results obtained by the large models are also different. Therefore, the prompt information determines the accuracy of the large model output.

[0003] However, the current evaluation method for large model prompt information is manual evaluation, which has low efficiency.

[0004] Therefore, there is an urgent need for a method for determining target prompt information that can improve efficiency and evaluation accuracy without manual evaluation. Summary of the Invention

[0005] To solve the above technical problems or at least partially solve the above technical problems, embodiments of the present disclosure provide a method, device, and storage medium for determining target prompt information.

[0006] A first aspect of embodiments of the present disclosure provides a method for determining target prompt information, the method comprising:

[0007] Obtain a first preset number of evaluation set data and a second preset number of prompt information; wherein, the evaluation set data includes data to be input and expected data; the data to be input is data to be input into a preset model; the expected data is data that a user expects the preset model to output based on the data to be input; the prompt information is used to indicate that the preset model outputs data of a preset type;

[0008] Input each of the data to be input into the preset model, and based on the preset model and each of the prompt information, obtain corresponding output results; wherein, the number of the output results is equal to the first preset number;

[0009] Based on each of the prompt information, compare the expected data with the corresponding output results to obtain a comparison result;

[0010] Determine target prompt information from the second preset number of prompt information according to the comparison result.

[0011] In one example, the step of based on each of the prompt information, comparing the expected data with the corresponding output results to obtain a comparison result includes:

[0012] Based on each piece of the prompt word information, obtain an evaluation algorithm, and compare the expected data and the corresponding output result according to the evaluation algorithm to obtain a comparison result; wherein, the evaluation algorithm is used to represent the comparison method for comparing the expected data and the corresponding output result.

[0013] In one example, the comparing the expected data and the corresponding output result according to the evaluation algorithm to obtain a comparison result includes:

[0014] Based on each piece of the prompt word information, compare each piece of the expected data and each piece of the corresponding output result in sequence according to the evaluation algorithm to obtain the score value of each piece of the data to be input and the scoring duration of each piece of the data to be input.

[0015] Based on each piece of the prompt word information, determine the average score value and the average scoring duration of each piece of the prompt word information according to the score value of each piece of the data to be input, the scoring duration of each piece of the data to be input, and the first preset quantity.

[0016] Determine the average score value and the average scoring duration of each piece of the prompt word information as the comparison result.

[0017] In one example, the determining the target prompt word information from the second preset quantity of the prompt word information according to the comparison result includes:

[0018] Search for the maximum score value among the average score values of the second preset quantity of the prompt word information.

[0019] Determine the prompt word information corresponding to the maximum score value as the target prompt word information.

[0020] In one example, the method further includes:

[0021] If there is an abnormal result in the comparison result, re - execute the comparison of the expected data and the corresponding output result based on each piece of the prompt word information to obtain an updated comparison result.

[0022] In one example, the first preset quantity of the evaluation set data is pre - configured by the user.

[0023] In one example, after obtaining the comparison result, the method further includes:

[0024] Output the comparison result in a preset form to obtain a visualization result; wherein, the visualization result is used for the user to view.

[0025] The second aspect of the embodiments of the present disclosure provides an apparatus for determining target prompt information, the apparatus comprising:

[0026] An acquisition module, configured to acquire a first preset number of evaluation set data and a second preset number of prompt information; wherein, the evaluation set data includes data to be input and expected data; the data to be input is data to be input into a preset model; the expected data is data that a user expects the preset model to output according to the data to be input; the prompt information is used to indicate that the preset model outputs data of a preset type;

[0027] An input module, configured to input each piece of the data to be input into the preset model, and based on the preset model and each piece of the prompt information, obtain a corresponding output result; wherein, the number of the output results is equal to the first preset number;

[0028] A comparison module, configured to compare the expected data and the corresponding output result based on each piece of the prompt information, and obtain a comparison result;

[0029] A determination module, configured to determine target prompt information from the second preset number of prompt information according to the comparison result.

[0030] The third aspect of the embodiments of the present disclosure provides an electronic device, the electronic device comprising: a processor and a memory, wherein, a computer program is stored in the memory, and when the computer program is executed by the processor, the processor executes the method of the first aspect above.

[0031] The fourth aspect of the embodiments of the present disclosure provides a computer-readable storage medium, in which a computer program is stored, and when the computer program is executed by a processor, the method of the first aspect above can be implemented.

[0032] The fifth aspect of the embodiments of the present disclosure provides a computer program product, comprising a computer program, and when the computer program is executed by a processor, the method described in the first aspect is implemented.

[0033] Embodiments of the present disclosure provide a method, device, and storage medium for determining target prompt information. The method includes: obtaining a first preset number of evaluation set data and a second preset number of prompt information; wherein, the evaluation set data includes data to be input and expected data; the data to be input is data to be input into a preset model; the expected data is data that a user expects the preset model to output based on the data to be input; the prompt information is used to indicate that the preset model outputs data of a preset type; inputting each piece of the data to be input into the preset model, and based on the preset model and each piece of the prompt information, obtaining a corresponding output result; wherein, the number of the output results is equal to the first preset number; based on each piece of the prompt information, comparing the expected data with the corresponding output result to obtain a comparison result; and determining target prompt information from the second preset number of prompt information according to the comparison result. By adopting the technical solution, it is possible to improve the efficiency and accuracy of determining target prompt information without manual evaluation. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The accompanying drawings are incorporated herein and form a part of this specification, showing embodiments consistent with the present disclosure and, together with the specification, are used to explain the principles of the present disclosure.

[0035] To more clearly illustrate the technical solutions in the embodiments of the present disclosure or in the prior art, the following will briefly introduce the accompanying drawings required for use in the description of the embodiments or the prior art. Obviously, for those of ordinary skill in the art, other drawings can also be obtained based on these drawings without creative efforts.

[0036] Figure 1 is a flowchart of a method for determining target prompt information provided by an embodiment of the present disclosure;

[0037] Figure 2 is a flowchart of a method for determining target prompt information provided by an embodiment of the present disclosure;

[0038] Figure 3 is a structural diagram of a device for determining target prompt information provided by an embodiment of the present disclosure;

[0039] Figure 4 is a structural diagram of an electronic device in an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0040] In order to better understand the above objects, features, and advantages of the present disclosure, the following will further describe the solutions of the present disclosure. It should be noted that, without conflict, the embodiments of the present disclosure and the features in the embodiments can be combined with each other.

[0041] In the following description, many specific details are set forth in order to provide a thorough understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; obviously, the embodiments in the specification are only a part of the embodiments of the present disclosure, rather than all of the embodiments.

[0042] Figure 1 FIG. is a schematic flowchart of a method for determining target prompt word information provided by an embodiment of the present disclosure, and this method can be executed by an electronic device. The electronic device can be exemplarily understood as devices such as mobile phones, tablet computers, laptop computers, desktop computers, smart TVs, etc. As Figure 1 shown, the method provided in this embodiment includes the following steps:

[0043] S101. Obtain a first preset number of evaluation set data and a second preset number of prompt word information; wherein, the evaluation set data includes data to be input and expected data; the data to be input is the data to be input into a preset model; the expected data is the data that the user expects the preset model to output based on the data to be input; the prompt word information is used to indicate that the preset model outputs data of a preset type.

[0044] In one example, the first preset number of evaluation set data is preconfigured by the user and can be 30 or 50. The second preset number is also pre-set and can be 10. The evaluation set data includes data to be input and expected data, and the evaluation set data is pre-filled by the user. Each piece of data to be input corresponds to an expected data. When the evaluation set data is 30 pieces, the data to be input is 30, and the expected data is also 30. For example, the data to be input can be "I need to hold a weekly meeting at 11 o'clock", and the expected data can be "hold a weekly meeting".

[0045] In one example, the prompt word information can be a word or a piece of text information, and is used to indicate that the preset model outputs data of a preset type. For example, the prompt word information can be "generate a follow-up content". The same data to be input is input into the same preset model, but with different prompt word information, the preset model will output different results.

[0046] S102. Input each piece of data to be input into the preset model, and based on the preset model and each piece of prompt word information, obtain corresponding output results; wherein, the number of output results is equal to the first preset number.

[0047] In one example, there can be multiple preset models, which can be selected in advance. After determining the preset model, for the same prompt information, each piece of input data in the first preset quantity is input into the preset model, and then the corresponding output result is obtained. Then, the prompt information is changed, and each piece of input data in the first preset quantity is input into the preset model again, and then the corresponding output result is obtained. This process continues until all the prompt information has been processed.

[0048] S103. Based on each piece of prompt information, compare the expected data with the corresponding output result to obtain a comparison result.

[0049] In one example, based on each piece of prompt information, perform a text similarity comparison between the expected data and the corresponding output result to obtain a comparison result.

[0050] S104. Determine the target prompt information from the second preset quantity of prompt information according to the comparison result.

[0051] In one example, if the comparison result indicates that the similarity between the expected data and the corresponding output result is relatively high, it means that this prompt information is relatively accurate. Then, the target prompt information can be determined from the second preset quantity of prompt information. This target prompt information is used to configure in the preset model to extract preset content according to the user's input.

[0052] The embodiments of the present disclosure provide a method for determining target prompt information. The method includes: obtaining the first preset quantity of evaluation set data and the second preset quantity of prompt information; inputting each piece of input data into the preset model, and based on the preset model and each piece of prompt information, obtaining the corresponding output result; wherein, the number of output results is equal to the first preset quantity; based on each piece of prompt information, comparing the expected data with the corresponding output result to obtain a comparison result; and determining the target prompt information from the second preset quantity of prompt information according to the comparison result. By adopting this technical solution, it is possible to improve the efficiency and accuracy of determining the target prompt information without manual evaluation.

[0053] Figure 2 The flowchart of a method for determining target prompt information provided by the embodiments of the present disclosure is shown. The embodiments of the present disclosure are optimized on the basis of the above embodiments, and the embodiments of the present disclosure can be combined with each optional solution in one or more of the above embodiments.

[0054] As Figure 2 shown, the method for determining target prompt information may include the following steps:

[0055] S201. Obtain a first preset quantity of evaluation set data and a second preset quantity of prompt word information; wherein, the evaluation set data includes data to be input and expected data; the data to be input is the data to be input into a preset model; the expected data is the data that the user expects the preset model to output based on the data to be input; the prompt word information is used to indicate that the preset model outputs data of a preset type.

[0056] In one example, the content of this step can refer to the content of step S101.

[0057] S202. Input each piece of data to be input into the preset model, and based on the preset model and each piece of prompt word information, obtain corresponding output results; wherein, the number of output results is equal to the first preset quantity.

[0058] In one example, the content of this step can refer to the content of step S102.

[0059] S203. Based on each piece of prompt word information, obtain an evaluation algorithm, and compare the expected data with the corresponding output result according to the evaluation algorithm to obtain a comparison result; wherein, the evaluation algorithm is used to represent the comparison method of comparing the expected data with the corresponding output result.

[0060] In one example, the evaluation algorithm can be precise evaluation, json evaluation, and manual evaluation. Among them, precise evaluation is to compare the expected data with the corresponding output result word by word, and if they are exactly the same, points can be added to the comparison result. Json evaluation is to compare the structured data in the expected data and the corresponding output result, and score according to the data in different structural parts to obtain a comparison result. Manual evaluation is to score the corresponding output result by the user to obtain a comparison result.

[0061] In one example, comparing the expected data with the corresponding output result according to the evaluation algorithm to obtain a comparison result includes:

[0062] Based on each piece of prompt word information, compare each piece of expected data with each corresponding output result in turn according to the evaluation algorithm to obtain the score value of each piece of data to be input and the scoring duration of each piece of data to be input.

[0063] Based on each piece of prompt word information, determine the average score value and average scoring duration of each piece of prompt word information according to the score value of each piece of data to be input, the scoring duration of each piece of data to be input, and the first preset quantity.

[0064] Determine the average score value and average scoring duration of each piece of prompt word information as the comparison result.

[0065] In one example, the score value of each piece of data to be input is used to characterize the similarity between each piece of expected data and each corresponding output result. The higher the score value of each piece of data to be input, the more similar each piece of expected data and each corresponding output result are. The score value of the data to be input is a value between 1 and 100.

[0066] In one example, the unit of the scoring duration of each piece of data to be input can be S. For example, the scoring duration of each piece of data to be input is 7.5S.

[0067] In one example, based on each piece of prompt word information, according to the score value of each piece of data to be input and the first preset quantity, the average score value of each piece of prompt word information is determined; based on each piece of prompt word information, according to the scoring duration of each piece of data to be input and the first preset quantity, the average scoring duration of each piece of prompt word information is determined, and then the average score value and the average scoring duration of each piece of prompt word information are determined as the comparison result.

[0068] In one example, the method further includes:

[0069] If there is an abnormal result in the comparison result, then re - execute the comparison of the expected data and the corresponding output result based on each piece of prompt word information to obtain an updated comparison result.

[0070] In one example, for example, if the average scoring duration in the comparison result is 0, it is determined that there is an abnormal result in the comparison result, and then the content of step S203 is re - executed.

[0071] In one example, after obtaining the comparison result, the method further includes:

[0072] Output the comparison result in a preset form to obtain a visualization result; wherein, the visualization result is used for the user to view.

[0073] In one example, the preset form can be excel format, csv format, and tsv format. Specifically, the comparison result can be exported in the above - mentioned preset form to obtain a visualization result and displayed to the user.

[0074] S204. Search for the maximum score value among the average score values of the second preset quantity of prompt word information.

[0075] In one example, for example, the second preset quantity is 3. The average score value of the first piece of prompt word information is 85 points, the average score value of the second piece of prompt word information is 75 points, and the average score value of the third piece of prompt word information is 70 points, then the maximum score value is 85 points.

[0076] S205. Determine the target prompt information as the prompt information corresponding to the maximum evaluation score value.

[0077] In one example, determine the first prompt information corresponding to the maximum evaluation score value of 85 points as the target prompt information.

[0078] The embodiments of the present disclosure provide a method for determining target prompt information. The method includes: based on each piece of prompt information, obtain an evaluation algorithm, and compare the expected data and the corresponding output result according to the evaluation algorithm to obtain a comparison result. Search for the maximum evaluation score value among the average evaluation score values of the second preset number of pieces of prompt information, and determine the prompt information corresponding to the maximum evaluation score value as the target prompt information. By adopting this technical solution, it is possible to support multiple preset models and multiple evaluation algorithms, and thus the applicable range is wider.

[0079] Figure 3 It is a schematic structural diagram of a device for determining target prompt information provided by the embodiments of the present disclosure. The device for determining target prompt information can be understood as the above-mentioned electronic device or some functional modules in the above-mentioned electronic device. As Figure 3 shown, the device 30 for determining target prompt information includes:

[0080] An acquisition module 301, configured to acquire the first preset number of evaluation set data and the second preset number of pieces of prompt information; wherein, the evaluation set data includes the data to be input and the expected data; the data to be input is the data to be input into the preset model; the expected data is the data that the user expects the preset model to output according to the data to be input; the prompt information is used to represent indicating that the preset model outputs data of a preset type.

[0081] An input module 302, configured to input each piece of data to be input into the preset model, and obtain the corresponding output result based on the preset model and each piece of prompt information; wherein, the number of output results is equal to the first preset number.

[0082] A comparison module 303, configured to compare the expected data and the corresponding output result based on each piece of prompt information to obtain a comparison result.

[0083] A determination module 304, configured to determine the target prompt information from the second preset number of pieces of prompt information according to the comparison result.

[0084] In one example, the comparison module 303 includes:

[0085] A comparison sub-module, configured to obtain an evaluation algorithm based on each piece of prompt information, and compare the expected data and the corresponding output result according to the evaluation algorithm to obtain a comparison result; wherein, the evaluation algorithm is used to represent the comparison method for comparing the expected data and the corresponding output result.

[0086] In one example, a comparison sub-module is configured to compare each piece of expected data and each corresponding output result successively according to an evaluation algorithm based on each piece of prompt word information, so as to obtain the evaluation score value of each piece of data to be input and the evaluation duration of each piece of data to be input;

[0087] Based on each piece of prompt word information, determine the average evaluation score value and the average evaluation duration of each piece of prompt word information according to the evaluation score value of each piece of data to be input, the evaluation duration of each piece of data to be input, and a first preset quantity;

[0088] Determine the average evaluation score value and the average evaluation duration of each piece of prompt word information as the comparison result.

[0089] In one example, the determination module 304 includes:

[0090] A search sub-module is configured to search for the maximum evaluation score value among the average evaluation score values of a second preset quantity of prompt word information;

[0091] A determination sub-module is configured to determine the prompt word information corresponding to the maximum evaluation score value as the target prompt word information.

[0092] In one example, the apparatus 30 further includes:

[0093] An execution module 305 is configured to, if there is an abnormal result in the comparison result, re-execute the comparison of the expected data and the corresponding output result based on each piece of prompt word information to obtain an updated comparison result.

[0094] In one example, the first preset quantity of evaluation set data is pre-configured by a user.

[0095] In one example, after obtaining the comparison result, the apparatus 30 further includes:

[0096] An output module 306 is configured to output the comparison result in a preset form to obtain a visualization result; wherein, the visualization result is used for the user to view.

[0097] The apparatus provided in this embodiment can execute the method of any of the above embodiments, and its execution manner and beneficial effects are similar, which will not be elaborated here.

[0098] This embodiment of the present disclosure further provides an electronic device, which includes: a memory storing a computer program; a processor configured to execute the computer program, and when the computer program is executed by the processor, the method of any of the above embodiments can be implemented.

[0099] By way of example, Figure 4 is a schematic structural diagram of an electronic device in an embodiment of the present disclosure. Specifically, refer to the followingFigure 4 , which shows a schematic structural diagram suitable for implementing the electronic device 1000 in the embodiments of the present disclosure. The electronic device 1000 in the embodiments of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Tablet Computers), PMPs (Portable Multimedia Players), in-vehicle terminals (such as in-vehicle navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 4 The electronic device shown is only an example and should not impose any limitations on the functions and usage scope of the embodiments of the present disclosure.

[0100] As Figure 4 shown, the electronic device 1000 may include a processing device (such as a central processing unit, a graphics processing unit, etc.) 1001, which may perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 1002 or the program loaded from the storage device 1008 into the random access memory (RAM) 1003. In the RAM 1003, various programs and data required for the operation of the electronic device 1000 are also stored. The processing device 1001, the ROM 1002, and the RAM 1003 are connected to each other through a bus 1004. The input / output (I / O) interface 1005 is also connected to the bus 1004.

[0101] Generally, the following devices may be connected to the I / O interface 1005: an input device 1006 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 1007 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 1008 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 may allow the electronic device 1000 to communicate with other devices wirelessly or wiredly to exchange data. Although Figure 4 the electronic device 1000 with various devices is shown, it should be understood that it is not required to implement or include all the shown devices. More or fewer devices may be alternatively implemented or included.

[0102] Specifically, according to the embodiments of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, the embodiments of the present disclosure include a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program includes program codes for executing the methods shown in the flowcharts. In such an embodiment, the computer program may be downloaded and installed from the network through the communication device 1009, or installed from the storage device 1008, or installed from the ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the methods of the embodiments of the present disclosure are executed.

[0103] It should be noted that the above-mentioned computer-readable medium in the present disclosure can be a computer-readable signal medium, a computer-readable storage medium, or any combination of the two. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of the computer-readable storage medium can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, the computer-readable storage medium can be any tangible medium that contains or stores a program, and this program can be used by or in conjunction with an instruction execution system, apparatus, or device. In the present disclosure, the computer-readable signal medium can include a data signal propagated in a baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable signal medium can also be any computer-readable medium other than the computer-readable storage medium, and this computer-readable signal medium can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted by any appropriate medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0104] In some embodiments, the client and the server can communicate using any currently known or future-developed network protocol such as HTTP (HyperText Transfer Protocol), and can be interconnected with digital data communication in any form or medium (for example, a communication network). Examples of communication networks include local area networks ("LAN"), wide area networks ("WAN"), the Internet (for example, the Internet), and end-to-end networks (for example, ad hoc end-to-end networks), as well as any currently known or future-developed networks.

[0105] The above-mentioned computer-readable medium can be included in the above-mentioned electronic device; it can also exist separately without being assembled into the electronic device.

[0106] The above computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to: obtain a first preset number of evaluation set data and a second preset number of prompt word information; wherein, the evaluation set data includes data to be input and expected data; the data to be input is the data to be input into a preset model; the expected data is the data that the user expects the preset model to output according to the data to be input; the prompt word information is used to indicate that the preset model outputs data of a preset type; input each piece of data to be input into the preset model, and based on the preset model and each piece of prompt word information, obtain corresponding output results; wherein, the number of output results is equal to the first preset number; based on each piece of prompt word information, compare the expected data with the corresponding output result to obtain a comparison result; according to the comparison result, determine the target prompt word information from the second preset number of prompt word information.

[0107] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages or combinations thereof. The programming languages include, but are not limited to, object-oriented programming languages such as Java, Smalltalk, C++, and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code may execute entirely on the user's computer, partially on the user's computer, execute as a stand-alone software package, execute partially on the user's computer and partially on a remote computer, or execute entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (for example, by connecting through an Internet service provider using the Internet).

[0108] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code that contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and the combinations of blocks in the block diagram and / or flowchart, may be implemented by a dedicated hardware-based system for performing the specified functions or operations, or may be implemented by a combination of dedicated hardware and computer instructions.

[0109] The units involved in the embodiments of the present disclosure may be implemented in software or in hardware. In some cases, the name of a unit does not constitute a limitation on the unit itself.

[0110] The functions described above in this document may be performed, at least in part, by one or more hardware logic components. By way of example, and without limitation, the types of hardware logic components that may be used include: Field Programmable Gate Arrays (FPGA), Application Specific Integrated Circuits (ASIC), Application Specific Standard Products (ASSP), System on a Chip (SOC), Complex Programmable Logic Devices (CPLD), and the like.

[0111] In the context of the present disclosure, a machine-readable medium may be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a Read-Only Memory (ROM), an Erasable Programmable Read-Only Memory (EPROM or Flash memory), an optical fiber, a portable Compact Disc Read-Only Memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0112] The embodiments of the present disclosure also provide a computer-readable storage medium storing a computer program which, when executed by a processor, can implement the method of any of the foregoing embodiments. The execution manner and beneficial effects are similar and will not be elaborated herein.

[0113] It should be noted that, in this document, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises", "comprising" or any other variation thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that comprises the element.

[0114] The above are only specific embodiments of the present disclosure, enabling those skilled in the art to understand or implement the present disclosure. Various modifications to these embodiments will be obvious to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to these embodiments herein, but rather to the broadest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for determining target prompt word information, characterized in that, The method includes: Obtaining a first preset number of evaluation set data and a second preset number of prompt word information; wherein, the evaluation set data includes data to be input and expected data; the data to be input is the data to be input into a preset model; the expected data is the data that the user expects the preset model to output according to the data to be input; the prompt word information is used to indicate that the preset model outputs data of a preset type; Inputting each of the data to be input into the preset model, and based on the preset model and each of the prompt word information, obtaining corresponding output results; wherein, the number of the output results is equal to the first preset number; Based on each of the prompt word information, comparing the expected data with the corresponding output result to obtain a comparison result; Determining target prompt word information from the second preset number of prompt word information according to the comparison result.

2. The method according to claim 1, characterized in that, The step of comparing the expected data with the corresponding output result based on each of the prompt word information to obtain a comparison result includes: Based on each of the prompt word information, obtaining an evaluation algorithm, and comparing the expected data with the corresponding output result according to the evaluation algorithm to obtain a comparison result; wherein, the evaluation algorithm is used to represent the comparison method for comparing the expected data with the corresponding output result.

3. The method according to claim 2, wherein The step of comparing the expected data with the corresponding output result according to the evaluation algorithm to obtain a comparison result includes: Based on each of the prompt word information, comparing each piece of the expected data with each piece of the corresponding output result in sequence according to the evaluation algorithm to obtain a score value for each piece of the data to be input and a scoring duration for each piece of the data to be input; Based on each of the prompt word information, determining an average score value and an average scoring duration for each of the prompt word information according to the score value for each piece of the data to be input, the scoring duration for each piece of the data to be input, and the first preset number; Determining the average score value and the average scoring duration for each of the prompt word information as the comparison result.

4. The method according to claim 3, wherein The step of determining target prompt word information from the second preset number of prompt word information according to the comparison result includes: Searching for the maximum score value among the average score values of the second preset number of the prompt word information; Determining the prompt word information corresponding to the maximum score value as the target prompt word information.

5. The method according to claim 3, characterized in that The method further includes: If there is an abnormal result in the comparison result, re-executing the step of comparing the expected data with the corresponding output result based on each of the prompt word information to obtain an updated comparison result.

6. The method according to claim 1, wherein The first preset number of evaluation set data is pre-configured by the user.

7. The method according to claim 1, characterized in that, After obtaining the comparison result, the method further includes: Outputting the comparison result in a preset form to obtain a visualization result; wherein, the visualization result is for the user to view.

8. An electronic device, characterized in that, Including: A processor and a memory, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the processor executes the method according to any one of claims 1-7.

9. A computer-readable storage medium, characterized in that, A computer program is stored in the storage medium, and when the computer program is executed by a processor, the method according to any one of claims 1-7 is implemented.

10. A computer program product, characterized in that, It includes a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1-7 is implemented.

Images