Question Prompt Generation Method, System and Electronic Device

Abstract

This application relates to the technical field of large language models, and discloses a method, system and electronic device for generating problem prompting words. This application scores the text quality of the initial prompting words, screens out alternative prompting words in the initial prompting words according to the quality score, and scores the question-and-answer effect of the alternative prompting words. Using the effect score as the optimization criterion, it optimizes the text of the alternative prompting words, thereby generating the problem prompting words corresponding to the target problem according to the optimized alternative prompting words. It evaluates the text quality of the prompting words themselves according to the quality score, thereby screening out prompting words with high text quality, evaluates the question-and-answer effect of the prompting words during use according to the effect score, and optimizes the prompting words based on the effect score, so that the automatically generated prompting words are guaranteed in terms of text quality and question-and-answer effect, improving the generation quality of the problem prompting words, and further improving the output accuracy of the large language model.

Description

Technical Field

[0001] The present invention relates to the technical field of large language models, and in particular to a method, system and electronic device for generating problem prompt words. Background Art

[0002] In the field of natural language processing (NLP), large language models (LLMs) are a cutting-edge artificial intelligence technology. Through deep learning theory, they train a vast amount of text data, master the grammar, semantics, and context information of language, and thus can accurately process and generate human language, with great potential and broad application prospects. Among them, the question answering accuracy rate is one of the key indicators to measure the model performance. As a bridge between users and the model, prompt words guide the model output through clear and specific guiding language. The design quality of prompt words is directly related to the degree of the model's understanding of the user's intention and the accuracy of the generated answer. If the prompt words are too simple or ambiguous, they cannot accurately convey the user's intention. If the prompt words are too complex or redundant, the model cannot correctly understand and generate the corresponding answer.

[0003] Since the design of prompt words requires developers to have profound professional knowledge and rich practical experience in order to accurately grasp the core intention of the target problem and design prompt words that can stimulate the potential of the model, some developers have tried to use machine learning models to replace manual design and automatically generate problem prompt words. However, due to the significant differences in the fields involved in different large language models, machine learning models often require a large amount of standard data to ensure the usability of prompt words.

[0004] Therefore, due to the significant differences in language characteristics and problem types in different fields, the generated quality of prompt words generated by machine learning models is uneven, resulting in a low output accuracy rate of large language models. Summary of the Invention

[0005] To provide a basic understanding of some aspects of the disclosed embodiments, a simple summary is given below. This summary is not a comprehensive review, nor is it intended to identify key / important elements or delineate the scope of protection of these embodiments. Instead, it serves as a preface to the subsequent detailed description.

[0006] In view of the above-mentioned disadvantages of the prior art, the present application provides a method, system and electronic device for generating problem prompt words to improve the generation quality of problem prompt words.

[0007] The present application provides a method for generating problem prompting words, including: obtaining a plurality of initial prompting words corresponding to a target problem; scoring the text quality of each of the initial prompting words to obtain the quality scores respectively corresponding to each of the initial prompting words, and screening each of the initial prompting words according to the quality scores to obtain alternative prompting words; scoring the question-and-answer effects of the alternative prompting words to obtain effect scores, and using the effect scores as an optimization criterion to optimize the texts of the alternative prompting words; generating the problem prompting words corresponding to the target problem according to the optimized alternative prompting words.

[0008] In an embodiment of the present application, obtaining a plurality of initial prompting words corresponding to a target problem includes: obtaining a plurality of historical questions input by a user; grouping each of the historical questions according to the text similarity between the historical questions to obtain similar question groups; if the number of historical questions in the similar question group is greater than or equal to a preset number threshold, determining high-frequency questions according to the historical questions in the similar question group, so as to form a high-frequency question library according to the high-frequency questions; determining the target problem from the high-frequency question library, and inputting the target problem into a plurality of preset prompting word generation models to obtain the initial prompting words respectively output by each of the prompting word generation models, wherein the prompting word generation model is trained based on a large language model.

[0009] In an embodiment of the present application, scoring the text quality of each of the initial prompting words to obtain the quality scores respectively corresponding to each of the initial prompting words includes: obtaining one or more prompting word scoring models, wherein the prompting word scoring model is trained based on a large language model; using each of the prompting word scoring models to evaluate the rationality of the initial prompting words; if the initial prompting word does not meet the rationality, the initial prompting word does not have a quality score; if the initial prompting word meets the rationality, using each of the prompting word scoring models to score the text quality of the initial prompting word according to a preset scoring dimension to obtain the quality scoring results respectively output by each of the prompting word scoring models, and calculating by combining each of the quality scoring results to obtain the quality score.

[0010] In an embodiment of the present application, screening each of the initial prompting words according to the quality scores to obtain alternative prompting words includes at least one of the following: using the initial prompting word with the highest quality score as the alternative prompting word; using the initial prompting words with quality scores greater than or equal to a preset quality threshold as the alternative prompting words.

[0011] In an embodiment of the present application, by scoring the question-and-answer effect of the alternative prompt words, an effect score is obtained, including at least one of the following: determining a model output result corresponding to the alternative prompt words by using a preset question-and-answer model, and scoring the model output result by using a preset result scoring model to obtain an effect score; collecting user feedback data corresponding to the alternative prompt words, and scoring according to the user feedback data to obtain an effect score, where the user feedback data includes at least one of the number of user modifications, the number of user uses, user positive evaluations, and user negative evaluations.

[0012] In an embodiment of the present application, taking the effect score as an optimization criterion, text optimization is performed on the alternative prompt words, including: in response to the alternative prompt words, comparing the effect score corresponding to the alternative prompt words according to a preset effect threshold; if the alternative prompt words meet the first preset condition and the second preset condition, setting the text optimization result of the alternative prompt words to optimization completed, where the first preset condition includes that the effect score is less than the preset effect threshold, and the second preset condition includes that the number of text optimization times of the alternative prompt words is less than a preset optimization times threshold; if the alternative prompt words do not meet the first preset condition, performing text optimization on the alternative prompt words to obtain new alternative prompt words; if the alternative prompt words do not meet the second preset condition, setting the text optimization result of the alternative prompt words to optimization failed.

[0013] In an embodiment of the present application, text optimization is performed on the alternative prompt words by at least one of the following methods: performing text processing on the alternative prompt words, where the text processing includes at least one of deleting stop words, replacing synonyms, replacing near-synonyms, and adjusting the text order; using a prompt word scoring model to output a quality evaluation corresponding to the alternative prompt words, where the prompt word scoring model is used to score the text quality of each of the initial prompt words; performing text parsing on the quality evaluation to extract a model positive evaluation and a model negative evaluation from the quality evaluation according to the text parsing result; generating a prompt word generation text according to at least one of the alternative prompt words, the model positive evaluation, and the model negative evaluation, and inputting the prompt word generation text into a prompt word generation model, so that the prompt word generation model outputs new alternative prompt words based on the model positive evaluation and / or the model negative evaluation.

[0014] In one embodiment of the present application, after generating the question prompt word corresponding to the target question according to the optimized alternative prompt words, the method further includes: presenting the question prompt word to the user; if a modification instruction input by the user is received, modifying the question prompt word according to the prompt word modification instruction; if a selection instruction input by the user is received, inputting the question prompt word into a preset question-and-answer model, so that the preset question-and-answer model outputs the question-and-answer result corresponding to the target question.

[0015] The present application provides a question prompt word generation system, including: an acquisition module configured to acquire a plurality of initial prompt words corresponding to a target question; a screening module configured to obtain the quality scores respectively corresponding to the initial prompt words by scoring the text quality of each of the initial prompt words, and screening the initial prompt words according to the quality scores to obtain alternative prompt words; an optimization module configured to obtain an effect score by scoring the question-and-answer effect of the alternative prompt words, and taking the effect score as an optimization criterion to perform text optimization on the alternative prompt words; a generation module configured to generate the question prompt word corresponding to the target question according to the optimized alternative prompt words.

[0016] The present application provides an electronic device, including: a processor and a memory; the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory, so that the electronic device executes the above method.

[0017] Advantages of the present application:

[0018] By scoring the text quality of the initial prompt words, screening out the alternative prompt words in the initial prompt words according to the quality scores, and scoring the question-and-answer effect of the alternative prompt words, and taking the effect score as an optimization criterion to perform text optimization on the alternative prompt words, so as to generate the question prompt word corresponding to the target question according to the optimized alternative prompt words. In this way, the text quality of the prompt words themselves is evaluated according to the quality scores, so as to screen out the prompt words with high text quality, and the question-and-answer effect of the prompt words during use is evaluated according to the effect scores, and the prompt words are optimized based on the effect scores, so that the automatically generated prompt words are guaranteed in terms of text quality and question-and-answer effect, improving the generation quality of the question prompt words, and further improving the output accuracy of the large language model. Description of the Drawings

[0019] Figure 1 is a flowchart of a question prompt word generation method in an embodiment of the present application;

[0020] Figure 2 is a flowchart of an initial prompt word acquisition method in an embodiment of the present application;

[0021] Figure 3 It is a schematic flow diagram of a text quality scoring method in an embodiment of the present application;

[0022] Figure 4 It is a schematic flow diagram of a text optimization method in an embodiment of the present application;

[0023] Figure 5 It is a schematic flow diagram of another problem prompt word generation method in an embodiment of the present application;

[0024] Figure 6 It is a schematic structural diagram of a problem prompt word generation system in an embodiment of the present application;

[0025] Figure 7 It is a schematic structural diagram of an electronic device in an embodiment of the present invention.

[0026] Reference numerals:

[0027] 601 - Acquisition module; 602 - Screening module; 603 - Optimization module; 604 - Generation module;

[0028] 700 - Computer system; 701 - Central processing unit; 702 - Read - only memory; 703 - Random access memory; 704 - Bus; 705 - Input / output interface; 706 - Input part; 707 - Output part; 708 - Storage part; 709 - Communication part; 710 - Driver; 711 - Removable medium. Detailed implementation manners

[0029] The following uses specific specific examples to illustrate the implementation manners of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific implementation manners. Various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that, without conflict, the following embodiments and sub - samples in the embodiments can be combined with each other.

[0030] It should be noted that the diagrams provided in the following embodiments only illustrate the basic concept of the present invention in a schematic manner. Therefore, only the components related to the present invention are shown in the diagrams, rather than being drawn according to the number, shape, and size of the components in actual implementation. The type, quantity, and proportion of each component in actual implementation can be arbitrarily changed, and the component layout type may also be more complex.

[0031] In the following description, numerous specific details are explored to provide a more thorough explanation of the embodiments of the present invention. However, it will be apparent to those skilled in the art that the embodiments of the present invention can be practiced without these specific details. In other embodiments, well-known structures and devices are shown in block diagram form rather than in detail to avoid obscuring the embodiments of the present invention.

[0032] In the specification, claims and drawings of this application, the terms "first", "second", etc. are used to distinguish similar objects and do not necessarily have to be used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances for the embodiments of this application described herein. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion.

[0033] Unless otherwise specified, the term "plurality" means two or more.

[0034] In this application, the character " / " indicates that the objects before and after are in an "or" relationship. For example, A / B means: A or B.

[0035] The term "and / or" is a description of the association relationship of objects, indicating that three relationships can exist. For example, A and / or B means: A or B, or, the three relationships of A and B.

[0036] In combination Figure 1 As shown, this application provides a method for generating problem prompting words, including:

[0037] Step S101, obtaining a plurality of initial prompting words corresponding to the target problem;

[0038] Step S102, by scoring the text quality of each initial prompting word, obtaining the quality score corresponding to each initial prompting word, and screening each initial prompting word according to the quality score to obtain alternative prompting words;

[0039] Step S103, by scoring the question-and-answer effect of the alternative prompting words, obtaining an effect score, and using the effect score as an optimization criterion to optimize the text of the alternative prompting words;

[0040] Step S104, generating problem prompting words corresponding to the target problem according to the optimized alternative prompting words.

[0041] By using the problem prompt word generation method provided in this application, the text quality of the initial prompt words is scored, so as to screen out the alternative prompt words in the initial prompt words according to the quality score, and the Q&A effect of the alternative prompt words is scored, and the effect score is used as the optimization criterion to optimize the text of the alternative prompt words, so as to generate the problem prompt words corresponding to the target problem according to the optimized alternative prompt words. In this way, the text quality of the prompt words themselves is evaluated according to the quality score, so as to screen out the prompt words with high text quality, and the Q&A effect of the prompt words in the use process is evaluated according to the effect score, and the prompt words are optimized based on the effect score, so that the automatically generated prompt words are guaranteed in terms of text quality and Q&A effect, improving the generation quality of the problem prompt words, and further improving the output accuracy of the large language model.

[0042] Combined with Figure 2 As shown, in step S101, multiple initial prompt words corresponding to the target problem are obtained, including:

[0043] Step S10101, obtain multiple historical questions input by the user;

[0044] Among them, the historical questions are text-segmented to obtain tokens (words);

[0045] Among them, stop words including "de" (of), "shi" (is), etc. are removed from the historical questions;

[0046] Step S10102, group each historical question according to the text similarity between historical questions to obtain similar question groups;

[0047] Among them, the historical questions are converted into text feature vectors by using a preset text feature extraction algorithm, and the text feature extraction algorithm includes the TF-IDF (Term Frequency-Inverse Document Frequency) algorithm;

[0048] Among them, if the cosine similarity of two historical questions on the text feature vectors is greater than the preset similarity threshold, the two historical questions are divided into the same similar question group to group similar historical questions together to form a similar question group;

[0049] Step S10103, if the number of historical questions in the similar question group is greater than or equal to the preset number threshold, determine the high-frequency questions according to the historical questions in the similar question group, so as to form a high-frequency question library according to the high-frequency questions;

[0050] Among them, when the content in the similar question group reaches the preset number threshold, the similar question group is determined as a high-frequency group;

[0051] Among them, the historical question with the highest similarity to other similar questions within the high-frequency group is used as the high-frequency question;

[0052] Step S10104, determine the target question from the high-frequency question library, and input the target question into a plurality of preset prompt word generation models to obtain the initial prompt words respectively output by each prompt word generation model;

[0053] Among them, the prompt word generation model is trained based on a large language model.

[0054] In some embodiments, if the number of characters of the target question is too small, for example, "Hello", "1", etc., the target question will lack effective information; if the number of characters of the target question is too large, the length of the prompt words generated by the target question will deviate, resulting in the continuous expansion of the reply length during training, affecting the model's ability to generate concise and efficient Q&A. To ensure the quality of the target question, filtering the historical questions by the character length can effectively improve the final quality of the prompt words.

[0055] In some embodiments, the target question includes question Q1, question Q2, and question Q3. Among them, the target question is directly input into a preset Q&A model to obtain the original Q&A result corresponding to the target question, as shown in Table 1.

[0056] Table 1

[0057]

[0058] In some embodiments, multiple large language models are used as the role of prompt word content producers to establish a prompt word generation model. Among them, the establishment method of the prompt word generation model includes third-party models and / or self-developed models; a generation task is arranged for the prompt word generation model according to the target question to obtain alternative prompt words; the alternative prompt words are input into a preset Q&A model to obtain the current output result. For example, the prompt word generation model includes model A, model B, and model C. The target question Q2 is input into each prompt word generation model to obtain the initial prompt words P1, P2, and P3 respectively, as shown in Table 2.

[0059] Table 2

[0060]

[0061] In some embodiments, the initial prompt words P1, P2, and P3 are input into a preset Q&A model to obtain the current output result corresponding to the target question. A part of the current output result is shown in Table 3; by comparing the original question result and the current output result, it can be seen that the prompt words can improve the output effect of the large language model.

[0062] Table 3

[0063]

[0064] Combine Figure 3 As shown, in step S102, by scoring the text quality of each initial prompt, the quality scores corresponding to each initial prompt are obtained, including:

[0065] Step S10201, obtain one or more prompt scoring models;

[0066] Among them, the prompt scoring model is trained based on a large language model;

[0067] Step S10202, use each prompt scoring model to evaluate the rationality of the initial prompt;

[0068] Step S10203, if the initial prompt does not meet the rationality, the initial prompt does not have a quality score;

[0069] Step S10204, if the initial prompt meets the rationality, use each prompt scoring model to score the text quality of the initial prompt according to the preset scoring dimensions, and obtain the quality scoring results respectively output by each prompt scoring model;

[0070] Step S10205, combine the calculation of each quality scoring result to obtain the quality score.

[0071] In some embodiments, legality is used to represent whether the initial prompt contains content such as illegality, prejudice, and discrimination.

[0072] In some embodiments, the preset scoring dimensions include accuracy, practicability, and logicality. Among them, accuracy is used to represent whether the initial prompt is true and reliable, practicability is used to represent whether the initial prompt contains specific content rather than a generalization description, and logicality is used to represent whether the initial prompt has characteristics such as correct grammar, clear structure, and strict logic.

[0073] In some embodiments, the initial prompts include P1, P2, and P3. At the same time, the prompt scoring models include model D, model E, and model F; the initial prompts are scored by model D, model E, and model F in terms of accuracy, practicability, logicality, and rationality respectively, and the scoring results are shown in Table 4.

[0074] Table 4

[0075]

[0076] In some embodiments, score weights corresponding to accuracy, usability, and logic are determined according to business requirements. For example, the score weight for accuracy is 40%, the score weight for usability is 30%, and the score weight for logic is 30%.

[0077] In some embodiments, the accuracy score of the initial prompt P1 is , the usability score of the initial prompt P1 is , and the logic score of the initial prompt P1 is . Then, based on the accuracy score, usability score, and logic score, a weighted calculation is performed, and the quality score of the initial prompt P1 is 7.57.

[0078] In some embodiments, the accuracy score of the initial prompt P2 is , the usability score of the initial prompt P2 is , and the logic score of the initial prompt P2 is . Then, based on the accuracy score, usability score, and logic score, a weighted calculation is performed, and the quality score of the initial prompt P2 is 8.23.

[0079] In some embodiments, the accuracy score of the initial prompt P3 is , the usability score of the initial prompt P3 is , and the logic score of the initial prompt P3 is . Then, based on the accuracy score, usability score, and logic score, a weighted calculation is performed, and the quality score of the initial prompt P3 is 8.33.

[0080] Optionally, each initial prompt is screened according to the quality score to obtain candidate prompts, including: using the initial prompt with the highest quality score as the candidate prompt.

[0081] In some embodiments, the quality score of the initial prompt P3 is 8.33, which is higher than the quality scores of the initial prompts P1 and P2. Therefore, the initial prompt P3 is used as the candidate prompt.

[0082] Optionally, each initial prompt is screened according to the quality score to obtain candidate prompts, including: using the initial prompt with a quality score greater than or equal to a preset quality threshold as the candidate prompt.

[0083] In some embodiments, the preset quality threshold is 8. Since the quality score of the initial prompt P1 is less than the preset quality threshold, and the quality scores of the initial prompts P2 and P3 are both greater than the preset quality threshold, the initial prompts P2 and P3 are used as the candidate prompts.

[0084] Optionally, by scoring the Q&A effect of the alternative prompt words, an effect score is obtained, including: determining the model output result corresponding to the alternative prompt words using a preset Q&A model, and scoring the model output result using a preset result scoring model to obtain the effect score.

[0085] In some embodiments, the result scoring model is established based on user requirements; the effects of the alternative prompt words and their model output results are comprehensively scored through one or more result scoring models to obtain an effect score, which is used to characterize whether the Q&A of the alternative prompt words meets user requirements; the alternative prompt words before and after optimization are screened by the effect score.

[0086] Optionally, by scoring the Q&A effect of the alternative prompt words, an effect score is obtained, including: collecting user feedback data corresponding to the alternative prompt words to score according to the user feedback data to obtain the effect score, where the user feedback data includes at least one of the number of user modifications, the number of user uses, user positive evaluations, and user negative evaluations.

[0087] In some embodiments, the number of user modifications and user negative evaluations have a negative correlation with the effect score, and the number of user uses and user positive evaluations have a positive correlation with the effect score.

[0088] Combined with Figure 4 As shown, in step S103, using the effect score as the optimization criterion, the text of the alternative prompt words is optimized, including:

[0089] Step S10301, in response to the alternative prompt words, comparing the effect score corresponding to the alternative prompt words according to a preset effect threshold;

[0090] Wherein, the preset effect threshold is 8;

[0091] Step S10302, if the alternative prompt words meet the first preset condition and the second preset condition, set the text optimization result of the alternative prompt words to optimization completed;

[0092] Wherein, the first preset condition includes that the effect score is less than the preset effect threshold;

[0093] Wherein, the second preset condition includes that the number of text optimization times of the alternative prompt words is less than the preset optimization times threshold;

[0094] Among them, the alternative prompt words with an effect score greater than or equal to 8 are used as the question prompt words of the target question and added to the question prompt word library;

[0095] Step S10303, if the alternative prompt words do not meet the first preset condition, optimize the text of the alternative prompt words to obtain new alternative prompt words;

[0096] Among them, if the effect score of the alternative prompt word is less than 8, a new round of text optimization is performed on the alternative prompt word to generate a new alternative prompt word until the preset effect threshold or the optimization times threshold is reached;

[0097] Step S10304, if the alternative prompt word does not meet the second preset condition, set the text optimization result of the alternative prompt word as optimization failure.

[0098] Optionally, the text of the alternative prompt word is optimized in the following way: perform text processing on the alternative prompt word, where the text processing includes at least one of removing stop words, replacing synonyms, replacing near-synonyms, and adjusting the text order.

[0099] In some embodiments, training is performed based on a large language model to obtain a prompt word optimization model, which tentatively optimizes the alternative prompt word through a preset optimization strategy to generate one or more new alternative prompt words, where the preselected optimization strategy includes at least one of removing stop words, replacing synonyms, replacing near-synonyms, and adjusting the text order.

[0100] Optionally, the text of the alternative prompt word is optimized in the following way: use a prompt word scoring model to output the quality evaluation corresponding to the alternative prompt word, where the prompt word scoring model is used to score the text quality of each initial prompt word; perform text parsing on the quality evaluation to extract the model positive evaluation and the model negative evaluation from the quality evaluation according to the text parsing result; generate a prompt word generation text based on at least one of the alternative prompt word, the model positive evaluation, and the model negative evaluation, and input the prompt word generation text into a prompt word generation model, so that the prompt word generation model outputs a new alternative prompt word based on the model positive evaluation and / or the model negative evaluation.

[0101] In some embodiments, the alternative prompt word is "Please tell me about the latest developments in artificial intelligence", and the quality evaluation includes semantic clarity, high relevance, but the semantics is slightly broad. Through analysis, the model positive evaluation is obtained as semantic clarity and high relevance, while the model negative evaluation includes that the semantics is slightly broad; based on the alternative prompt word and the model evaluation, a prompt word generation text is generated, for example, "Please design a prompt word about the latest developments in artificial intelligence, requiring semantic clarity and more pertinence", input the above text into the prompt word generation model, and get "Please share the latest breakthroughs in the field of artificial intelligence in healthcare". In this way, in each iteration, the direction and focus of the prompt word can be adjusted according to the positive and negative evaluations of the model, so that the prompt word is significantly improved in terms of semantic clarity, relevance, fluency, and innovation.

[0102] Optionally, after generating the question prompt words corresponding to the target question according to the alternative prompt words completed by optimization, the method further includes: presenting the question prompt words to the user; if a modification instruction input by the user is received, modifying the question prompt words according to the prompt word modification instruction; if a selection instruction input by the user is received, inputting the question prompt words into a preset question and answer model, so that the preset question and answer model outputs the question and answer result corresponding to the target question.

[0103] In some embodiments, the question prompt words are presented to the user through the platform home page or the main platform page. After the user views the prompt words, they can select and modify them by clicking, which helps the user generate higher-quality and more expected outputs.

[0104] In some embodiments, a display area is designed on the page of user terminals such as vehicle terminals and mobile terminals to display the question prompt words in the question prompt word library; the user feedback times for the question prompt words are recorded.

[0105] In some embodiments, the target question includes "What is the solenoid valve on a new energy vehicle for?", and the question prompt words corresponding to the target question include "You are a senior expert in the automotive industry, with in-depth understanding and rich experience in automotive technology, proficient in various automotive knowledge and technologies such as automotive common sense, automotive principles, automotive maintenance, and automotive design. Now, please give a detailed and professional answer to the following question about automobiles". The user obtains the question and answer result of the target question by selecting the question prompt words corresponding to the target question.

[0106] Combined with Figure 5 As shown, a method for generating question prompt words includes:

[0107] Step S501, obtaining a plurality of historical questions input by the user;

[0108] Step S502, grouping each historical question according to the text similarity between historical questions to obtain similar question groups;

[0109] Step S503, determining high-frequency questions according to the historical questions in the similar question groups, and forming a high-frequency question library according to the high-frequency questions;

[0110] Step S504, determining a target question from the high-frequency question library;

[0111] Step S505, inputting the target question into a plurality of preset prompt word generation models to obtain initial prompt words respectively output by each prompt word generation model;

[0112] Step S506, obtaining the quality scores respectively corresponding to each initial prompt word by scoring the text quality of each initial prompt word;

[0113] Step S507, use the initial prompt words with a mass fraction greater than or equal to the preset mass threshold as alternative prompt words;

[0114] Step S508, obtain an effect score by scoring the Q&A effect of the alternative prompt words;

[0115] Step S509, determine whether the effect score is greater than or equal to the preset effect threshold. If so, jump to step S510; if not, jump to step S513;

[0116] Step S510, use the current alternative prompt word as the question prompt word corresponding to the target question;

[0117] Step S511, display the question prompt word to the user;

[0118] Step S512, in response to the selection instruction, input the question prompt word into the preset Q&A model so that the preset Q&A model outputs the Q&A result corresponding to the target question.

[0119] Step S513, optimize the text of the alternative prompt word to obtain a new alternative prompt word, and jump to step S507.

[0120] Using the question prompt word generation method provided by the present application, by scoring the text quality of the initial prompt words, alternative prompt words are selected from the initial prompt words according to the quality scores, and by scoring the Q&A effect of the alternative prompt words, with the effect score as the optimization criterion, the text of the alternative prompt words is optimized, so as to generate the question prompt word corresponding to the target question according to the optimized alternative prompt words. In this way, the text quality of the prompt words themselves is evaluated according to the quality scores, so as to select prompt words with high text quality, and the Q&A effect of the prompt words during use is evaluated according to the effect scores, and the prompt words are optimized based on the effect scores, so that the automatically generated prompt words are guaranteed in terms of text quality and Q&A effect, improving the generation quality of the question prompt words, and further improving the output accuracy of the large language model.

[0121] Combined Figure 6 As shown in

[0122] The acquisition module 601 is configured to acquire a plurality of initial prompt words corresponding to the target question.

[0123] The screening module 602 is configured to obtain the quality scores corresponding to the respective initial prompt words by scoring the text quality of the respective initial prompt words, and screen the respective initial prompt words according to the quality scores to obtain alternative prompt words.

[0124] The optimization module 603 is configured to score the Q&A effects of alternative prompt words to obtain effect scores, and use the effect scores as the optimization criteria to optimize the texts of the alternative prompt words.

[0125] The generation module 604 is configured to generate a question prompt word corresponding to the target question according to the optimized alternative prompt word.

[0126] By using the question prompt word generation system provided in this application, the text quality of the initial prompt word is scored, alternative prompt words in the initial prompt words are screened out according to the quality scores, and the Q&A effects of the alternative prompt words are scored. Taking the effect scores as the optimization criteria, the texts of the alternative prompt words are optimized, so as to generate a question prompt word corresponding to the target question according to the optimized alternative prompt word. In this way, the text quality of the prompt word itself is evaluated according to the quality score, so as to screen out prompt words with high text quality, and the Q&A effect of the prompt word during use is evaluated according to the effect score, and the prompt word is optimized based on the effect score, so that the automatically generated prompt word is guaranteed in terms of text quality and Q&A effect, improving the generation quality of the question prompt word, and further improving the output accuracy of the large language model.

[0127] This application also provides an electronic device, including: a processor and a memory; the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory, so that the electronic device executes the above method.

[0128] Figure 7 The structure diagram of a computer system of an electronic device suitable for implementing the embodiments of this application is shown. It should be noted that Figure 7 The shown computer system 700 of the electronic device is only an example, and should not bring any limitations to the functions and usage ranges of the embodiments of this application.

[0129] As Figure 7 shown, the computer system 700 includes a central processing unit (CPU) 701, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 702 or the program loaded from the storage section 708 into the random access memory (RAM) 703, such as executing the method in the above embodiments. In the random access memory 703, various programs and data required for system operation are also stored. The central processing unit 701, the read-only memory 702, and the random access memory 703 are connected to each other through a bus 704. The input / output (I / O) interface 705 is also connected to the bus 704.

[0130] The following components are connected to the I / O interface 705: an input part 706 including a keyboard, a mouse, etc.; an output part 707 including, for example, a cathode ray tube (CRT), a liquid crystal display (LCD), etc. and a speaker, etc.; a storage part 708 including a hard disk, etc.; and a communication part 709 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication part 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the I / O interface 705 as required. A removable medium 711, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is mounted on the drive 710 as required so that a computer program read from it can be installed into the storage part 708 as required.

[0131] The electronic device disclosed in this embodiment includes a processor, a memory, a transceiver, and a communication interface. The memory and the communication interface are connected to the processor and the transceiver and complete communication with each other. The memory is used to store a computer program, the communication interface is used for communication, and the processor and the transceiver are used to run the computer program so that the electronic device executes each step of the above method.

[0132] The above description and drawings fully disclose embodiments of the present disclosure, enabling those skilled in the art to practice them. Other embodiments may include structural, logical, electrical, process, and other changes. Embodiments merely represent possible variations. Unless explicitly required, individual components and functions are optional, and the order of operations may vary. Parts and sub-samples of some embodiments may be included in or replace parts and sub-samples of other embodiments. Moreover, the terms used in this application are only for describing embodiments and do not limit the claims. As used in the description of embodiments and claims, unless the context clearly indicates otherwise, the singular forms "a", "an", and "the" are intended to also include the plural forms. Similarly, as used in this application, the term "and / or" refers to any and all possible combinations of one or more of the associated listed items. Additionally, when used in this application, the term "comprise" and its variants "comprises" and / or "comprising" etc. mean the presence of the stated sub-samples, wholes, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other sub-samples, wholes, steps, operations, elements, components, and / or groupings of these. Without further limitation, an element defined by the statement "comprising one..." does not exclude the presence of additional identical elements in the process, method, or device comprising the element. Herein, each embodiment may focus on the differences from other embodiments, and the same or similar parts among the embodiments may be referred to each other. For the methods, products, etc. disclosed in the embodiments, if they correspond to the method parts disclosed in the embodiments, the relevant parts may refer to the description of the method parts.

[0133] Those skilled in the art will realize that the units and algorithm steps of each example described in connection with the embodiments disclosed herein can be implemented in electronic hardware, or in a combination of computer software and electronic hardware. Whether these functions are executed in hardware or software can depend on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods for each specific application to implement the described functions, but such implementation should not be considered to exceed the scope of this application. Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the systems, devices, and units described above can refer to the corresponding processes in the foregoing method embodiments, and will not be elaborated herein.

[0134] In the embodiments disclosed herein, the disclosed methods, products (including but not limited to devices, equipment, etc.) can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of units can be merely a logical function division. In actual implementation, there can be other division methods. For example, multiple units or components can be combined or integrated into another system, or some sub-samples can be ignored or not executed. Additionally, the couplings or direct couplings or communication connections shown or discussed between each other can be through some interfaces. The indirect couplings or communication connections of devices or units can be in electrical, mechanical, or other forms. The units described as separate components can be or can not be physically separated. The components shown as units can be or can not be physical units, that is, they can be located in one place, or can be distributed to multiple network units. Some or all of the units can be selected according to actual needs to implement this embodiment. Additionally, in the present application, each functional unit can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit.

[0135] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of the systems, methods, and computer program products according to the present application. In this regard, each block in the flowchart or block diagram can represent a module, a program segment, or a part of code that contains one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions marked in the block can occur in a different order than that marked in the accompanying drawings. For example, two consecutive blocks can actually be executed substantially in parallel, and they can sometimes be executed in the reverse order, which can depend on the functions involved. In the descriptions corresponding to the flowcharts and block diagrams in the accompanying drawings, the operations or steps corresponding to different blocks can also occur in a different order than that disclosed in the description. Sometimes, there is no specific order between different operations or steps. For example, two consecutive operations or steps can actually be executed substantially in parallel, and they can sometimes be executed in the reverse order, which can depend on the functions involved. Each block in the block diagram and / or flowchart, as well as the combination of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system for performing the specified functions or actions, or can be implemented by a combination of dedicated hardware and computer instructions.

Claims

1. A method for generating problem prompting words, characterized in that, Including: Obtain multiple initial prompt words corresponding to the target question; By scoring the text quality of each of the initial prompt words, obtain the quality scores respectively corresponding to each of the initial prompt words, and filter each of the initial prompt words according to the quality scores to obtain candidate prompt words; By scoring the Q&A effect of the candidate prompt words, obtain the effect scores, and use the effect scores as the optimization criteria to optimize the text of the candidate prompt words; Generate the question prompt words corresponding to the target question according to the candidate prompt words after optimization; By scoring the text quality of each of the initial prompt words, obtain the quality scores respectively corresponding to each of the initial prompt words, including obtaining multiple prompt word scoring models, where the prompt word scoring models are trained based on large language models; using each of the prompt word scoring models to score the text quality of the initial prompt words according to multiple preset scoring dimensions to obtain the quality scoring results respectively output by each of the prompt word scoring models; combining and calculating each of the quality scoring results to obtain the quality scores.

2. The method according to claim 1, wherein Obtain multiple initial prompt words corresponding to the target question, including: Obtain multiple historical questions input by the user; Group each of the historical questions according to the text similarity between the historical questions to obtain similar question groups; If the number of historical questions in the similar question group is greater than or equal to the preset quantity threshold, determine the high-frequency questions according to the historical questions in the similar question group, so as to form a high-frequency question library according to the high-frequency questions; Determine the target question from the high-frequency question library, and input the target question into multiple preset prompt word generation models to obtain the initial prompt words respectively output by each of the prompt word generation models, where the prompt word generation models are trained based on large language models.

3. The method according to claim 1, wherein Use each of the prompt word scoring models to score the text quality of the initial prompt words according to multiple preset scoring dimensions to obtain the quality scoring results respectively output by each of the prompt word scoring models, including: Use each of the prompt word scoring models to evaluate the rationality of the initial prompt words; If the initial prompt word does not meet the rationality, the initial prompt word does not have a quality score; If the initial prompt word meets the rationality, use each of the prompt word scoring models to score the text quality of the initial prompt words according to the preset scoring dimensions to obtain the quality scoring results respectively output by each of the prompt word scoring models.

4. The method according to claim 1, characterized in that, Filter each of the initial prompt words according to the quality scores to obtain candidate prompt words, including at least one of the following: Use the initial prompt word with the highest quality score as the candidate prompt word; Use the initial prompt words with quality scores greater than or equal to the preset quality threshold as the candidate prompt words.

5. The method according to claim 1, wherein By scoring the Q&A effect of the candidate prompt words, obtain the effect scores, including at least one of the following: Use a preset Q&A model to determine the model output result corresponding to the candidate prompt word, and use a preset result scoring model to score the model output result to obtain the effect score; Collect the user feedback data corresponding to the alternative prompt words, and score according to the user feedback data to obtain an effect score, where the user feedback data includes at least one of the number of user modifications, the number of user usages, user positive evaluations, and user negative evaluations.

6. The method according to claim 1, wherein Taking the effect score as the optimization criterion, perform text optimization on the alternative prompt words, including: In response to the alternative prompt word, compare the effect score corresponding to the alternative prompt word according to a preset effect threshold; If the alternative prompt word meets the first preset condition and the second preset condition, set the text optimization result of the alternative prompt word to optimization completed, where the first preset condition includes that the effect score is less than the preset effect threshold, and the second preset condition includes that the number of text optimization times of the alternative prompt word is less than the preset optimization times threshold; If the alternative prompt word does not meet the first preset condition, perform text optimization on the alternative prompt word to obtain a new alternative prompt word; If the alternative prompt word does not meet the second preset condition, set the text optimization result of the alternative prompt word to optimization failed.

7. The method according to claim 1, wherein Perform text optimization on the alternative prompt word through at least one of the following methods: Perform text processing on the alternative prompt word, where the text processing includes at least one of removing stop words, replacing synonyms, replacing near-synonyms, and adjusting the text order; Use the prompt word scoring model to output the quality evaluation corresponding to the alternative prompt word, where the prompt word scoring model is used to score the text quality of each initial prompt word; perform text parsing on the quality evaluation to extract the model positive evaluation and the model negative evaluation from the quality evaluation according to the text parsing result; generate a prompt word generation text according to at least one of the alternative prompt word, the model positive evaluation, and the model negative evaluation, and input the prompt word generation text into the prompt word generation model, so that the prompt word generation model outputs a new alternative prompt word based on the model positive evaluation and / or the model negative evaluation.

8. The method according to any one of claims 1 to 7, characterized in that, After generating the question prompt word corresponding to the target question according to the optimized alternative prompt word, the method further includes: Display the question prompt word to the user; If a modification instruction input by the user is received, modify the question prompt word according to the prompt word modification instruction; If a selection instruction input by the user is received, input the question prompt word into a preset question and answer model, so that the preset question and answer model outputs the question and answer result corresponding to the target question.

9. A problem prompt word generation system, characterized in that, Includes: An acquisition module configured to acquire a plurality of initial prompt words corresponding to a target question; A screening module configured to score the text quality of each initial prompt word to obtain the quality score corresponding to each initial prompt word, and screen each initial prompt word according to the quality score to obtain alternative prompt words; An optimization module configured to score the question and answer effect of the alternative prompt word to obtain an effect score, and use the effect score as an optimization criterion to perform text optimization on the alternative prompt word; A generation module configured to generate a question prompt corresponding to the target question according to the optimized alternative prompt words; The screening module scores the text quality of each of the initial prompt words in the following manner to obtain multiple prompt word scoring models, where the prompt word scoring models are trained based on a large language model; uses each of the prompt word scoring models to score the text quality of the initial prompt words according to multiple preset scoring dimensions to obtain quality scoring results respectively output by each of the prompt word scoring models; and calculates by combining each of the quality scoring results to obtain a quality score.

10. An electronic device, characterized in that, Comprising: A processor and a memory; The memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory so that the electronic device executes the method according to any one of claims 1 to 8.

Images