Prompt word optimization method and system, storage medium and program product

Abstract

The invention relates to the technical field of computers, in particular to a cue word optimization method, a cue word optimization system for implementing the method, a computer readable storage medium and a computer program product. The method comprises the following steps: in response to receiving an initial query input by a user, determining a user intention; determining a representative question and answer pair corresponding to the user intention, wherein the representative question and answer pair is a structural text used for helping the large model to understand an exemplary query problem and generating a corresponding structured query language statement; judging whether missing key information exists or not according to the intention of the user; in response to the judgment that the missing key information exists, outputting prompt information to guide a user to supplement; and integrating all the user inputs and the representative question and answer pairs into optimized cue words for the large model to generate answers.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and more particularly to a prompt word optimization method, a prompt word optimization system for implementing the method, a computer-readable storage medium, and a computer program product. Background Art

[0002] Generative AI (Artificial Intelligence), particularly technologies based on large language models (LLMs), such as the GPT series and BERT, has made significant progress in natural language processing (NLP), demonstrating particularly strong capabilities in text generation tasks. These models are capable of generating fluent, natural-sounding text and can even mimic human writing styles in specific areas. In practical applications, such as data analysis and intelligent customer service, the quality of generative AI prompts directly impacts the performance of large models in question-and-answering.

[0003] Currently, most solutions rely on users directly inputting prompts into a large model, which then generates answers based on its existing data. However, this approach has significant drawbacks. For one thing, non-technical personnel, unfamiliar with the use of prompts in large models, often don't know how to optimize them, leading to omissions of key words or inaccurate descriptions. This directly reduces the accuracy of the content generated by the large model and creates a poor user experience. Furthermore, large models lack sufficient context and problem-solving strategies, resulting in answers that lack logic, reference, and accuracy, limiting their quality.

[0004] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present application, and therefore may include information that does not constitute prior art known to ordinary technicians in this field. Summary of the Invention

[0005] In order to solve or at least alleviate one or more of the above problems, embodiments of the present application provide a prompt word optimization method, a prompt word optimization system for implementing the method, a computer-readable storage medium, and a computer program product, which can significantly improve the ability of large models to understand user intent, generate accurate structured query language statements, and provide high-quality answers.

[0006] According to the first aspect of the present application, a prompt word optimization method is provided, which includes the following steps: determining the user intent in response to receiving an initial query input by a user; determining a representative question-answer pair corresponding to the user intent, wherein the representative question-answer pair is a structured text used to help a large model understand an exemplary query question and generate a corresponding structured query language statement; judging whether there is missing key information based on the user intent; in response to determining that there is missing key information, outputting prompt information to guide the user to supplement it; and integrating all user inputs and the representative question-answer pairs into optimized prompt words for the large model to generate an answer.

[0007] As an alternative or supplement to the above scheme, in a prompt word optimization method according to an embodiment of the present application, the representative question-answer pair includes: a representative question, which includes a description of an exemplary query question, specific requirements for generating a structured query language statement, and a chain reasoning process; and a representative answer, which includes a structured query language statement generated for the exemplary query question.

[0008] As an alternative or supplement to the above scheme, the prompt word optimization method according to an embodiment of the present application also includes: determining exemplary query questions that can represent each predefined user intent category based on coverage indicators and uniqueness indicators; determining specific requirements, which are instructions guiding the large model on how to convert user intent into executable structured query language statements; using zero-sample chain thinking technology to determine a chain reasoning process, which includes chain steps of a step-by-step reasoning process from the user intent to generating a structured query language statement; integrating the description of the exemplary query question, the specific requirements and the chain reasoning process into a structured text as a representative question in the representative question-answer pair.

[0009] As an alternative or supplement to the above scheme, in the prompt word optimization method according to one embodiment of the present application, the coverage index is the ratio between the number of patterns that the selected exemplary query questions can cover under a specific user intent category and the total number of exemplary query questions; and the uniqueness index is the degree of dissimilarity between the exemplary query questions under the specific user intent category.

[0010] As an alternative or supplement to the above scheme, the prompt word optimization method according to an embodiment of the present application also includes: generating multiple answer cases for the representative question using the large model, wherein each answer case is a structured query language statement for an exemplary query question; selecting the best answer case from the multiple answer cases based on the reasoning quality; and combining the representative question and the best answer case into a representative question-answer pair.

[0011] As an alternative or supplement to the above solution, in a prompt word optimization method according to an embodiment of the present application, the reasoning quality includes the rationality, coherence and relevance indicators of the answer to the question.

[0012] As an alternative or supplement to the above scheme, in a prompt word optimization method according to an embodiment of the present application, in response to receiving an initial query input by a user, determining the user intent includes: extracting keywords from the initial query using natural language processing technology; and inputting the extracted keywords into a pre-trained intent recognition model to determine the user intent.

[0013] As an alternative or supplement to the above scheme, in a prompt word optimization method according to an embodiment of the present application, judging whether there is missing key information based on the user intent includes: determining a structured description for the user intent, wherein the structured description includes multiple intent elements; and determining whether the query input by the user includes all intent elements.

[0014] As an alternative or supplement to the above scheme, in a prompt word optimization method according to an embodiment of the present application, integrating all user inputs and the representative question-answer pairs into optimized prompt words is performed in response to one of the following judgments: a judgment that there is no missing key information; a judgment that the current user input includes an expression to end the conversation; a judgment that the preset maximum number of conversation rounds has been reached; a judgment that the user has not responded for a long time.

[0015] As an alternative or supplement to the above scheme, the prompt word optimization method according to an embodiment of the present application also includes: after receiving supplementary information input by the user, determining whether the supplementary keywords extracted from the supplementary information correspond to the same user intent as the keywords extracted from the initial query; and if they do not correspond to the same user intent, outputting prompt information to guide the user to supplement information related to the previously determined user intent.

[0016] As an alternative or supplement to the above scheme, in a prompt word optimization method according to an embodiment of the present application, integrating all user inputs and the representative question-answer pairs into optimized prompt words includes: integrating all user inputs into the user's final question statement; and adding the final question statement and the representative question-answer pairs to corresponding parts of the prompt word template with descriptive sentences.

[0017] As an alternative or supplement to the above solution, the prompt word optimization method according to an embodiment of the present application further includes: inputting the optimized prompt word into a large model, and the large model generates the answer using retrieval enhancement generation technology.

[0018] According to a second aspect of the present application, a prompt word optimization system is provided, comprising: a memory; a processor; and a computer program stored on the memory and executable on the processor, wherein the execution of the computer program causes any one of the prompt word optimization methods described in the first aspect of the present application to be executed.

[0019] According to a third aspect of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium includes instructions, and the instructions, when run, execute any one of the prompt word optimization methods according to the first aspect of the present application.

[0020] According to a fourth aspect of the present application, a computer program product is provided, comprising a computer program. When the computer program is executed by a processor, the computer program implements any one of the prompt word optimization methods described in the first aspect of the present application.

[0021] According to one or more embodiments of the present application, the prompt word optimization scheme introduces representative question-answer pairs as a template for large-scale model learning, which not only provides exemplary query questions, but also demonstrates how to generate corresponding structured query language statements. This enables the large model to learn to imitate the correct question and answer patterns and apply them to new queries. At the same time, the representative question-answer pairs also provide the large model with contextual information and problem-solving logic, enabling the large model to better understand the deep meaning behind the user's intentions and improve the accuracy of understanding. In addition, the scheme also has the ability to identify missing key information. It can actively determine whether key information is missing in the user input and guide the user to supplement it, thereby avoiding inaccurate answers due to incomplete information. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The above and / or other aspects and advantages of the present application will become clearer and easier to understand through the following description of various aspects in conjunction with the accompanying drawings, in which the same or similar elements are represented by the same reference numerals. In the drawings:

[0023] Figure 1 is a schematic flow chart of a prompt word optimization method 10 according to one or more embodiments of the present application;

[0024] Figure 2 is a schematic flow chart of a method 20 for presetting a representative question-answer pair according to one or more embodiments of the present application;

[0025] Figure 3 is a schematic flow chart of a prompt word optimization method 30 according to one or more embodiments of the present application; and

[0026] Figure 4 FIG. 4 is a schematic block diagram of a prompt word optimization system 40 according to one or more embodiments of the present application. DETAILED DESCRIPTION

[0027] The description of the following specific embodiments is merely exemplary in nature and is not intended to limit the disclosed technology or the application and use of the disclosed technology. In addition, there is no intention to be bound by any express or implied theory presented in the foregoing technical field, background technology or the following specific embodiments.

[0028] In the following detailed description of the embodiments, numerous specific details are set forth to provide a more thorough understanding of the disclosed technology. However, it will be apparent to one of ordinary skill in the art that the disclosed technology can be practiced without these specific details. In other instances, well-known features are not described in detail to avoid unnecessarily complicating the description.

[0029] Terms such as "comprising" and "including" indicate that, in addition to the units and steps directly and explicitly stated in the specification, the technical solution of this application does not exclude the possibility of having other units and steps not directly or explicitly stated. Terms such as "first" and "second" do not indicate the order of units in terms of time, space, size, etc., but are merely used to distinguish between units.

[0030] In addition, it should be noted that the user information and data involved in one or more embodiments of the present application (including but not limited to data used for analysis, stored data, displayed data, etc.) are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0031] The proposed prompt word optimization solution has broad applicability and can be applied to prompt word generation scenarios for any type of large model. Whether the model is used in education, finance, law, image processing, or any other field, this solution can generate optimized prompt words suitable for the model in that field. Whether it is a large language model deployed for the first time or a model redeployed after optimization, both may face the problem of low prompt word quality. This solution can effectively solve this problem, providing the model with accurate and efficient prompt words.

[0032] Prompts in this article refer to text entered to guide the large model to complete a specific task. Specifically, prompts are questions, instructions, or requests posed to the large model. They can take the form of phrases, sentences, paragraphs, or even code or structured data. Functionally, prompts are the primary way users express their needs and intentions to the large model. They can trigger specific functions or operations, such as information retrieval, text translation, and code generation. The quality of prompts directly affects the output of the large model; high-quality prompts can provide more accurate and efficient responses.

[0033] Hereinafter, various exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings.

[0034] Referring to the accompanying drawings, Figure 1 1 is a schematic flow chart of a prompt word optimization method 10 according to one or more embodiments of the present application.

[0035] like Figure 1 As shown, in step 101 , in response to receiving an initial query input by a user, the user intent is determined.

[0036] The core idea of step 101 is to use the initial query provided by the user in the first round of dialogue to determine the user's intention. By analyzing the initial query in the first round of dialogue, the system can extract the user's core needs, such as whether it needs to summarize data, list data in detail, or conduct comparative analysis. It should be emphasized here that the determination of user intention is directly based on the user's first input. The intention determination process is independent of subsequent dialogue rounds. User intention is the key to success. Figure 1 Once determined, the intent remains unaffected by subsequent conversations unless the user actively changes it. It's foreseeable that basing user intent determination on multiple rounds of conversations could lead to intent drift, reducing accuracy. However, determining user intent based on the initial query ensures a clearer and more focused goal for subsequent steps, improving system responsiveness and efficiency.

[0037] In some embodiments according to the present application, first, the large model can use its natural language processing (NLP) capabilities to extract keywords from the initial query input by the user. This step can be regarded as a preliminary analysis of the natural language text, the purpose of which is to decompose the complex natural language text into representative words or phrases for subsequent intent analysis. Exemplarily, the large model can assist in keyword extraction through technologies such as word segmentation, stop word removal, part-of-speech tagging, and named entity recognition. Part-of-speech tagging can help the system identify key components such as nouns and verbs, while named entity recognition can help the system identify entity information such as names of people, places, and organizations, which are often the key to understanding user intentions. For example, for the prompt word "I want to inquire about the sales of last quarter", NLP technology can be used to extract keywords such as "query", "last quarter" and "sales". In addition, the large model can also use the attention mechanism to extract keywords, which can more flexibly and accurately identify important words in the text. Further,

[0038] Then, all the keywords extracted from the initial query can be input into the pre-trained intent recognition model to determine the user intent based on these keywords. The intent recognition model here can be regarded as part of the large model (that is, the intent recognition function within the large model), or implemented by the large model, and its function is to map keywords to predefined user intents. Based on a large amount of training data, the model has learned the correspondence between different keywords and specific user intents. Intent categories can be pre-defined according to actual application scenarios, such as "summary query", "list query", "comparison query", etc. Through the processing of the intent recognition model, the system can quickly and accurately determine the query intent raised by the user in the first round of conversation, and use it as the output result to provide accurate guidance for the subsequent prompt word optimization step. Exemplarily, the pre-trained intent recognition model can be implemented using deep learning methods such as FastText or BERT to improve the accuracy and efficiency of intent recognition.

[0039] In step 103 , a representative question-answer pair corresponding to the user's intention is determined.

[0040] The core of this step is to select structured text that can serve as a learning example for the large model based on the user's intention. The structured text is defined as a representative question-answer pair. Representative question-answer pairs are the core component of the method of this application. They can not only provide exemplary query questions, but also demonstrate how the corresponding structured query language statements are generated. Its purpose is to help the large model learn to imitate the correct question and answer patterns by providing clear examples, thereby improving the ability of the large model to generate structured query language statements and applying them to new queries. Therefore, representative question-answer pairs play a vital role in the entire optimization process, and their quality directly affects the accuracy and efficiency of the final generated structured query statements.

[0041] In some embodiments of the present application, each representative question-answer pair consists of two parts: a representative question (Q) and a representative answer (A). The representative question (Q) may include the following structural elements: a description of an exemplary query question (Input) to clarify the user's query intent; specific requirements for generating a structured query language statement (Instruction) to guide the large model on how to convert the user's intent into an executable structured query language statement (e.g., SQL statement); a chain reasoning process (Thought Chain) to demonstrate the step-by-step reasoning logic from user intent to generating a structured query language statement. Optionally, the representative question (Q) may also include a data context (Context) to describe the structure and field information of the data table, providing the large model with the data background required for the query; and / or style information (Style) to specify the code style of the structured query language statement generated by the large model, such as concise, clear, easy to understand, etc. The representative answer (A) may include a structured query language statement (e.g., SQL statement) generated for the exemplary query question, which is the final result generated based on information such as the intent, specific requirements, and chain reasoning process described in the representative question (Q).

[0042] In order to more specifically explain how to select high-quality representative question-answer pairs, this application provides the following optional implementation methods.

[0043] First, based on coverage and uniqueness metrics, exemplary queries that represent each predefined user intent category are identified. The coverage metric measures the extent to which the selected exemplary queries represent a specific user intent category. Exemplarily, the coverage metric can be calculated as the ratio of the number of patterns covered by the selected exemplary queries for a specific user intent category to the total number of exemplary queries for that user intent category. For example, if there are 10 different query patterns for the "aggregate query" intent category, and the selected exemplary query covers 8 of them, the coverage metric for the exemplary query is 80%. The uniqueness metric measures the degree of dissimilarity between the selected exemplary queries for a specific user intent category. The uniqueness metric aims to avoid selecting overly repetitive or similar exemplary queries, ensuring diversity and coverage of the examples. For example, if two very similar exemplary queries, such as "Query the transaction amount of the Shanghai branch in February" and "Query the total transaction amount of the Shanghai branch in February," have very similar patterns, their uniqueness metrics will be low. By comprehensively considering coverage and uniqueness metrics, we can effectively select exemplary query questions that can represent each predefined user intent category, ensuring the representativeness and diversity of the examples.

[0044] After selecting an example query question, the next step is to determine the specific instructions for generating a structured query language statement. Specific instructions are instructions used to guide the large model on how to convert user intent into executable structured query language statements. These instructions may include the table name, field name, aggregate function, and filter conditions to be queried. For example, for the example query question "Query the transaction amount of the Shanghai branch in February," the specific requirements may be: filter out the data of the Shanghai branch in February; calculate the total transaction amount; add comments to improve readability; and ensure that the query statement is concise and clear.

[0045] Next, the Zero-Shot Chain-of-Thought (CoT) technique can be used to determine the thought chain. This chained reasoning process involves a step-by-step process from user intent to the generation of a structured query language statement. This process is broken down into multiple logical steps, such as: determining the filter criteria—the branch name is Shanghai Branch, and the month is February; selecting the field to be calculated—the sum of the transaction amounts; adding a comment to explain the query purpose; and writing and validating the SQL statement. This zero-shot chained thinking technique allows the system to guide the large model's reasoning process without the need for additional training data.

[0046] Optionally, the description, specific requirements, and chained reasoning process for the exemplary query can be integrated into structured text, serving as the representative question (Q) in a representative question-answer pair. The large model is then used to generate multiple possible answer examples based on all the information in the representative question (Q). Each answer example is a structured query language statement, such as an SQL statement, for the selected exemplary query. In this step, the large model may generate multiple different, but potentially correct, answer examples based on different thinking styles.

[0047] Afterward, reasoning quality assessment techniques are used to select the best answer from multiple examples. Specific evaluation criteria include the answer's rationality, coherence, and relevance to the question. The selected best answer serves as the representative answer (A) in a representative question-answer pair. This answer is combined with the representative question (Q) to form the final representative question-answer pair. This representative question-answer pair serves as a learning example for the larger model, helping it better understand user intent and the corresponding structured query language statement generation method.

[0048] In step 105, it is determined whether there is any missing key information based on the user's intention.

[0049] The core of this step is to analyze whether the initial query entered by the user contains all the necessary information to satisfy their intent. Although step 101 has identified the user's initial intent, the user may not have provided complete query information in the first round of conversation. For example, the user may enter "Query the transaction volume of the Shanghai branch" without explicitly specifying the transaction volume for the time period. Therefore, further analysis of the user's query is required to determine whether any key information is missing and to prepare for the subsequent prompt information output.

[0050] In some embodiments of the present application, a structured description may be first defined for the user intent determined in step 101, and the description includes multiple intent elements. Intent elements refer to the key information necessary to meet the user intent. For example, for the intent of "querying transaction amount", its structured description may include: query target (for example, transaction amount), query scope (for example: Shanghai branch), time range (for example: February 2023), and other filtering conditions (for example: specific product type). Each intent element represents a key information point required for the query. The definition of these elements depends on the preliminary analysis of various possible user query intentions. After defining the structured description of the user intent, it is next possible to analyze whether the initial query entered by the user in the first round of conversation contains all the intent elements. For example, the query entered by the user can first be parsed, and the information contained therein can be extracted, and then this information can be compared with the intent elements in the structured description. For example, if a user enters the query "Query the transaction amount of the Shanghai branch," the system can parse out that the "Query Target" is the transaction amount and the "Query Scope" is the Shanghai branch, but lacks the "Time Range" and "Other Filters." If the user's query is missing any one or more intent elements, it can be determined that key information is missing.

[0051] In step 107 , in response to determining that there is missing key information, prompt information is output to guide the user to supplement it.

[0052] The purpose of this step is to output prompts to the user when step 105 determines that key information is missing from the initial query entered by the user, guiding them to provide additional information. This ensures that the large model can generate accurate responses based on complete information. The prompts can be clearly instructive, clearly indicating the type of intent elements the user needs to provide, for example, "For which time period would you like to inquire about transaction volume?" or "For which branch would you like to inquire about revenue?" This step aims to establish a dynamic, interactive dialogue process, gradually guiding users to complete their queries.

[0053] Optionally, after receiving the supplementary prompt word input by the user, an additional verification step may be performed to ensure that the supplementary information provided by the user is valid and relevant. For example, after the user receives the prompt and provides the supplementary information, the system parses the supplementary information and extracts the supplementary keywords contained therein. The system then determines whether these supplementary keywords correspond to the same user intent as the keywords extracted from the initial query. This means that the system not only needs to understand the supplementary information itself but also needs to correlate it with the previously determined user intent to ensure that the supplementary information is targeted at the user intent determined in step 101. For example, if the user initially queries for "Shanghai branch's transaction volume," and after being prompted to add a time range, the user adds "2023," the system should determine that the "2023" keyword aligns with the transaction volume intent of the previous query. However, if the user adds "Beijing branch's profit," the system should determine that this keyword does not align with the previous user intent. If the system determines that the supplementary keywords extracted from the supplementary information do not correspond to the same user intent as the keywords extracted from the initial query, it means that the user may have introduced a new query intent or confused the previous intent. In order to maintain the consistency and accuracy of the query, the system can output new prompt information to guide the user to supplement the information related to the previously determined user intent. For example, if the user adds "I want to know the profit of the Beijing branch" when querying "Shanghai branch transaction volume", the system can prompt "Your previous query was about the transaction volume of the Shanghai branch. Do you need to modify the query scope or supplement the information about the transaction volume of the Shanghai branch?" Through such prompts, the user can be guided back to the previous intent and avoid incorrect query results due to inconsistent intent. In this way, it can ensure that during the dialogue interaction process, the user is guided to supplement the necessary information while avoiding the user from deviating from the previously determined user intent, and ultimately achieve efficient and accurate prompt word optimization.

[0054] In step 109, all user inputs and representative question-answer pairs are integrated into optimized prompt words for the large model to generate answers.

[0055] This step, the final step in the optimization process, aims to integrate all user input (including the initial query and supplementary information) with selected representative question-answer pairs to form a well-structured, informative, and clearly guided prompt, providing sufficient input for the large model to generate high-quality responses. The optimized prompt incorporates representative question-answer pairs, providing the large model with contextual information and problem-solving logic, enabling it to better understand the deeper meaning behind user intent and improve accuracy.

[0056] Some embodiments of the present application limit the timing for prompt word integration. That is, all user inputs and representative question-answer pairs are integrated into optimized prompt words. This is not performed immediately after each conversation, but is triggered only when one of the following conditions is met: 1) It is determined that there is no missing key information. For example, if step 105 determines that the current user input already contains all intent elements and there is no missing key information, prompt word integration can be performed; 2) It is determined that the current user input includes an expression to end the conversation. For example, the key information method can be used in real-time conversations to continuously monitor the conversation content to accurately determine the natural end point of the conversation. If the user clearly indicates that they want to end the conversation in the input, such as using phrases such as "OK" or "No more", prompt word integration can be performed; 3) It is determined that a preset maximum number of conversation rounds has been reached. That is, to prevent the conversation from continuing indefinitely, a maximum number of conversation rounds can be preset. When the number of conversation rounds reaches the preset value, prompt word integration is forced; 4) It is determined that the user has not responded for a long time. That is, if the user has not provided any new input for a long time (for example, 2 minutes), it can be determined that the user has ended the conversation and prompt word integration is performed.

[0057] Furthermore, when prompt integration is triggered, the system can integrate all user input into the user's final question statement: the system integrates all input information provided by the user throughout the conversation to form a coherent and complete user question statement. This includes the initial query, all supplementary information, and any necessary structuring and polishing of this information. For example, if the user's initial query is "Query transaction amount," and after multiple rounds of conversation, "Shanghai branch" and "February 2023" are added, the final question statement can be "Please help me query the transaction amount of the Shanghai branch in February 2023." The final question statement and representative question-answer pairs can then be added to the corresponding sections of a prompt template with explanatory statements. The system can use a predefined prompt template that contains some explanatory statements (such as "Please organize the following user's question into coherent text and give an accurate answer based on this text") and reserves the corresponding sections for the user's final question statement and representative question-answer pairs. The system will fill the corresponding parts of the template with the final integrated question statement and the representative question-answer pairs determined in the previous step, thus forming the final optimized prompt word. For example, the prompt word template may be similar to:

[0058] {

[0059] Sample questions: (Representative question-answer pairs)

[0060] Please organize the following user's question into coherent text and give an accurate answer based on this text.

[0061] Question: "Please help me check the transaction volume of the Shanghai branch in February 2023"

[0062] }

[0063] Unlike simple natural language text prompts, these optimized prompts possess structural characteristics. Through this structured organization, they provide more comprehensive and clearer information to the large model, helping it better understand user intent and generate more accurate and high-quality results. Compared to simple text prompts, these structured prompts are more effective in guiding large models to complete complex tasks.

[0064] After generating optimized prompt words, the system can input them into a pre-trained large model. Furthermore, the large model can employ Retrieval-Augmented Generation (RAG) technology to search external knowledge bases and incorporate the retrieved information into the answer generation process, further improving the quality and accuracy of answers.

[0065] Reference below Figure 2 , Figure 2 2 is a schematic flowchart of a method 20 for presetting representative question-answer pairs according to one or more embodiments of the present application. Figure 2 This paper demonstrates how to pre-determine representative question-answer pairs to support the subsequent prompt word optimization process.

[0066] like Figure 2 As shown, in step 201, exemplary query questions that can represent each predefined user intent category are determined based on the coverage index and the uniqueness index. As described above, coverage is the ratio between the number of patterns that the selected exemplary query questions can cover under a specific user intent category and the total number of exemplary query questions, and uniqueness is the degree of dissimilarity between exemplary query questions under a specific user intent category. Exemplarily, coverage (C) and uniqueness (U) are calculated as follows:

[0067]

[0068] Among them, Nc is the number of patterns that can be covered by the selected exemplary query questions under a specific user intent category. Patterns refer to different query expressions or variations under a specific query intent. For example, for the intent of "querying the transaction amount of the Shanghai branch", there may be multiple expressions such as "querying the total transaction amount of the Shanghai branch", "querying the total transaction amount of the Shanghai branch", and "querying the transaction amount of the Shanghai branch". Nt is the total number of exemplary query questions under this specific user intent category, Sim(c i ,c j ) is case c under this specific user intent categoryi and c j The similarity between them can be calculated by methods such as cosine similarity. Here, c i and c j This can be a vector representing an example query, for example, represented by a word vector or sentence vector. The goal of the coverage metric is to select example queries that maximize the coverage of the various possible query patterns for the intent category. The goal of the uniqueness metric is to select example queries that are as different from each other as possible, thereby ensuring both diversity and coverage of the examples.

[0069] In step 203, specific instructions for generating a structured query language statement are determined.

[0070] In step 205 , a thought chaining process is determined using a zero-shot thought chaining technique.

[0071] In step 207 , the description, specific requirements, and chained reasoning process of the exemplary query question are integrated into a structured text as a representative question (Q) in a representative question-answer pair.

[0072] In step 209, a large model is used to generate multiple answer cases for representative questions, wherein each answer case is a structured query language statement, such as an SQL statement, for the selected exemplary query question.

[0073] In step 211, the best answer case is selected from the multiple answer cases based on the quality of reasoning. The specific evaluation content may include indicators such as the rationality, coherence and relevance of the answer to the question.

[0074] In step 213, the representative questions and the best answer cases are combined into representative question-answer pairs. These representative question-answer pairs will serve as learning examples for the large model, helping it to better understand user intent and the corresponding structured query language statement generation method.

[0075] The following examples illustrate the specific components and structure of representative question-answer pairs.

[0076] As mentioned above, the representative question-answer pair is a structured text used to help the large model understand the exemplary query question and generate the corresponding structured query language statement, which can provide a basis for learning and reasoning for the large model. In order to more clearly illustrate the structure of the representative question-answer pair, a specific summary query is used as an example: Assuming that for a predefined user intent - "summary query", an exemplary query question is set - "Query the year-on-year growth of the Shanghai branch's revenue in 2023", the structured text of the representative question (Q) generated for this exemplary query question can be:

[0077]

[0078] In the above example, the Input (description of the exemplary query question) part clearly expresses the user's summary query intention, that is, to query the year-on-year growth of the Shanghai branch's revenue in 2023. Context (data context) describes the structure of the data table, including the table name revenue, and the fields month, branch, and income, and explains the data type and description information of each field to help the big model understand the data table structure. Instruction (specific requirements) details the specific requirements for the big model to generate SQL statements. Thought Chain (chain reasoning process) breaks down user questions into multiple steps, clearly showing the reasoning process from user intent to SQL generation. Style (style information) specifies the style of SQL statements generated by the big model. Output (output format) clarifies that the big model should output SQL statements.

[0079] For the above representative questions (Q), the structured text of the generated representative answers (A) can be:

[0080]

[0081] In the above example, the representative answer (A) is the final structured query language statement, or SQL statement, generated based on the intentions, specific requirements, and chain reasoning process described in the representative question (Q). The statement first uses two CTEs (Common Table Expressions) to calculate the total revenue of the Shanghai branch in 2023 and 2022, respectively. Then, the COALESCE and NULLIF functions are used to calculate the year-on-year growth rate and handle the case where the total revenue in 2022 is zero. Comments are also included in the SQL statement to improve readability. The above representative answer (A) reflects all the requirements of the representative question (Q), including the use of CTE, the use of the COALESCE function, and the addition of necessary comments.

[0082] Figure 3 FIG. 3 is a schematic flow chart of a prompt word optimization method 30 according to one or more embodiments of the present application.

[0083] In step 301, for each predefined user intent category, one or more representative question-answer pairs are determined. For an exemplary implementation of this step, reference may be made to the description of method 20 above.

[0084] In step 303 , in response to receiving an initial query input by a user, keywords are extracted from the initial query using natural language processing technology.

[0085] In step 305 , the extracted keywords are input into a pre-trained intent recognition model to determine the user intent.

[0086] In step 307 , a representative question-answer pair corresponding to the user's intention is determined.

[0087] In step 309 , a structured description of the user's intention is determined, where the structured description includes multiple intention elements.

[0088] In step 311 , it is determined whether the query input by the user includes all the intent elements. If so, the process proceeds to step 313 ; otherwise, the process proceeds to step 317 .

[0089] In step 317, prompt information is output to guide the user to make supplements.

[0090] In step 313 , all user inputs are integrated into the user's final question statement, and the final question statement and the representative question-answer pair are added to the corresponding parts of the prompt word template with the descriptive sentence.

[0091] In step 315, the optimized prompt words are input into the large model, and the large model generates an answer.

[0092] Figure 4 FIG4 is a schematic block diagram of a prompt word optimization system 40 according to one or more embodiments of the present application. The prompt word optimization system 40 includes a memory 410, a processor 420, and a computer program 430 stored in the memory 410 and executable on the processor 420. The execution of the computer program 430 enables the above-described method 10, 20, or 30 to be executed.

[0093] In addition, as described above, the present application can also be implemented as a computer-readable storage medium, in which is stored information for enabling the Figure 1 A program for executing the process of the method 10, 20, or 30 shown. Here, as the computer-readable storage medium, various computer-readable storage media can be used, such as disks (e.g., magnetic disks, optical disks, etc.), cards (e.g., memory cards, optical cards, etc.), semiconductor memories (e.g., ROMs, non-volatile memories, etc.), and tapes (e.g., magnetic tapes, cassettes, etc.).

[0094] The present application may also be implemented as a computer program product, which includes a computer program. When the computer program is executed by a processor, the following Figure 1 The steps of method 10, 20 or 30 are shown.

[0095] In the applicable situation, the combination of hardware, software or hardware and software can be used to realize the various embodiments provided by the application. Moreover, in the applicable situation, without departing from the scope of the application, the various hardware components and / or software components set forth herein can be combined into a composite component comprising software, hardware and / or both. In the applicable situation, without departing from the scope of the application, the various hardware components and / or software components set forth herein can be divided into a subcomponent comprising software, hardware or both. In addition, in the applicable situation, it is contemplated that the software component can be implemented as a hardware component, and vice versa.

[0096] Software according to the present application (such as program code and / or data) can be stored on one or more computer-readable storage media. It is also contemplated that the software identified herein can be implemented using one or more general or special computers and / or computer systems, networked and / or otherwise. Where applicable, the order of the various steps described herein can be changed, combined into composite steps and / or divided into sub-steps to provide the features described herein.

[0097] The embodiments and examples set forth herein are provided to best illustrate embodiments according to the present application and its specific applications, and thereby enable those skilled in the art to make and use the present application. However, those skilled in the art will appreciate that the above description and examples are provided for ease of illustration and example only. The descriptions set forth are not intended to be exhaustive of all aspects of the present application or to limit the present application to the precise forms disclosed.

Claims

1. A prompt word optimization method, characterized in that: The method comprises the following steps: In response to receiving an initial query input by a user, determining user intent; Determining a representative question-answer pair corresponding to the user intent, wherein the representative question-answer pair is a structured text used to help the large model understand the exemplary query question and generate a corresponding structured query language statement; Determine whether there is missing key information based on the user's intent; In response to determining that there is missing key information, outputting prompt information to guide the user to supplement it; and All user inputs and the representative question-answer pairs are integrated into optimized prompt words for the large model to generate answers.

2. The prompt word optimization method according to claim 1, characterized in that: The representative question-answer pairs include: Representative problems, including descriptions of exemplary query problems, specific requirements for generating structured query language statements, and chained reasoning processes; and A representative answer includes a structured query language statement generated for the exemplary query question.

3. The prompt word optimization method according to claim 1 or 2, characterized in that: The method further comprises: Determine exemplary query questions that can represent each predefined user intent category based on coverage metrics and uniqueness metrics; Determining specific requirements, where the specific requirements are instructions for guiding the large model on how to convert user intent into executable structured query language statements; Determining a chained reasoning process using a zero-shot chaining technique, wherein the chained reasoning process includes chained steps of a step-by-step reasoning process from the user intention to generating a structured query language statement; The description of the exemplary query question, the specific requirements and the chain reasoning process are integrated into a structured text as a representative question in the representative question-answer pair.

4. The prompt word optimization method according to claim 3, characterized in that: The coverage index is the ratio between the number of patterns that the selected exemplary query questions can cover and the total number of exemplary query questions under a specific user intent category; and The uniqueness index is the degree of dissimilarity between exemplary query questions under the specific user intent category.

5. The prompt word optimization method according to claim 3, characterized in that: The method further comprises: For the representative questions, generate multiple answer cases using the large model, wherein each answer case is a structured query language statement for the exemplary query question; selecting the best answer case from the plurality of answer cases according to the reasoning quality; and The representative question and the best answer case are combined into the representative question-answer pair.

6. The prompt word optimization method according to claim 5, characterized in that: The quality of reasoning includes the rationality, coherence and relevance of the answer to the question.

7. The prompt word optimization method according to claim 1, characterized in that: In response to receiving an initial query input by a user, determining user intent includes: extracting keywords from the initial query using natural language processing technology; and The extracted keywords are input into a pre-trained intent recognition model to determine the user intent.

8. The prompt word optimization method according to claim 1 or 7, characterized in that: Based on the user intent, key information to determine whether there is missing information includes: Determining a structured description of the user's intent, wherein the structured description includes a plurality of intent elements; and Determine whether the query entered by the user includes all elements of intent.

9. The prompt word optimization method according to claim 1, characterized in that: Integrating all user inputs and the representative question-answer pairs into optimized prompt words is performed in response to one of the following determinations: Determine that there is no missing critical information; Determining whether the current user input includes an expression for ending the conversation; Determine that the preset maximum number of dialogue rounds has been reached; Determines if the user has been unresponsive for an extended period of time.

10. The prompt word optimization method according to claim 7, characterized in that: The method further comprises: After receiving supplementary information input by the user, determining whether the supplementary keywords extracted from the supplementary information correspond to the same user intent as the keywords extracted from the initial query; and If they do not correspond to the same user intention, prompt information is output to guide the user to supplement information related to the previously determined user intention.

11. The prompt word optimization method according to claim 1, characterized in that: Integrating all user inputs and the representative question-answer pairs into optimized prompt words includes: Consolidate all user input into the user's final question statement; and The final question statement and the representative question-answer pair are respectively added to corresponding parts of the prompt word template with the descriptive sentence.

12. The prompt word optimization method according to claim 1, characterized in that: The method further comprises: The optimized prompt words are input into a large model, and the large model generates the answer using retrieval enhancement generation technology.

13. A prompt word optimization system, characterized in that: The invention comprises: a memory; a processor; and a computer program stored on the memory and executable on the processor, wherein the execution of the computer program causes the following operations: In response to receiving an initial query input by a user, determining user intent; Determining a representative question-answer pair corresponding to the user intent, wherein the representative question-answer pair is a structured text used to help the large model understand the exemplary query question and generate a corresponding structured query language statement; Determine whether there is missing key information based on the user's intent; In response to determining that there is missing key information, outputting prompt information to guide the user to supplement it; as well as All user inputs and the representative question-answer pairs are integrated into optimized prompt words for the large model to generate answers.

14. The prompt word optimization system according to claim 13, characterized in that: The representative question-answer pairs include: Representative problems, including descriptions of exemplary query problems, specific requirements for generating structured query language statements, and chained reasoning processes; and A representative answer includes a structured query language statement generated for the exemplary query question.

15. The prompt word optimization system according to claim 13 or 14, characterized in that: The execution of the computer program also results in the following operations: Determine exemplary query questions that can represent each predefined user intent category based on coverage metrics and uniqueness metrics; Determining specific requirements, where the specific requirements are instructions for guiding the large model on how to convert user intent into executable structured query language statements; Determining a chained reasoning process using a zero-shot chaining technique, wherein the chained reasoning process includes chained steps of a step-by-step reasoning process from the user intention to generating a structured query language statement; The description of the exemplary query question, the specific requirements and the chain reasoning process are integrated into a structured text as a representative question in the representative question-answer pair.

16. The prompt word optimization system according to claim 15, characterized in that: The coverage index is the ratio between the number of patterns that the selected exemplary query questions can cover and the total number of exemplary query questions under a specific user intent category; and The uniqueness index is the degree of dissimilarity between exemplary query questions under the specific user intent category.

17. The prompt word optimization system according to claim 14, characterized in that: The execution of the computer program also results in the following operations: For the representative questions, using the large model, multiple answer cases are generated, wherein each answer case is a structured query language statement for an exemplary query question; selecting the best answer case from the plurality of answer cases according to the reasoning quality; and The representative question and the best answer case are combined into a representative question-answer pair.

18. The prompt word optimization system according to claim 17, characterized in that: The quality of reasoning includes the rationality, coherence and relevance of the answer to the question.

19. The prompt word optimization system according to claim 13, characterized in that: In response to receiving an initial query input by a user, determining user intent includes: extracting keywords from the initial query using natural language processing technology; and The extracted keywords are input into a pre-trained intent recognition model to determine the user intent.

20. The prompt word optimization system according to claim 13 or 19, characterized in that: Based on the user intent, key information to determine whether there is missing information includes: Determining a structured description of the user's intent, wherein the structured description includes a plurality of intent elements; and Determine whether the query entered by the user includes all elements of intent.

21. The prompt word optimization system according to claim 13, characterized in that: Integrating all user inputs and the representative question-answer pairs into optimized prompt words is performed in response to one of the following determinations: Determine that there is no missing critical information; Determining whether the current user input includes an expression for ending the conversation; Determine that the preset maximum number of dialogue rounds has been reached; Determines if the user has been unresponsive for an extended period of time.

22. The prompt word optimization system according to claim 19, characterized in that: The execution of the computer program also results in the following operations: After receiving the supplementary information input by the user, determining whether the supplementary keywords extracted from the supplementary information correspond to the same user intent as the keywords extracted from the initial query; as well as If they do not correspond to the same user intention, prompt information is output to guide the user to supplement information related to the previously determined user intention.

23. The prompt word optimization system according to claim 13, characterized in that: Integrating all user inputs and the representative question-answer pairs into optimized prompt words includes: Consolidate all user input into the user's final question statement; and The final question statement and the representative question-answer pair are respectively added to corresponding parts of the prompt word template with the descriptive sentence.

24. The prompt word optimization system according to claim 13, characterized in that: The execution of the computer program also results in the following operations: The optimized prompt words are input into a large model, and the large model generates the answer using retrieval enhancement generation technology.

25. A computer-readable storage medium, characterized in that The computer-readable storage medium includes instructions, and the instructions, when executed, execute the prompt word optimization method according to any one of claims 1-12.

26. A computer program product, characterized in that The computer program product includes a computer program, and when the computer program is executed by a processor, the computer program implements the steps of the prompt word optimization method according to any one of claims 1 to 12.

Images