An agent prompt word derivation method and system, and a storage medium

Abstract

The application discloses an agent prompt word derivation method and system and a storage medium, and belongs to the technical field of language processing. The method comprises the following steps: performing intention analysis on user input information through a main model to obtain an initial task abstract; determining whether the logic of the initial task abstract is complete through an auxiliary model; if not, revising the initial task abstract through the main model to obtain a target task abstract; converting the target task abstract into a structured prompt word through the auxiliary model; evaluating and optimizing the structured prompt word based on a domain knowledge database to obtain an optimized prompt word; performing compliance filtering on the optimized prompt word to obtain a compliant prompt word; evaluating the complexity of the compliant prompt word through a lightweight evaluation model to obtain an evaluation result; if the evaluation result is simple, calling an expert model to generate text content according to the compliant prompt word; and if the evaluation result is complex, calling a large language model to generate text content according to the compliant prompt word.

Description

Technical Field

[0001] This application relates to the field of language processing technology, and in particular to a method, system, and storage medium for intelligent agent prompt word deduction. Background Technology

[0002] With the continuous development of language processing technology, prompt words, as the core of user needs and system processing, directly determine the business adaptation effect in terms of their quality.

[0003] To improve the effectiveness of prompt words, existing technologies employ a method of manually constructing prompt words. This method involves professionals manually breaking down user needs, combining business rules and knowledge bases, and gradually optimizing the prompt word structure through trial and error iteration to ultimately form an executable instruction template.

[0004] However, in the long-term application process, the lack of automatic parsing and structuring of the user's original intent makes it impossible to transform vague and non-professional natural language requirements into standardized instructions that the machine can accurately understand and execute. Summary of the Invention

[0005] To address the aforementioned technical problems, this application provides a method, system, and storage medium for inferring intelligent agent prompt words.

[0006] The technical solution provided in this application is described below:

[0007] The first aspect of this application provides a method for inferring prompt words for an intelligent agent, the method comprising:

[0008] The main model parses the user input information to obtain an initial task summary;

[0009] The logic of the initial task summary is determined by an auxiliary model to ensure its completeness.

[0010] If not, the initial task summary is revised using the main model to obtain the target task summary;

[0011] The auxiliary model converts the target task summary into structured prompt words.

[0012] The structured prompt words are evaluated and optimized based on a domain knowledge database to obtain preferred prompt words;

[0013] The preferred prompt words are then filtered for compliance to obtain compliant prompt words;

[0014] The complexity of the compliance prompt words is evaluated using a lightweight evaluation model to obtain the evaluation results;

[0015] If the assessment result is simple, then the expert model is invoked to generate text content based on the compliance prompt words;

[0016] If the evaluation result is complex, then the large language model is invoked to generate text content based on the compliance prompt words.

[0017] Optionally, the step of parsing the user input information through the main model to obtain an initial task summary includes:

[0018] Extract key information from user input to obtain structured key data;

[0019] The structured key data is semantically understood using the main model to generate a weighted initial task summary.

[0020] Optionally, after determining whether the logic of the initial task summary is complete through the auxiliary model, the method further includes:

[0021] If so, the initial task summary is converted into structured prompt words using the auxiliary model.

[0022] Optionally, the evaluation and optimization of the structured prompt words based on the domain knowledge database to obtain preferred prompt words includes:

[0023] Based on the structured prompt term retrieval domain knowledge database, a set of candidate prompt terms is generated;

[0024] A cost-benefit assessment is performed on the set of candidate prompts to obtain the preferred prompts.

[0025] Optionally, the step of generating a candidate prompt word set based on the structured prompt word retrieval domain knowledge database includes:

[0026] Extract keywords from the structured prompts to obtain a keyword list;

[0027] The relevant domain knowledge data can be obtained by searching the domain knowledge base using the keyword list.

[0028] Based on the relevant domain knowledge data, a set of alternative prompt words is generated.

[0029] Optionally, the step of conducting a cost-benefit evaluation on the set of candidate prompts to obtain preferred prompts includes:

[0030] The API call cost of each prompt word in the set of alternative prompt words is calculated using a parallel testing method, and a token consumption data report for each prompt word is obtained.

[0031] The expected effect of each prompt word is evaluated based on the Token consumption data report, and the expected satisfaction score is obtained.

[0032] The preferred prompt words are obtained by weighting the token consumption data report and the expected satisfaction score.

[0033] Optionally, after generating the text content, the method further includes:

[0034] The text content is subjected to multi-dimensional evaluation and detection to determine whether the multi-dimensional evaluation report of the text content meets the quality standards.

[0035] If so, the structured prompts and the text content will be stored as examples in the domain knowledge database;

[0036] If not, the problem location is determined based on the aforementioned multi-dimensional assessment report;

[0037] If the problem is identified as excessive response latency, then the complexity of the auxiliary model in generating the structured prompt words should be reduced.

[0038] If the problem is identified as an accuracy or user satisfaction rate falling below a threshold, then the number of initial task summaries generated by the main model is increased.

[0039] A second aspect of this application provides an intelligent agent prompt word deduction system, the system comprising:

[0040] The data parsing unit is used to parse the user input information into an intent using the main model to obtain an initial task summary.

[0041] The first judgment unit is used to determine whether the logic of the initial task summary is complete through the auxiliary model;

[0042] A data correction unit is used to correct the initial task summary through the main model to obtain the target task summary when the first judgment unit determines that the initial task summary logic is incomplete.

[0043] The first data transformation unit is used to convert the target task summary into structured prompt words through the auxiliary model;

[0044] The optimization unit is used to evaluate and optimize the structured prompt words based on the domain knowledge database to obtain preferred prompt words;

[0045] A data filtering unit is used to filter the preferred prompt words for compliance, thereby obtaining compliant prompt words;

[0046] The second judgment unit is used to evaluate the complexity of the compliance prompt words through a lightweight evaluation model and obtain the evaluation result;

[0047] The first generation unit is used to call the expert model to generate text content based on the compliance prompt words when the second judgment unit determines that the evaluation result is simple.

[0048] The second generation unit is used to call the large language model to generate text content based on the compliance prompt words when the second judgment unit determines that the evaluation result is complex.

[0049] A third aspect of this application provides an intelligent agent prompt word deduction system, the system comprising:

[0050] Processor, memory, input / output units, and bus;

[0051] The processor is connected to the memory, the input / output unit, and the bus;

[0052] The memory stores a program, which the processor invokes to execute the first aspect and any one of the optional methods in the first aspect.

[0053] A fourth aspect of this application provides a computer-readable storage medium on which a program is stored, which, when executed on a computer, performs the methods of the first aspect and any one of the first aspects.

[0054] As can be seen from the above technical solutions, this application has the following beneficial effects:

[0055] This application automatically parses user input to generate an initial task summary using a main model, replacing manual breakdown of requirements and achieving automatic recognition of the user's original intent, thus solving the inefficiency problem caused by manual reliance. Next, an auxiliary model determines the logical completeness of the initial task summary. If the initial task summary is logically incomplete, the main model automatically corrects it to obtain the target summary, solving the problem of translating vague and non-professional natural language requirements into logically rigorous instructions. Then, the auxiliary model converts the target task summary into structured prompts, transforming natural language requirements into a standardized format that machines can accurately understand and execute, overcoming the bottleneck of converting natural language into machine instructions during manual construction. Subsequently, the structured prompts are evaluated and optimized based on a domain knowledge database, making them more suitable for business scenario characteristics. Furthermore, compliance filtering of prompt parsing proactively avoids violation risks. Finally, after evaluating complexity using a lightweight model, expert models are invoked as needed to improve the efficiency of simple tasks, or large language models are invoked to ensure the quality of complex tasks, thereby generating the text content required by the user. The entire process requires no manual trial and error iteration, which greatly improves the efficiency of instruction conversion, while achieving standardization, automation and accuracy of prompt words from generation to application, and also solves the problem of insufficient ability to convert vague requirements by relying on manual intervention. Attached Figure Description

[0056] To more clearly illustrate the technical solutions in this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0057] Figure 1 A schematic flowchart of an embodiment of the intelligent agent prompt word inference method provided in this application;

[0058] Figure 2 A flowchart illustrating an embodiment of the intelligent agent prompt word inference method provided in this application, which determines whether to respond to a remote operation request based on a network connection configuration file being in operation;

[0059] Figure 3 A schematic diagram of an embodiment of the intelligent agent prompt word inference method provided in this application, which evaluates and optimizes structured prompt words based on a domain knowledge database;

[0060] Figure 4 A schematic flowchart of an embodiment of the intelligent agent prompt word inference method provided in this application for multi-dimensional evaluation and detection of text content;

[0061] Figure 5 A flowchart illustrating an embodiment of the intelligent agent prompt word inference system provided in this application;

[0062] Figure 6 A schematic flowchart of another embodiment of the intelligent agent prompt word inference system provided in this application. Detailed Implementation

[0063] It should be noted that the intelligent agent prompt word inference method provided in this application can be applied to terminals or servers, and can also be applied to servers. For example, the terminal can be a smartphone, computer, tablet, smart TV, smartwatch, portable computer terminal, or a desktop computer or other fixed terminal. For ease of explanation, this application uses a remote management platform on the terminal as the execution subject for illustration.

[0064] Please see Figure 1 This application first provides an embodiment of an intelligent agent prompt word inference method, which includes:

[0065] S101. The user input information is parsed using the main model to obtain an initial task summary;

[0066] In this embodiment, the main model is a model with high-order semantic understanding capabilities, such as GPT-4, which is mainly responsible for receiving user input information and parsing the requirements contained therein. The initial task summary is a structured task description with weighted annotations generated by the main model after parsing the user input information, for example: {core intent: "optimize customer service response", subtasks: ["emotional soothing: 0.8", "policy explanation: 0.9"]}.

[0067] Before performing intent parsing, key information needs to be extracted from the natural language input of the user, filtering out core elements such as user emotion, question type, and core needs, and transforming them into structured key data with clearly defined fields. Next, this structured key data is fed into the main model for in-depth analysis, breaking down the user's core intent, such as "optimize customer service dialogue strategy" or "generate medical diagnosis suggestions." Then, sub-tasks related to the core intent are identified, such as "emotional soothing," "policy explanation," and "symptom analysis," and each sub-task is weighted according to its importance to the core intent, such as "policy explanation: 0.9" and "emotional soothing: 0.8." Finally, the main model integrates the core intent, weighted related sub-tasks, and key need features to obtain an initial task summary with clear fields and a well-defined direction.

[0068] S102. Use an auxiliary model to determine whether the logic of the initial task summary is complete;

[0069] In this embodiment, the auxiliary model is Llama2, which is mainly responsible for receiving the weighted initial task summary generated by the main model and converting the weighted initial task summary into a multi-level JSON template containing roles, formats, and examples. This multi-level JSON template is used to present the fields of the task summary in a structured way, which facilitates subsequent field-level comparison with the domain rule library. Then, the LangChain PromptTemplate is used to dynamically fill the fields in the prompt template. The auxiliary model also detects logical contradictions in the main model summary, such as missing key fields, to trigger the main model to correct them, forming a two-way error correction closed loop.

[0070] When determining the logical completeness of the initial task summary, a pre-defined domain logic rule base is first invoked. This rule base contains the key elements required for tasks in each domain. For example, in the financial domain, "loan approval" requires user qualifications and credit limit requirements, while in the medical domain, "consultation" requires symptoms and medical history. Then, the initial task summary is compared with the rule base one by one to check for missing key information or logical conflicts in sub-tasks, such as contradictory statements about fast loan disbursement and low interest rates. Further, if problems exist, it is marked as logically incomplete, and step S104 is executed; if it perfectly matches the rule base, it is determined to be logically complete, and step S103 is executed.

[0071] S103. Convert the initial task summary into structured prompt words using an auxiliary model;

[0072] In this embodiment, structured prompts are standardized instructions that machines can understand. They are usually presented in a fixed JSON template format, which can avoid the ambiguity of natural language.

[0073] When determining the logically complete matching rule base for the initial task summary, it is necessary to first call the preset multi-level JSON template, then extract key information such as the core intent and weighted subtasks from the initial task summary, match the information with the template fields through LangChain's PromptTemplate, and then, after the fields are dynamically filled and the format is validated, a structured prompt word that can be directly recognized and executed by the machine can be generated.

[0074] S104. The initial task summary is revised using the master model to obtain the target task summary;

[0075] In this embodiment, when the auxiliary model determines that there is a logical problem in the initial task summary, it needs to send the incomplete logical information, such as missing key fields or sub-task contradictions, to the main model. Then, the main model combines the user's original input information with domain knowledge to analyze the rationality of the problem, confirm the content that needs to be supplemented or adjusted, and then supplement the missing fields and correct the logical contradictions in the initial task summary. Finally, it verifies the completeness and consistency of the corrected summary to obtain a logically rigorous and informationally complete target task summary.

[0076] S105. Convert the target task summary into structured prompt words using an auxiliary model;

[0077] After the auxiliary model obtains the target task summary, it needs to first call the preset multi-level JSON template, then extract key information such as the core intent and weighted subtasks from the target task summary, and accurately match the information with the template fields through LangChain's PromptTemplate. Finally, it completes the dynamic field filling and format validation to generate structured prompt words that the machine can directly recognize and execute.

[0078] S106. Evaluate and optimize the structured prompt words based on the domain knowledge database to obtain the preferred prompt words;

[0079] In this embodiment, the domain knowledge database is a structured database that stores professional information in a specific domain, including business rules, historical high-quality prompt words, and professional terminology. Relevant information can be matched through keyword retrieval, providing relevant domain knowledge for prompt word generation. Preferred prompt words are selected from the set of candidate prompt words generated based on domain knowledge, after a cost-effectiveness weighted evaluation, choosing prompt words that both meet business needs and balance resource consumption.

[0080] When evaluating and optimizing structured prompts, the core keywords are first extracted to form a keyword list. Then, using this list as search criteria, relevant domain knowledge data is matched and retrieved from a domain knowledge database. Subsequently, multiple sets of candidate prompts are generated based on this knowledge data. Next, the API call cost of each candidate prompt is calculated using parallel testing methods, generating a token consumption data report. Further, the expected effect of each prompt is evaluated based on the token consumption data report, resulting in an expected satisfaction score. Finally, a weighted calculation is performed on the token consumption data and the expected satisfaction score, comprehensively balancing cost and effect, to select the prompts that both meet business needs and balance resource consumption as the preferred prompts.

[0081] S107. Filter the preferred prompts for compliance to obtain compliant prompts;

[0082] In this embodiment, when performing compliance filtering, a domain rule base matching the business scenario needs to be loaded. For example, the financial domain rule base includes clauses such as "prohibiting the generation of false profit promises" and "risk warning clauses must not be omitted," while the medical domain rule base includes requirements such as "prohibiting absolute efficacy statements" and "must be labeled as for reference only and does not constitute medical advice." Next, the preferred prompt words are compared sentence by sentence with the clauses in the domain rule base. Keyword matching and semantic similarity analysis are used to detect whether any illegal content exists. If a violation is found, the system automatically removes the illegal expression or replaces it with compliant content; if no illegal content is detected, the prompt word is directly marked as a compliant prompt word.

[0083] S108. Evaluate the complexity of compliance prompt words using a lightweight evaluation model to obtain the evaluation results;

[0084] In this embodiment, before evaluating the complexity of compliance prompts, a large number of compliance prompt sample words in the domain, such as those from financial and medical scenarios, need to be collected. These samples are labeled as "simple" or "complex," with simple corresponding to short texts and single-task target samples, and complex corresponding to long texts and samples with multiple inference steps. Training, validation, and test sets are then created. Next, a lightweight evaluation model is initialized, and the processed data is fed into it. The initial lightweight evaluation model then predicts whether the text is simple or complex. Further, the prediction results are compared with the actual labels to calculate the error, and the parameters in the initial lightweight evaluation model are gradually adjusted to reduce the error. This training is repeated many times until the initial lightweight evaluation model's accuracy in judging new texts stabilizes at a high level, at which point the trained lightweight evaluation model is obtained.

[0085] Next, the system inputs the compliance prompts into the trained lightweight evaluation model. The lightweight evaluation model first extracts the key features of the prompts, including the number of text characters, task type (e.g., "simple question answering" vs. "complex analysis"), number of reasoning steps (e.g., "single-step query" vs. "multi-round logical verification"), and domain knowledge depth (e.g., "common sense statement" vs. "professional parameter calculation"). Subsequently, the lightweight evaluation model compares the extracted features with the "simple / complex" feature thresholds in the training samples. If the feature thresholds are met (e.g., text length < 200 characters, reasoning steps ≤ 1 step), the system outputs a "simple" evaluation result and executes step S109; if the feature thresholds are met (e.g., text length > 500 characters, reasoning steps ≥ 3 steps), the system outputs a "complex" evaluation result and executes step S110.

[0086] S109. If the assessment result is simple, then call the expert model to generate text content based on the compliance prompt words;

[0087] In this embodiment, the expert model has a built-in domain rule base and standardized templates, eliminating the need to call large, resource-intensive models. It can quickly process simple tasks and generate compliant and standardized text according to domain rules.

[0088] When the evaluation result is "simple," the expert model pre-integrates mature business rules and standardized templates within the domain. For example, in a financial customer service scenario, the template for "simple Q&A" includes a structure of "greeting + answering the question + closing remarks," and has built-in fixed answer rules such as "bank card loss reporting process" and "password reset steps." Upon invocation, the expert model first receives compliance prompts, parses the task objectives (e.g., "answering the bank card loss reporting process") and output requirements (e.g., "point-by-point explanations"), and then matches the template corresponding to the task objective in its built-in template library, such as a "bank card loss reporting process point-by-point template." It then fills the corresponding fields in the template with key information from the prompts, such as "loss reporting channel: APP or offline branch." Further, rule validation ensures that the filled content conforms to domain standards, such as not omitting reminders like "replace the card promptly after reporting the loss," and finally generates well-structured and accurate text content to send to the user.

[0089] S110. If the evaluation result is complex, the large language model is invoked to generate text content based on compliance prompts.

[0090] In this embodiment, the large language model has extensive domain knowledge reserves and cross-scenario reasoning capabilities, and can handle tasks that expert models find difficult to handle, such as multi-dimensional analysis, complex solution derivation, and multi-modal content integration.

[0091] When the assessment result is determined to be complex, the system will invoke a large language model to deeply analyze compliance prompts, extract task objectives, identify implicit requirements, and output constraints. Next, the large language model will utilize its built-in domain knowledge module, combined with real-time integrated industry dynamic data, to perform multi-step reasoning. This includes analyzing the maturity of various cutting-edge technologies, matching corresponding enterprise application cases, and finally predicting the difficulties in technology implementation. Then, text content will be generated. During the generation process, the model will dynamically adjust the output logic to ensure that the text content is clear, professionally in-depth, and conforms to the format requirements of the compliance prompts. Finally, professional, logical, and complete text content will be generated and sent to the user.

[0092] This embodiment automatically parses user input to generate an initial task summary using a main model, replacing manual breakdown of requirements and achieving automatic recognition of the user's original intent, thus solving the inefficiency problem caused by manual reliance. Next, an auxiliary model determines the logical completeness of the initial task summary. If the initial task summary is logically incomplete, the main model automatically corrects it to obtain the target summary, solving the problem of being unable to convert vague and non-professional natural language requirements into logically rigorous instructions. Then, the auxiliary model converts the target task summary into structured prompts, transforming natural language requirements into a standardized format that machines can accurately understand and execute, overcoming the bottleneck of converting natural language into machine instructions during manual construction. Subsequently, the structured prompts are evaluated and optimized based on a domain knowledge database, making them more suitable for the characteristics of the business scenario. Furthermore, compliance filtering of prompt parsing proactively avoids violation risks. Finally, after evaluating the complexity using a lightweight model, expert models are called as needed to improve the efficiency of simple tasks, or large language models are called to ensure the quality of complex tasks, thereby generating the text content required by the user. The entire process requires no manual trial and error iteration, which greatly improves the efficiency of instruction conversion, while achieving standardization, automation and accuracy of prompt words from generation to application, and also solves the problem of insufficient ability to convert vague requirements by relying on manual intervention.

[0093] For a detailed implementation of the intent parsing of user input information using the main model in step S101 above, please refer to [link to relevant documentation]. Figure 2 As shown, Figure 2 An embodiment of the agent prompt word inference method provided in this application for obtaining an initial task summary includes:

[0094] S201. Extract key information from user input to obtain structured key data;

[0095] In this embodiment, after receiving user input, the system uses natural language processing and other technologies to deeply analyze the input content. In a financial context, this analysis identifies the emotional tone of the user's statements, such as positive, negative, or neutral; it also accurately determines the type of question, such as inquiries, complaints, or suggestions. Through this analysis, key information is extracted from massive amounts of text and organized according to preset rules and formats, ultimately generating structured key data.

[0096] S202. Perform semantic understanding of structured key data through the main model to generate a weighted initial task summary.

[0097] In this embodiment, when performing semantic understanding of structured key data through the main model, the main model first needs to clarify the user's core intent. Then, it identifies the sub-tasks directly related to the core intent. Subsequently, based on the importance of each sub-task to the core intent, a weight is assigned to each sub-task, such as "Policy Explanation: 0.9" and "Emotional Soothing: 0.8". Further, the core intent, weighted related sub-tasks, and key requirement features are integrated to generate a structured weighted task summary.

[0098] This embodiment extracts key information and generates structured key data, transforming the originally fragmented and unstructured natural language input by users into standardized information that can be accurately recognized by machines, thus avoiding the omission of key information or interpretation errors. Then, the high-order semantic understanding capability of the main model generates a weighted initial task summary based on the structured data. This not only clarifies the core intent but also quantifies the priority of sub-tasks through weight annotation, providing a clear basis for subsequent auxiliary model logical judgment, prompt word conversion, and resource allocation, and significantly reducing the cost of manual decomposition.

[0099] For a detailed implementation of step S106 above, which involves evaluating and optimizing structured prompt words based on a domain knowledge database, please refer to [link to relevant documentation]. Figure 3 As shown, Figure 3 An embodiment of the intelligent agent prompt word inference method provided in this application for obtaining preferred prompt words includes:

[0100] S301. Extract keywords from the structured prompts to obtain a keyword list;

[0101] In this embodiment, when extracting keywords from structured prompts, the core fields of the structured prompts need to be located first, and these fields are the key sources for keyword extraction. Then, a keyword extraction algorithm such as the TF-IDF algorithm is called to screen for core vocabulary from the content of each field. For example, in the structured prompt of the financial scenario, "loan approval" and "process" are extracted from "Task objective: Answer the loan approval process". At the same time, function words such as "of" and "need" that have no practical meaning are filtered out to ensure that the keywords are strongly relevant to the task requirements. Next, the extracted core vocabulary is sorted according to domain relevance and organized into a keyword list with a standardized format.

[0102] S302. Retrieve the domain knowledge base through the keyword list to obtain relevant domain knowledge data;

[0103] In this embodiment, before retrieving the domain knowledge base through the keyword list, relevant professional data in the corresponding domain needs to be obtained first. Then, these relevant professional data are input into GPT-4 to generate a summary containing core terms, business rules, and application scenarios. Subsequently, the generated summary is classified and indexed according to technical categories such as "loan approval" and "product recommendation", and the classified summary is stored in a vector database to complete the construction of the domain knowledge base.

[0104] Next, the generated keyword list is used as a retrieval condition, and the BERT-base semantic similarity matching algorithm technology is used to retrieve in the knowledge base. For example, for the keyword "frontier terms in quantum computing", definitions and application scenario data of related terms such as "quantum error correction technology" and "qubit optimization" in the knowledge base will be matched. Further, the system will screen the data according to the set matching degree, such as similarity ≥ 80%, and eliminate irrelevant or low-correlation content, and finally output relevant domain knowledge data that can support prompt optimization.

[0105] S303. Generate an alternative prompt word set according to the relevant domain knowledge data;

[0106] In this embodiment, when generating an alternative prompt word set, the original task requirements of the structured prompt need to be analyzed, and the prompt word optimization direction is clarified in combination with the relevant domain knowledge data. Then, based on the original structured prompt, multiple differentiated versions are generated by supplementing knowledge details, adjusting expression norms, adding domain adaptation content, etc. For example, for "loan consultation response", one version focuses on "concise process description", another version focuses on "detailed risk point prompts", and another version adds "supplementary examples of common problems". Finally, all the differentiated versions are integrated to form an alternative prompt word set containing multiple effective versions.

[0107] S304. Calculate the API call cost of each prompt word in the alternative prompt word set through the parallel test method to obtain a Token consumption data report for each prompt word;

[0108] In this embodiment, the API call cost is the fee for using the API, determined by both the token consumption and the price per token. A token is the basic unit for processing text; one token is approximately equal to one or two Chinese characters. Token consumption refers to the total number of tokens converted from input prompts and output text during task processing. The parallel testing method, based on the A / B testing framework, involves randomly dividing the target objects into two groups: one using scheme A and the other using scheme B, keeping other variables consistent. The superiority of the schemes is scientifically judged by statistically analyzing the differences in key indicators.

[0109] When calculating the API call cost for each suggestion in the candidate suggestion set, all suggestions in the candidate suggestion set must first be randomly assigned to different test groups. Each group contains only one suggestion to be tested, ensuring that all test variables are completely consistent across groups except for the "suggestion content," thus avoiding interference from irrelevant variables in the cost data. Then, parallel testing is initiated, synchronously launching API call simulations to the suggestions in each group, recording the input and output token counts for each suggestion in real time. Based on the API's token unit price rules, the cost per call for each suggestion is calculated. Furthermore, after the test, the data from each group is integrated and organized in the format of "Suggestion ID - Input Token Count - Output Token Count - Total Token Consumption - Estimated Call Cost," generating a token consumption report containing cost data for all candidate suggestions.

[0110] S305. Evaluate the expected effect of each prompt word based on the Token consumption data report and obtain the expected satisfaction score;

[0111] In this embodiment, before obtaining the expected satisfaction score, a "cost-effectiveness" correlation evaluation system needs to be established. This system combines historical data from the domain and sets evaluation dimensions, including core need coverage, output accuracy, and user acceptance. Then, each candidate prompt word is evaluated against the token consumption data report. If a prompt word has moderate token consumption and contains complete risk warnings, its "core need coverage" score is high; if a prompt word has excessive token consumption but redundant content, its "output accuracy" score is low. Finally, an expected satisfaction score is assigned to each prompt word on a scale of 1 to 10. The score results correspond one-to-one with the token consumption data, providing a basis for subsequent weighted calculations of effectiveness dimensions.

[0112] S306. Calculate the optimal prompt words by weighting the token consumption data report and the expected satisfaction score.

[0113] In this embodiment, when performing weighted calculations, weight coefficients need to be set according to the business scenario. If the business prioritizes cost control, the token consumption weight is set to 0.4 and the expected satisfaction weight is set to 0.6; if the focus is on performance, the satisfaction weight is set to 0.7 and the cost weight is set to 0.3. Then, the formula is used: Overall Score = (Expected Satisfaction Score / 10) × Satisfaction Weight + (1 - Relative Token Consumption) × Cost Weight, where "Relative Token Consumption" is the ratio of the consumption of the prompt word to the highest consumption in the set, used to ensure that the cost item is a positive indicator. Next, after the calculation is completed, the overall scores of all candidate prompt words are sorted, and the prompt word with the highest score is selected as the preferred prompt word.

[0114] This embodiment extracts structured prompt keywords to pinpoint core task requirements, providing a clear direction for subsequent searches and avoiding interference from irrelevant information. Then, it searches a domain knowledge base using these keywords, introducing professional data support to address the issue of insufficient professionalism in the prompts and ensure their relevance to the business scenario. Subsequently, it generates a set of alternative prompts based on domain knowledge, breaking the limitations of a single design and providing multiple optimization possibilities. Next, it calculates the API call cost of each prompt through parallel testing, generating a token consumption report to reduce resource waste. Furthermore, it evaluates the expected results based on consumption data, obtaining a satisfaction score to avoid the bias of focusing solely on cost while neglecting quality. Finally, it obtains the optimal prompts through weighted calculation, efficiently balancing execution performance and resource costs, progressively improving the cost-effectiveness of subsequent tasks.

[0115] Please refer to Figure 4 , Figure 4 An embodiment of the intelligent agent prompt word inference method provided in this application for multi-dimensional evaluation and detection of text content includes:

[0116] S401. Conduct multi-dimensional evaluation and testing of the text content, and determine whether the multi-dimensional evaluation report of the text content meets the quality standards.

[0117] In this embodiment, before determining whether the multi-dimensional evaluation report of the text content meets the quality standards, it is necessary to first collect key evaluation data corresponding to the current text content. In terms of effectiveness, this involves detecting the text content's ability to solve user problems, such as the accuracy rate of answering loan questions in a financial customer service scenario and user satisfaction ratings based on likes or negative reviews. In terms of cost, it involves obtaining response latency. Next, the collected indicator values ​​are compared one by one with preset thresholds, as shown in Table 1.

[0118] Table 1

[0119]

[0120] If all indicators meet or exceed the threshold, the text content is judged to meet the quality standard and step S402 is executed; if any indicator fails to meet the standard, the quality is judged not to meet the quality standard and step S403 is executed.

[0121] S402. Store the structured prompts and text content as examples in the domain knowledge database;

[0122] In this embodiment, when the text content is determined to meet the quality standard, the system automatically combines the structured prompts corresponding to the text content with the final text content to form a complete "prompt-output" example pair. Then, the example pair is transmitted to the domain knowledge database through a web interface built with Flask, and simultaneously categorized and stored according to technical categories.

[0123] S403. Problem identification is obtained based on the multi-dimensional assessment report;

[0124] In this embodiment, when it is determined that the text content does not meet the quality standard, the system will first check the specific data of each indicator in the multi-dimensional evaluation report, analyze which type of indicator has a deviation, and if the problem is located that the response delay value exceeds the threshold, then step S404 is executed; if the problem is located that the accuracy or user satisfaction is lower than the threshold, then step S405 is executed.

[0125] S404. If the problem is identified as excessive response delay, reduce the complexity of the auxiliary model in generating structured prompts.

[0126] In this embodiment, when the problem is identified as excessive response latency, the first step is to clarify the cause of the excessive latency. As the supplementary example has indicated, it often stems from the complex processing of the auxiliary model in the structured prompt generation stage, such as redundant field conversion and repeated semantic verification. Therefore, the system first calls LangChain's ContextCompressor algorithm to truncate redundant historical dialogues or non-critical fields during auxiliary model processing. Next, it reduces the format conversion steps of the auxiliary model. For example, for specific fields such as healthcare, it directly injects a preset medical compliance declaration template to avoid the model repeatedly generating basic formats. Then, it lowers the computational accuracy requirements of the auxiliary model, such as reducing the semantic matching accuracy threshold from 80% to 70%, and retaining only core fields such as task objectives and output format verification for strictness, while eliminating redundant verification items. Furthermore, after the adjustments are completed, the system tracks the response latency data in real time through Prometheus. If the latency falls back to within the threshold, this round of optimization is complete; if it does not meet the standard, further reduction of processing steps is required.

[0127] S405. If the problem is identified as an accuracy or user satisfaction rate below a threshold, increase the number of initial task summaries generated by the main model.

[0128] In this embodiment, when the problem is identified as either accuracy or user satisfaction falling below a threshold, the system automatically analyzes the core reason for the accuracy decline. For example, it might be due to insufficient initial task summaries generated by the main model, resulting in incomplete coverage of key needs or scenarios, such as generating only one version of the summary without considering potential user questions. Therefore, the system first increases the number of versions of initial task summaries generated by the main model for similar user input, such as increasing it from 1 version to 5 versions. Next, combined with a dynamic parameter adjustment mechanism, the temperature parameter of the main model is increased from a base value of 0.7 to 0.9, making the generated summaries more diverse and covering more user needs and scenarios. Then, the newly added summaries are converted into multiple versions of structured prompts by an auxiliary model. When generating text content based on these multiple versions of structured prompts, the optimal version can be selected from the multiple versions for subsequent text content generation. Furthermore, after the adjustments are completed, the system continues to monitor the accuracy of the text content and user satisfaction metrics to verify the effectiveness.

[0129] This embodiment conducts multi-dimensional evaluation and testing of text content, combining indicators such as accuracy, user satisfaction, and response latency to determine whether it meets the standards, controlling text quality from the source and preventing inferior content from entering subsequent stages. Next, when the multi-dimensional evaluation report of the text content meets the standards, the "structured prompt word - text content" example is stored in the domain knowledge base, accumulating high-quality samples for subsequent prompt word generation and assisting in system iteration and optimization. When the multi-dimensional evaluation report of the text content does not meet the standards, the system accurately locates the problem based on the evaluation report, avoiding blind adjustments. Then, when the problem is determined to be excessive response latency, the system reduces the complexity of the auxiliary model in generating structured prompt words, quickly optimizing time-consuming steps and reducing resource waste. When the problem is determined to be that accuracy or user satisfaction does not meet the preset standards, the system increases the number of initial task summaries generated by the main model, improving the quality of the prompt word source. The entire process follows a "evaluation - standard-compliant data accumulation - non-compliant data location - targeted optimization" progression, forming a closed-loop iteration that ensures text quality while dynamically optimizing system performance, achieving a dual improvement in effectiveness and cost.

[0130] Please see Figure 5 This application provides an intelligent agent prompt word inference system, including:

[0131] The data parsing unit 501 is used to parse the user input information through the main model to obtain an initial task summary.

[0132] The first judgment unit 502 is used to determine whether the logic of the initial task summary is complete through the auxiliary model;

[0133] The data correction unit 503 is used to correct the initial task summary through the main model to obtain the target task summary when the first judgment unit determines that the initial task summary logic is incomplete.

[0134] The first data conversion unit 504 is used to convert the target task summary into structured prompt words through an auxiliary model;

[0135] The optimization unit 505 is used to evaluate and optimize the structured prompt words based on the domain knowledge database to obtain the preferred prompt words;

[0136] The data filtering unit 506 is used to filter the preferred prompt words for compliance to obtain compliant prompt words;

[0137] The second judgment unit 507 is used to evaluate the complexity of compliance prompt words through a lightweight evaluation model and obtain the evaluation result.

[0138] The first generation unit 508 is used to call the expert model to generate text content based on compliance prompts when the second judgment unit determines that the evaluation result is simple.

[0139] The second generation unit 509 is used to call the large language model to generate text content based on compliance prompts when the second judgment unit determines that the evaluation result is complex.

[0140] Optionally, the data parsing unit 501 is specifically used for:

[0141] Extract key information from user input to obtain structured key data;

[0142] The main model performs semantic understanding of structured key data to generate a weighted initial task summary.

[0143] Optionally, a second data conversion unit 510 is also included, specifically for:

[0144] If so, the initial task summary is converted into structured prompts using an auxiliary model.

[0145] Optionally, optimization unit 505 is specifically used for:

[0146] Based on the structured keyword search domain knowledge database, generate a set of candidate keywords;

[0147] A cost-benefit assessment of the set of alternative prompts is conducted to determine the preferred prompts.

[0148] Optionally, optimization unit 505 is also used for:

[0149] Extract keywords from the structured prompts to obtain a keyword list;

[0150] Retrieve relevant domain knowledge data from the domain knowledge base by searching a keyword list;

[0151] Based on relevant domain knowledge data, generate a set of alternative prompt words.

[0152] Optionally, optimization unit 505 is also used for:

[0153] The API call cost of each prompt word in the candidate prompt word set is calculated using parallel testing methods, and a token consumption data report for each prompt word is obtained.

[0154] Evaluate the expected effect of each prompt word based on the token consumption data report and obtain the expected satisfaction score;

[0155] The optimal suggestion words are obtained by weighting the token consumption data report and the expected satisfaction score.

[0156] Optionally, a verification optimization unit 511 is also included, specifically for:

[0157] The text content is evaluated and tested from multiple dimensions to determine whether the multi-dimensional evaluation report of the text content meets the quality standards.

[0158] If so, the structured prompts and text content will be stored as examples in the domain knowledge database.

[0159] If not, the problem will be identified based on the multi-dimensional assessment report;

[0160] If the problem is identified as excessive response latency, then reduce the complexity of the auxiliary model in generating structured prompts.

[0161] If the problem is identified as accuracy or user satisfaction falling below a threshold, the number of initial task summaries generated by the main model is increased.

[0162] Please see Figure 6 This application also provides an intelligent agent prompt word inference system, including:

[0163] Processor 601, memory 602, input / output unit 603, bus 604;

[0164] The processor 601 is connected to the memory 602, the input / output unit 603, and the bus 604;

[0165] The memory 602 stores a program, and the processor 601 calls the program to execute any of the methods described above.

[0166] This application also relates to a computer-readable storage medium on which a program is stored, which, when run on a computer, causes the computer to perform any of the methods described above.

[0167] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0168] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between apparatuses or units through some interfaces, and may be electrical, mechanical, or other forms.

[0169] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0170] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0171] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A method for inferring prompt words for an intelligent agent, characterized in that, The method includes: The main model parses the user input information to obtain an initial task summary; The logic of the initial task summary is determined by an auxiliary model to ensure its completeness. If not, the initial task summary is revised using the main model to obtain the target task summary; The auxiliary model converts the target task summary into structured prompt words. The structured prompt words are evaluated and optimized based on a domain knowledge database to obtain preferred prompt words; The preferred prompt words are then filtered for compliance to obtain compliant prompt words; The complexity of the compliance prompt words is evaluated using a lightweight evaluation model to obtain the evaluation results; If the assessment result is simple, then the expert model is invoked to generate text content based on the compliance prompt words; If the evaluation result is complex, then the large language model is invoked to generate text content based on the compliance prompt words; The structured prompt words are evaluated and optimized based on a domain knowledge database to obtain preferred prompt words, including: Based on the structured prompt term retrieval domain knowledge database, a set of candidate prompt terms is generated; The API call cost of each prompt word in the set of alternative prompt words is calculated using a parallel testing method, and a token consumption data report for each prompt word is obtained. The expected effect of each prompt word is evaluated based on the Token consumption data report, and the expected satisfaction score is obtained. The expected satisfaction score for each prompt word is normalized and weighted according to the satisfaction weight to obtain the effect dimension score; Based on the cost weight, the relative tokens in the token consumption data report of each prompt word are inversely weighted to obtain the cost dimension score; The comprehensive evaluation score is obtained by adding the scores of the effectiveness dimension and the cost dimension. Based on the comprehensive evaluation score, each of the candidate prompt words is sorted to obtain a score-sorted list; The highest-rated prompt word in the score ranking list is selected as the preferred prompt word.

2. The agent prompt word inference method according to claim 1, characterized in that, The process of parsing user input information using the main model to obtain an initial task summary includes: Extract key information from user input to obtain structured key data; The structured key data is semantically understood using the main model to generate a weighted initial task summary.

3. The agent prompt word inference method according to claim 1, characterized in that, After determining the completeness of the logic for the initial task summary using an auxiliary model, the method further includes: If so, the initial task summary is converted into structured prompt words using the auxiliary model.

4. The agent prompt word inference method according to claim 1, characterized in that, The step of generating a candidate prompt word set based on the structured prompt word retrieval domain knowledge database includes: Extract keywords from the structured prompts to obtain a keyword list; The relevant domain knowledge data can be obtained by searching the domain knowledge base using the keyword list. Based on the relevant domain knowledge data, a set of alternative prompt words is generated.

5. The agent prompt word inference method according to any one of claims 1 to 4, characterized in that, Following the generation of the text content, the following is also included: The text content is subjected to multi-dimensional evaluation and detection to determine whether the multi-dimensional evaluation report of the text content meets the quality standards. If so, the structured prompts and the text content will be stored as examples in the domain knowledge database; If not, the problem location is determined based on the aforementioned multi-dimensional assessment report; If the problem is identified as excessive response latency, then the complexity of the auxiliary model in generating the structured prompt words should be reduced. If the problem is identified as an accuracy or user satisfaction rate falling below a threshold, then the number of initial task summaries generated by the main model is increased.

6. An intelligent agent prompt word inference system, characterized in that, The system includes: The data parsing unit is used to parse the user input information into an intent using the main model to obtain an initial task summary. The first judgment unit is used to determine whether the logic of the initial task summary is complete through the auxiliary model; A data correction unit is used to correct the initial task summary through the main model to obtain the target task summary when the first judgment unit determines that the initial task summary logic is incomplete. The first data transformation unit is used to convert the target task summary into structured prompt words through the auxiliary model; The optimization unit is used to evaluate and optimize the structured prompt words based on the domain knowledge database to obtain preferred prompt words; A data filtering unit is used to filter the preferred prompt words for compliance, thereby obtaining compliant prompt words; The second judgment unit is used to evaluate the complexity of the compliance prompt words through a lightweight evaluation model and obtain the evaluation result; The first generation unit is used to call the expert model to generate text content based on the compliance prompt words when the second judgment unit determines that the evaluation result is simple. The second generation unit is used to call the large language model to generate text content based on the compliance prompt words when the second judgment unit determines that the evaluation result is complex. The optimization unit is specifically used for: Based on the structured prompt term retrieval domain knowledge database, a set of candidate prompt terms is generated; The API call cost of each prompt word in the set of alternative prompt words is calculated using a parallel testing method, and a token consumption data report for each prompt word is obtained. The expected effect of each prompt word is evaluated based on the Token consumption data report, and the expected satisfaction score is obtained. The expected satisfaction score for each prompt word is normalized and weighted according to the satisfaction weight to obtain the effect dimension score; Based on the cost weight, the relative tokens in the token consumption data report of each prompt word are inversely weighted to obtain the cost dimension score; The comprehensive evaluation score is obtained by adding the scores of the effectiveness dimension and the cost dimension. Based on the comprehensive evaluation score, each of the candidate prompt words is sorted, resulting in a score-sorted list; The highest-rated prompt word in the score ranking list is selected as the preferred prompt word.

7. An intelligent agent prompt word inference system, characterized in that, The system includes: Processor, memory, input / output units, and bus; The processor is connected to the memory, the input / output unit, and the bus; The memory stores a program, which the processor invokes to perform the method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium contains a program that, when executed on a computer, performs the method as described in any one of claims 1 to 5.

Images