Defect case-based prompt word closed-loop optimization method and device, equipment and medium

Abstract

This invention relates to the field of artificial intelligence technology, and discloses a method, apparatus, device, and medium for closed-loop optimization of prompt words based on defect cases. The method includes: acquiring defective dialogue cases and preprocessing them to generate standardized defective dialogue cases; locating problems in the standardized defective dialogue cases to determine the target intelligent agent, the problem prompt word module within the target intelligent agent, and configuration identifiers, and generating a location report; extracting and analyzing defects in the standardized defective dialogue cases to generate optimization direction information; updating the original prompt words in the problem prompt word module to generate target prompt words; verifying the effect of the target prompt words to obtain verification results; if the verification is successful, updating the prompt word content of the corresponding prompt word module according to the configuration identifiers. This invention can be applied to service dialogue scenarios in the medical and fintech fields, and has the advantages of achieving accurate problem location of defective cases, intelligent optimization of prompt words, and closed-loop verification of effects.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology and can be applied to the medical and financial technology fields. In particular, it relates to a method, device, equipment and medium for closed-loop optimization of prompt words based on defect cases. Background Technology

[0002] With the rapid development of artificial intelligence technology, intelligent dialogue systems based on large language models have been widely used in highly specialized fields such as fintech and healthcare. In fintech scenarios, these systems undertake key tasks such as account inquiries, financial product consultations, risk assessments, and business processing guidance; in the healthcare field, the systems provide services such as preliminary symptom analysis, departmental referrals, medication consultations, and report interpretation. To implement complex business logic, current mainstream intelligent dialogue systems generally adopt multi-agent collaboration or modular architecture designs, decomposing the complete dialogue process into multiple independent stages such as intent recognition, slot extraction, business question answering, and process guidance. Each stage is driven by a specific prompt word module that drives the large language model to generate corresponding content.

[0003] However, in applications with extremely high accuracy and security requirements, such as financial transactions and medical diagnosis, defects inevitably occur during system operation, such as responses deviating from user needs, incorrect technical terminology, unclear process guidance, missing information, or logical contradictions. Existing technologies face significant challenges in handling these defects: once a defect is detected by user feedback or system monitoring mechanisms, the troubleshooting and remediation process heavily relies on manual intervention. Technicians must manually trace the entire dialogue history, analyze intent recognition results and dialogue flow paths, and examine each involved agent and prompt word module to determine the root cause. Due to the large number of system modules and the intricate call chains, this process is not only time-consuming and labor-intensive, but the accuracy of the location also highly depends on the experience and domain knowledge of the technicians. Once the problematic module is located, the modification of the prompt words also requires manual work, making it difficult to cope with the need for rapid response to batches of defect cases. More importantly, in existing technologies, the discovery of defect cases, problem localization, and prompt word optimization are isolated processes, lacking an effective automated closed-loop mechanism. Summary of the Invention

[0004] The purpose of this application is to propose a method, apparatus, device, and medium for closed-loop optimization of prompt words based on defect cases, which has the advantages of accurate positioning of defect cases, intelligent optimization of prompt words, and closed-loop verification of effects.

[0005] To address the aforementioned technical problems, embodiments of this application provide a method for optimizing prompt words based on defect cases, including: Obtain defective dialogue cases and preprocess them to generate standardized defective dialogue cases; The standardized defect dialogue cases are used to locate problems, thereby determining the target agent corresponding to the standardized defect dialogue cases, the problem prompt word module and configuration identifier within the target agent, and generating a location report; Defects are extracted and analyzed from the standardized defect dialogue cases to generate optimization direction information; Based on the optimized direction information and the positioning report, the original prompt words in the problem prompt word module are updated to generate target prompt words; The effectiveness of the target prompt words was verified, and the verification results were obtained. If the verification result is successful, the prompt word content of the corresponding prompt word module is updated according to the configuration identifier.

[0006] To address the aforementioned technical problems, embodiments of this application provide a prompt word closed-loop optimization device based on defect cases, comprising: The case acquisition module is used to acquire defect dialogue cases and preprocess the defect dialogue cases to generate standardized defect dialogue cases. The problem localization module is used to localize the problems in the standardized defect dialogue cases, so as to determine the target agent corresponding to the standardized defect dialogue cases, the problem prompt word module and configuration identifier in the target agent, and generate a localization report; The defect analysis module is used to extract and analyze defects from the standardized defect dialogue cases and generate optimization direction information. The prompt word update module is used to update the original prompt words in the problem prompt word module based on the optimization direction information and the positioning report, and generate target prompt words; The effect verification module is used to verify the effect of the target prompt words and obtain the verification results. The prompt word content update module is used to update the prompt word content of the corresponding prompt word module according to the configuration identifier if the verification result is successful.

[0007] To solve the above-mentioned technical problems, one technical solution adopted by the present invention is to provide a computer device, including one or more processors; and a memory for storing one or more programs, such that the one or more processors implement the closed-loop optimization method for prompt words based on defect cases as described above.

[0008] To solve the above-mentioned technical problems, one technical solution adopted by the present invention is: a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the prompt word closed-loop optimization method based on defect cases as described above.

[0009] This invention provides a method, apparatus, device, and medium for closed-loop optimization of prompt words based on defect cases. By constructing an automated closed-loop process, it achieves accurate identification and efficient processing of defect cases, solving the problems of low efficiency caused by manual fault diagnosis and fragmented processes in the prior art. It has the advantages of automatic collection of defect cases, accurate problem location, intelligent optimization of prompt words, and closed-loop verification of effects, thereby significantly improving the fault diagnosis efficiency and iteration speed of multi-module dialogue systems and ensuring the stability of dialogue quality. Attached Figure Description

[0010] To more clearly illustrate the solutions in this application, the accompanying drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 This is a schematic diagram of an application environment for a prompt word closed-loop optimization method based on defect cases in one embodiment of the present invention; Figure 2 This is a flowchart illustrating the implementation of the closed-loop optimization method for prompt words based on defect cases provided in this application embodiment; Figure 3 yes Figure 2 A flowchart illustrating a specific implementation method of step S1; Figure 4 yes Figure 2 A flowchart illustrating a specific implementation method of step S2; Figure 5 yes Figure 2 A flowchart illustrating a specific implementation method of step S3; Figure 6 yes Figure 2 A flowchart illustrating a specific implementation of step S4; Figure 7 yes Figure 2 A flowchart illustrating a specific implementation of step S5; Figure 8 yes Figure 2 A flowchart illustrating a specific implementation of step S6; Figure 9 This is a schematic diagram of a prompt word closed-loop optimization device based on defect cases provided in an embodiment of this application; Figure 10 This is a schematic diagram of the computer device provided in the embodiments of this application. Detailed Implementation

[0012] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein in the specification of the application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application; the terms "comprising" and "having," and any variations thereof, in the specification, claims, and foregoing drawings of this application, are intended to cover non-exclusive inclusion. The terms "first," "second," etc., in the specification, claims, or foregoing drawings of this application are used to distinguish different objects, not to describe a particular order.

[0013] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0014] Traditional multi-module intelligent dialogue systems, when applied in professional fields such as fintech and healthcare, suffer from significant reliance on manual intervention for troubleshooting and resolving issues such as inaccurate responses, incorrect business information, and improper process connections. Technical personnel must manually trace back the dialogue context, examining each involved agent and prompt word module to pinpoint the root cause. This process is time-consuming, and its accuracy depends heavily on the experience of the troubleshooters. Similarly, prompt word modification relies on manual intervention, making it difficult to quickly respond to a large number of problematic cases. More importantly, the discovery of problematic cases, problem localization, and prompt word optimization are fragmented, lacking an automated closed-loop linkage mechanism. This results in long system iteration response cycles, difficulty in consistently ensuring dialogue quality, and significant potential risks.

[0015] To address this, this application proposes a closed-loop optimization method for prompt words based on defect cases. This method involves acquiring and preprocessing defective dialogue cases to generate standardized defective dialogue cases; locating problems in these standardized cases to determine the target agent, the prompt word module, and the configuration identifier, and generating a location report; extracting and analyzing defects in the standardized dialogue cases to generate optimization direction information; updating the original prompt words in the prompt word module based on the optimization direction information and the location report to generate target prompt words; validating the target prompt words to obtain the verification results; and updating the prompt word content of the corresponding prompt word module according to the configuration identifier if the verification result is successful.

[0016] For ease of understanding, the following explains some key terms in this embodiment: Defective dialogue cases refer to dialogue records in which intelligent dialogue systems exhibit poor user experience or abnormal system performance due to inaccurate answers, incorrect information, unclear process guidance, or other reasons during actual operation. These cases are usually identified through user feedback or system quality monitoring mechanisms.

[0017] Standardized defect dialogue cases refer to pre-processed defect dialogue cases whose content and format are standardized to facilitate subsequent automated processing steps such as problem localization, defect analysis, and prompt word optimization.

[0018] Problem localization refers to the process of identifying the specific agent that caused the defect, the problem prompt word module within that agent, and the corresponding configuration identifier based on standardized defect dialogue cases.

[0019] In a multi-agent collaborative architecture, the target agent is the agent that is identified by the problem localization process as being directly related to the defective dialogue case.

[0020] The problem prompt word module refers to the specific prompt word module within the target intelligent agent that is identified by the problem localization process as causing the defect to occur.

[0021] The configuration identifier is a unique identifier used to uniquely identify and locate the problem prompt word module in the system configuration database.

[0022] The problem localization report is the output of the problem localization process. It contains key information such as the target agent, problem prompt word module, and configuration identifier, providing clear guidance for subsequent optimization work.

[0023] Defect extraction and analysis refers to the in-depth analysis of standardized defect dialogue cases, identifying defect types, and determining the corresponding optimization directions and priorities.

[0024] Optimization direction information refers to the output of the defect extraction and analysis process, which includes defect type, optimization direction, and optimization priority, and is used to guide the updating of prompt words.

[0025] The original prompt words refer to the prompt words in the problem prompt word module before optimization and updates.

[0026] Target cue words refer to the new cue words generated after updating the original cue words based on optimization direction information and positioning reports.

[0027] Effectiveness verification refers to the process of testing the generated target prompts to evaluate whether they effectively solve the original defects and meet the preset quality standards.

[0028] The validation result refers to the output of the effect validation process, indicating whether the target prompt has passed the validation, such as "validation passed" or "validation failed".

[0029] This application achieves a closed-loop update of prompt content by automating the processing of defective dialogue cases, accurately locating problematic modules, systematically analyzing defects and generating optimization directions, intelligently updating prompt words, and verifying effects from multiple dimensions. This effectively solves the problems of traditional multi-module intelligent dialogue systems in professional fields such as finance and healthcare, where defect detection and repair rely excessively on manual processes and are fragmented across different stages. It significantly improves troubleshooting efficiency and system iteration response speed, continuously ensures dialogue quality, and reduces potential risks in financial transactions and medical advice.

[0030] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0031] It should be noted that the closed-loop optimization method for prompt words based on defect cases provided in this application embodiment is generally executed by the server, and correspondingly, the closed-loop optimization device for prompt words based on defect cases is generally configured in the server.

[0032] The prompt word closed-loop optimization method based on defect cases provided in this invention can be applied to, for example... Figure 1 In this application environment, the client communicates with the server via a network. The server can receive the client's dialogue records and generate corresponding prompts based on these records. In this invention, the server sends the optimized prompts to the client. The client can be, but is not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices. The server can be implemented using a standalone server or a server cluster consisting of multiple servers. The invention will now be described in detail through specific embodiments.

[0033] The closed-loop optimization method for prompt words based on defect cases provided in this application can be applied to service dialogue scenarios in the medical and fintech fields.

[0034] Please see Figure 2 , Figure 2 This paper illustrates a specific implementation of a closed-loop optimization method for prompt words based on defect cases.

[0035] It should be noted that if substantially the same result is obtained, the method of this invention is not based on... Figure 2 Limited to the order of the processes shown, this method includes the following steps: S1: Obtain defective dialogue cases and preprocess them to generate standardized defective dialogue cases.

[0036] Specifically, defect dialogue cases can be obtained through proactive channels and passively driven mechanical energy. Subsequently, these original defect dialogue cases are formatted, sensitive information is anonymized, and content is screened to ensure that they conform to the unified specifications for subsequent processing, thereby generating standardized defect dialogue cases.

[0037] Please see Figure 3 , Figure 3 A specific implementation of step S1 is shown below: S11: Collect user-marked dialogue records through the user feedback entry point, and identify dialogue records exceeding the threshold through preset quality monitoring indicators to obtain the defective dialogue cases. S12: Perform data preprocessing on the defective dialogue cases to obtain preprocessed defective dialogue cases. S13: Extract the user question content, dialogue flow path, interaction records of each agent's prompt word module, final reply content, and user feedback tags from each preprocessed defective dialogue case, and assemble the extracted information into the standardized defective dialogue case.

[0038] Among these, collecting user-marked dialogue records through the user feedback portal refers to the system providing users with a channel to proactively report dialogue problems or dissatisfaction. Users can use this channel to mark specific dialogue records, indicating their defects. For example, buttons such as "Dissatisfied," "Feedback Issue," or "Report Error" can be set in the dialogue interface, allowing users to select dialogue segments or provide brief descriptions after clicking. Alternatively, defect reports proactively submitted by users, containing specific dialogue record IDs or content, can be collected through user satisfaction surveys, customer service hotlines, or email feedback systems. Simultaneously, identifying dialogue records exceeding thresholds through preset quality monitoring indicators means that the system uses automated methods to analyze dialogue records in real-time or offline according to pre-set standards for measuring dialogue quality to discover potential defects. For example, indicators such as "excessive response time", "too many repeated questions from users", "high conversation interruption rate", and "high frequency of negative sentiment words" can be set. When these indicators exceed the preset threshold, the system will automatically mark the conversation record as a defective case. Alternatively, machine learning models can be used to perform semantic analysis on the conversation content to identify patterns such as "response does not match user intent", "information is missing", and "logical errors", and the system can determine whether the threshold is exceeded based on the model confidence or error rate.

[0039] Data preprocessing of the defective dialogue cases aims to clean and standardize the original defective dialogue case data, remove redundant information, and unify the data format to provide high-quality input for subsequent feature extraction and analysis. For example, this may include removing non-text content such as HTML tags, special characters, and emoticons; performing natural language processing operations such as word segmentation, part-of-speech tagging, and named entity recognition; and standardizing the text by performing case conversion and synonym replacement. Alternatively, it may involve format validation and standardization of metadata such as dialogue timestamps, user IDs, and session IDs; identifying and merging session boundaries in multi-turn dialogues; and performing deduplication and error correction on the text content.

[0040] Extracting user questions from each preprocessed defective dialogue case aims to obtain the specific questions or instructions raised by the user in the dialogue. This is core information for understanding user intent and evaluating the accuracy of agent responses. For example, text messages sent by the user can be directly parsed from the dialogue log; or, for multi-turn dialogues, user questions from each turn can be extracted and semantically analyzed in conjunction with the context to obtain a complete expression of user intent. Extracting the dialogue flow path aims to record key nodes such as agent modules, intent recognition results, and slot filling status from the beginning to the end of the dialogue, which helps to track where the problem occurred. For example, the call order and output results of components such as the intent recognizer, slot extractor, and business logic module can be obtained by parsing the system's internal dialogue management logs; or, specific business process IDs triggered during the dialogue, jump nodes, and collaborative call relationships between agents can be recorded. Extracting the interaction records of each agent's prompt word module aims to obtain detailed information on how each agent module calls its internal prompt words during the dialogue and how these prompt words affect the large model's generated responses, which is key to locating prompt word problems. For example, it's possible to record the prompt word templates, filled variable values, and raw output returned by the large model that each agent module actually sends to the large model when processing user requests; or, to track the number of calls, input parameters, output results, and scope and impact of a specific prompt word module throughout the dialogue process. Extracting the final response content aims to obtain the text of the response ultimately presented to the user by the agent. This is information directly perceived by the user and a direct basis for evaluating the agent's performance and shortcomings. For example, the last message sent to the user by the agent can be directly obtained from the dialogue log; or, for scenarios with multiple responses, the specific response that the user believes caused the defect, or the final summary response of the entire dialogue record, can be extracted. Extracting user feedback tags aims to attach the defect types actively marked by the user or the defect categories automatically identified by the system as important metadata to the defect cases, guiding subsequent defect analysis and optimization directions. For example, if a user selects a preset defect type (such as "inaccurate answer" or "incorrect information") through the feedback entry, the label is recorded directly; or, if the defect is identified through quality monitoring indicators, the system can automatically generate corresponding defect labels based on the triggered indicator type (such as "response timeout" or "intent recognition failure"). Assembling the extracted information into the standardized defect dialogue case aims to integrate the extracted key information in a unified, structured format, forming standard data units that are easy to process and analyze automatically later. For example, all extracted information can be encapsulated into a JSON or XML data structure, containing explicit field names and corresponding values; or, a database table structure can be defined to store each piece of information as a different field, ensuring data consistency and queryability.

[0041] S2: Locate the problem in the standardized defect dialogue case to determine the target agent corresponding to the standardized defect dialogue case, the problem prompt word module and configuration identifier in the target agent, and generate a location report.

[0042] Specifically, based on the user's questions, dialogue flow, and final response in standardized defect dialogue cases, combined with an understanding of the system architecture, it's possible to analyze and determine which agent might be causing the problem. Once the target agent is identified, the various prompt word modules within that agent are examined to find the module most relevant to the defect, and the unique identifier of that module in the system configuration is recorded. Finally, this information is compiled into a location report.

[0043] Please see Figure 4 , Figure 4 A specific implementation of step S2 is shown below: S21: Input the user's question content, dialogue flow path, and final response content from the standardized defect dialogue case into the intent distribution and fault analysis model for analysis, and output the target agent to which the problem belongs. S22: Based on the functional division and collaboration logic of each prompt word module within the target agent, identify the problem prompt word module directly related to the standardized defect dialogue case, and obtain the unique configuration identifier of the problem prompt word module from the configuration database. S23: Generate the location report based on the target agent, the problem prompt word module, and the unique configuration identifier.

[0044] Specifically, the user's question content, dialogue flow path, and final response content from standardized defective dialogue cases are input into an intent distribution and fault analysis model for analysis, to output the target agent to which the problem belongs. This step aims to preliminarily determine, through automated analysis, which specific agent the defective dialogue case occurred in. The user's question content is the starting point of the user's interaction with the system, directly reflecting the user's intent and question; the dialogue flow path records the jumps and interaction order between different agents or modules, revealing the possible stages of the problem; the final response content reflects the system's final response to the user's question and is an important basis for judging the nature of the defect. Inputting this key information into the intent distribution and fault analysis model allows the model to comprehensively analyze this data and identify the agent most likely to cause the defect. For example, the intent distribution and fault analysis model can be a deep learning-based multi-classification model, such as using the Transformer architecture, by encoding the sequence information of the user's question content, the dialogue flow path, and the textual features of the final response content, and then training the model to identify fault patterns related to a specific agent. Alternatively, the model can be a combination of rules and machine learning. First, it uses preset rules such as keyword matching and regular expressions to initially filter out the range of possible agents. Then, it uses machine learning models such as support vector machines (SVM) or random forests, combined with the structured features in the dialogue flow path and response content, to further refine and confirm the target agents.

[0045] Based on this, and according to the functional division and collaboration logic of each prompt word module within the target agent, the problem prompt word module directly related to the standardized defective dialogue case is identified, and its unique configuration identifier is obtained from the configuration database. After determining the target agent to which the problem belongs, this step further focuses on the specific prompt word modules within that agent. Each agent typically consists of multiple prompt word modules, each performing a specific function and collaborating according to pre-defined logic. By analyzing these functional divisions and collaboration logics, the root cause prompt word module leading to the defect can be accurately identified. Obtaining the unique configuration identifier ensures that subsequent modifications to this module can be made accurately. For example, by constructing a module dependency graph or flowchart within the agent, combined with the module interaction records in the defective dialogue case, the actual path of the dialogue flow between modules can be traced, and compared with the pre-defined functional divisions and collaboration logic, thereby locating the abnormal or unexpected prompt word module. Furthermore, by defining the scope of responsibility and input / output specifications for each prompt word module, a module can be identified as a prompt word module when the context information of a defective dialogue case (such as user intent, slot information, and output of preceding modules) does not match the input conditions of a certain module, or when the output of that module does not meet expectations. Simultaneously, the configuration database stores the metadata of each prompt word module, including its unique configuration identifier, which can be retrieved through a query.

[0046] Finally, a location report is generated based on the target agent, the question prompt word module, and the unique configuration identifier. This step aims to integrate the key information identified in the previous two steps to form a structured location report. This report is a crucial input for subsequent defect analysis, prompt word optimization, and effect verification, ensuring the accuracy and completeness of information transmission. For example, the location report can be a structured document in JSON or XML format, containing fields such as "target agent name," "question prompt word module name," "unique configuration identifier," "location time," and "related dialogue case ID," facilitating programmatic parsing and subsequent processing. Alternatively, the location report can also be a database record, storing information such as the target agent, question prompt word module, and unique configuration identifier as fields in a dedicated location report table, and associating it with the original defective dialogue cases for easy querying and management.

[0047] This application's embodiments introduce an intent distribution and fault analysis model, and combine it with the functional division and collaborative logic of modules within the target intelligent body to achieve automated, hierarchical localization of defective dialogue cases. This scheme utilizes comprehensive dialogue information, including user question content, dialogue flow path, and final response content, to quickly identify the macro-level intelligent body to which the problem belongs. Subsequently, within the identified intelligent body, the system further pinpoints the specific prompting word module causing the defect based on the established responsibilities and collaborative relationships of each prompting word module, and obtains its unique configuration identifier. Finally, these key location information are integrated to generate a localization report, providing an accurate and reliable basis for subsequent defect analysis and prompting word optimization. This hierarchical automated localization method effectively solves the problems of low efficiency, reliance on experience for accuracy, and lack of standardized processes in traditional manual localization. Initial model screening significantly narrows the scope of manual investigation, avoiding blind spots; precise localization through module logic ensures accurate identification of the root cause of the problem, avoiding redundant investigation across the entire system. Obtaining a unique configuration identifier ensures the accuracy of subsequent prompting word update operations, avoiding update mismatches. Overall, this solution significantly improves the efficiency and accuracy of problem localization, providing a solid foundation for the entire closed-loop optimization process of prompt words. It enables the system to respond quickly and handle defect cases automatically, thereby continuously ensuring the service quality of the intelligent dialogue system.

[0048] S3: Extract and analyze defects from the standardized defect dialogue cases to generate optimization direction information.

[0049] Specifically, based on the semantic analysis capabilities of the large model, the defect types in standardized defective dialogue cases are identified (such as answers deviating from requirements, incorrect professional terminology, unclear process guidance, missing information, etc.). Combined with industry standards, the corresponding optimization directions for the defects are marked (such as supplementing professional terminology constraints, strengthening contextual connections, optimizing process guidance logic, etc.). Simultaneously, based on the severity of the defect (the degree of impact on medical professionalism) and the frequency of defect types occurring in the problem module, the optimization priority of the prompt word module is determined.

[0050] Please see Figure 5 , Figure 5 A specific implementation of step S3 is shown below: S31: Input the standardized defect dialogue case and the location report into the defect classification model to identify the defect type, wherein the defect type includes answers deviating from requirements, incorrect technical terms, unclear process guidance, missing information, or logical contradictions. S32: Determine the optimization direction corresponding to the defect type based on the defect type and preset domain specifications. S33: Query the defect occurrence frequency of the problem prompt word module from the historical defect case database, and calculate the optimization priority score and level based on the preset defect severity score. S34: Determine the optimization priority of the problem prompt word module based on the optimization priority score and level, and generate the optimization direction information based on the optimization direction and the optimization priority.

[0051] Specifically, the standardized defect dialogue cases refer to preprocessed and structured defect dialogue data, which includes key information such as user question content, dialogue flow path, interaction records of each agent's prompt word module, final response content, and user feedback tags, providing comprehensive and unified input data for defect analysis. The localization report, generated during the problem localization phase, clarifies the target agent corresponding to the standardized defect dialogue case, the prompt word module within the target agent, and its unique configuration identifier. Its function is to provide the large-scale defect classification model with the specific context and module affiliation information of the problem, assisting the large-scale model in more accurately understanding the root cause of the defect. The large-scale defect classification model refers to an artificial intelligence model built based on deep learning technology, possessing powerful text understanding and classification capabilities, such as a pre-trained language model based on the Transformer architecture or a model specifically fine-tuned for dialogue defect classification tasks. This model can receive structured or unstructured text input and classify it into preset defect categories based on its semantic content, thereby automatically and intelligently identifying defect types in the dialogue system, replacing human experience-based judgment, and improving the accuracy and efficiency of classification. The aforementioned defect types refer to predefined categories of specific problems that describe poor performance of the dialogue system in particular scenarios. Examples include: answers deviating from the user's needs (the agent's response fails to accurately address the user's intent or the core of the question), incorrect terminology (the use of inaccurate or inappropriate technical terms in the response), unclear process guidance (the agent's guidance to the user in completing a business process is vague or illogical), missing information (the response fails to provide all the key information the user needs), or logical contradictions (inconsistent content in the response or contrary to common sense). These defect types provide a clear classification basis for subsequent optimization.

[0052] The pre-defined domain specifications refer to a series of behavioral guidelines, business rules, compliance requirements, professional knowledge systems, or best practices pre-established for specific business domains (such as finance and healthcare). These specifications can be stored in a knowledge base, rule engine, or expert system, serving as the basis for guiding agent behavior and content generation. This ensures that defect repair and prompt word optimization not only solve technical problems but also comply with industry standards and business requirements, avoiding optimization directions that deviate from actual business needs. The optimization direction refers to specific improvement strategies, modification suggestions, or action plans proposed for specific defect types. For example, for the defect type of "answer deviating from needs," the optimization direction might be "enhancing the accuracy of intent recognition" or "expanding relevant knowledge base content"; for "incorrect professional terminology," the optimization direction might be "correcting professional vocabulary in prompt words" or "introducing a domain terminology verification mechanism." Its function is to provide clear and actionable guidance for subsequent prompt word updates.

[0053] The historical defect case database refers to a system used to store and manage all discovered, processed, or ongoing defect dialogue cases and related information. This database typically contains data such as defect type, occurrence time, involved modules, repair records, and user feedback, providing historical statistical data for trend analysis and priority assessment. The defect occurrence frequency refers to the number or proportion of times a specific issue prompt module exhibits a certain defect type within a certain time period. For example, the number of times a prompt module caused an "information missing" defect in the past month. Its function is to quantify the prevalence or repetitiveness of defects, serving as an important indicator for evaluating optimization priorities. The preset defect severity score refers to a numerical value or level that quantitatively assesses the defect's impact on business, user experience, compliance risks, etc. For example, defect severity can be divided into three levels: "high," "medium," and "low," or assigned a numerical value of 1-5, where "high" or "5" indicates a wide-ranging and highly harmful defect. This score can be defined by domain experts or automatically calculated through user feedback and business loss assessment models. Its function is to objectively measure the potential harm of defects and provide an important basis for priority calculation. The optimization priority score and level refer to the result of quantitatively ranking defects after comprehensively considering their occurrence frequency and severity. For example, a comprehensive score can be calculated using weighted average, product, or other algorithms, and then mapped to priority levels such as "urgent," "high," "medium," and "low." Its purpose is to provide an objective and scientific basis for resource allocation and optimization order. The optimization priority refers to determining the order or importance of which defects should be prioritized among multiple defects to be optimized. For example, high-priority defects should be processed immediately, while low-priority defects can be delayed. Its purpose is to guide the rational allocation of optimization resources and ensure that critical issues are resolved in a timely manner. The optimization direction information refers to a structured data package containing optimization directions and corresponding optimization priorities for specific defect types and problem prompt word modules. For example, it can be a JSON object containing fields such as "defect_type," "optimization_direction," "priority_score," and "priority_level." Its purpose is to serve as input for the subsequent prompt word update module, providing comprehensive and prioritized optimization guidance.

[0054] This application effectively solves the problems of inaccurate defect analysis, unclear optimization direction, and unreasonable optimization priority during the optimization of prompt words. Specifically, standardized defect dialogue cases and location reports are input into a large defect classification model. Leveraging the powerful semantic understanding capabilities of the large model, combined with the module affiliation information provided in the location report, defect types can be identified more accurately. This avoids the bias that may occur when relying solely on dialogue content for classification, significantly improving the accuracy and standardization of defect identification. Preset defect types further unify the classification standards, ensuring the objectivity of defect analysis. Based on this, optimization directions are determined according to the identified defect types and preset domain specifications, ensuring that the optimization strategy is not only technically feasible but also conforms to the business requirements and compliance standards of professional fields such as finance and healthcare, avoiding a disconnect between the optimization direction and actual business needs. Furthermore, by querying the frequency of defect occurrence of the problem prompt word module from a historical defect case database and combining it with a preset defect severity score to calculate the optimization priority score and level, this application introduces an objective and quantitative evaluation mechanism. This allows optimization priorities to move beyond subjective judgment based on human experience. Instead, it enables scientific assessment based on the actual frequency of defects and their potential impact on business operations, accurately reflecting the actual impact of defects on the overall dialogue quality of the system. The resulting optimization direction information includes clear optimization directions and priorities, providing clear and focused guidance for subsequent prompt word updates. Through the above technical solution, this application automates and standardizes the defect extraction and analysis process, ensuring the accuracy and rationality of optimization directions. It also enables priority ranking based on the actual impact of defects, allowing for the rational allocation of optimization resources and prioritizing critical defects with significant impact and high frequency of occurrence. This significantly improves the efficiency and effectiveness of prompt word optimization iteration, ensuring that the intelligent dialogue system can continuously provide high-quality services in complex business scenarios and effectively reducing the risks associated with financial transactions and the professional expertise required for medical advice.

[0055] S4: Update the original prompt words in the problem prompt word module based on the optimization direction information and the positioning report, and generate target prompt words.

[0056] Specifically, the optimization direction information and positioning report are input into the automated prompt word rewriting and optimization engine to call the prompt word optimization rule base and the domain case base corresponding to the defective dialogue cases, update the original prompt words in the problem prompt word module, and generate target prompt words.

[0057] Please see Figure 6 , Figure 6 A specific implementation of step S4 is shown below: S41: Invoke the prompt word optimization rule base and the preset domain case base to process the optimization direction information and the original prompt words, generating a preliminary rewriting scheme. S42: Perform semantic reconstruction and verification on the preliminary rewriting scheme to obtain candidate prompt words. S43: Pre-evaluate the candidate prompt words. If the candidate prompt words meet the preset basic quality requirements, then use the candidate prompt words as the target prompt words; if the candidate prompt words do not meet the preset basic quality requirements, then adjust the optimization strategy and regenerate the candidate prompt words until a target prompt word that meets the preset basic quality requirements is output.

[0058] In this embodiment, the system first invokes a prompt word optimization rule base and a preset domain case library to process the optimization direction information and the original prompt words, generating a preliminary rewriting scheme. The prompt word optimization rule base aims to provide structured guidance, ensuring that the rewriting of prompt words conforms to specific optimization goals and domain specifications. For example, this rule base can be a set of rules built based on expert experience and domain knowledge, including rewriting templates, keyword replacement strategies, and sentence structure adjustment guidelines for different defect types (such as answers deviating from requirements or incorrect terminology); or it can be a rule reasoning engine trained by a machine learning model, capable of automatically generating or recommending rewriting rules based on the input optimization direction information. The preset domain case library is used to store and manage historically successful prompt word optimization cases, providing reference and guidance for this optimization. For example, it can be a structured database recording the original prompt words, optimization direction, optimized prompt words, and corresponding effect verification results; or it can be an unstructured text collection, combined with an information retrieval model, capable of retrieving similar, successfully optimized cases for reference based on the current optimization direction information and the original prompt words. By combining the aforementioned prompt word optimization rule base and the preset domain case library, the optimization direction information and the original prompt words are comprehensively analyzed and processed. For example, firstly, corresponding rewriting rules can be matched from the rule base based on the optimization direction information, and relevant cases can be retrieved from the case library for reference. Then, natural language generation (NLG) technology or rule-based template filling methods can be used to generate preliminary rewritten text. The preliminary rewriting scheme refers to the prompt word draft obtained after the above processing, which has not yet undergone fine-tuning and rigorous verification. It aims to initially reflect the optimization direction, but may still have aspects that need improvement in semantics, syntax, or logic.

[0059] Secondly, the preliminary rewriting scheme undergoes semantic reconstruction and verification to obtain candidate prompt words. Semantic reconstruction aims to adjust the expression logic and structure of the preliminary rewriting scheme to better adapt it to the functional division and collaborative logic of each prompt word module within the target agent, as well as the contextual interaction requirements of the dialogue. For example, techniques such as grammatical analysis and syntactic transformation can be used to reorganize the sentences in the preliminary rewriting scheme, making its expression more fluent and more in line with the agent's expected response pattern; alternatively, a large-scale language model can be used to adjust the style and structure of the preliminary rewriting scheme, making it more in line with the expression habits of the professional domain while maintaining the original meaning. Verification aims to perform error detection and standardization checks on the reconstructed scheme to correct potential semantic errors, logical conflicts, grammatical irregularities, and other fundamental problems. For example, rule-based grammar checkers and lexical analyzers can be used to identify spelling and grammatical errors; alternatively, a pre-trained language model can be used for consistency checks to ensure that the rewritten prompt words maintain the functional intent of the original prompt words without introducing new ambiguities. The candidate prompt words refer to prompt word versions that have reached a certain standard in terms of semantics, syntax and logic after semantic reconstruction and verification, but have not yet undergone final quality assessment.

[0060] Finally, the candidate prompts are pre-evaluated. If a candidate prompt meets the preset basic quality requirements, it is adopted as the target prompt. If a candidate prompt does not meet the preset basic quality requirements, the optimization strategy is adjusted and the candidate prompt is regenerated until a target prompt that meets the preset basic quality requirements is output. The pre-evaluation aims to perform a preliminary, rapid quality check on the candidate prompts to prevent unqualified prompts from entering the subsequent time-consuming effect verification stage. For example, a rule-based scoring system can be designed to quantitatively evaluate the candidate prompts' grammatical correctness, semantic clarity, and matching degree with module function descriptions; alternatively, a small amount of simulated dialogue data can be used to quickly test the candidate prompts and evaluate their initial performance in specific scenarios. The preset basic quality requirements refer to the minimum standards set for prompts in terms of syntax, semantics, and functional matching. For example, prompts may be required to be grammatically correct, unambiguous, accurately guide user intent, and not contain sensitive words. These requirements are usually in the form of configuration parameters or expert-defined standards. When candidate prompts fail to meet the preset basic quality requirements, the system adjusts the current optimization strategy based on the pre-evaluation results. For example, it can provide feedback to the prompt optimization rule base to adjust the weights or priorities of the rules; or it can modify the parameters of the generation model, or even select different domain cases for reference. Subsequently, based on the adjusted strategy, the system re-executes the steps of generating a preliminary rewrite plan, semantic reconstruction, and verification until candidate prompts that meet the basic quality requirements are generated, and these are designated as target prompts. The target prompt refers to the version of the prompt that ultimately passes the pre-evaluation, meets all preset basic quality requirements, and can proceed to the subsequent effect verification stage.

[0061] The embodiments of this application can ensure that the output target prompt words not only meet the direction requirements of this optimization, but also meet the basic quality standards, which significantly improves the operating efficiency and output quality of the entire prompt word closed-loop optimization, and ensures that the intelligent dialogue system can continuously and stably provide high-quality services when facing complex business scenarios.

[0062] S5: Verify the effect of the target prompt words and obtain the verification results.

[0063] Specifically, a dual verification method of simulation testing and traffic testing is used to verify the effectiveness of the target prompt words and obtain verification results. Simulation testing generates multiple sets of simulated dialogue data based on the functional scenarios of the target intelligent agent, which are then input into the rewritten prompt word module to verify the professionalism, accuracy, and flow adaptability of the responses. Traffic testing involves configuring the rewritten prompt word module online, allocating preset real user traffic (e.g., 1%-5% of real user traffic) for testing, and collecting user feedback data (e.g., satisfaction ratings, repeat question rate, etc.). If both simulation testing and traffic testing meet the preset quality standards, the verification is successful; otherwise, optimization is performed again.

[0064] Please see Figure 7 , Figure 7 A specific implementation of step S5 is shown below: S51: Generate multiple sets of simulated dialogue data based on the functional scenarios of the target intelligent agent, and conduct simulated tests on the target prompt words using the multiple sets of simulated dialogue data to obtain simulated test results. S52: Receive a preset proportion of real user traffic, conduct traffic tests based on the real user traffic, and collect user feedback data during the traffic test process. S53: Compare the simulated test results and the user feedback data with preset quality standards to obtain the verification results.

[0065] The generation of multiple sets of simulated dialogue data based on the functional scenarios of the target intelligent agent refers to constructing a series of dialogue data simulating user-agent interaction according to the specific business functions and application scenarios undertaken by the target intelligent agent. For example, simulated dialogue data covering different intents, slots, and dialogue paths can be automatically generated by a program based on preset dialogue templates, keyword combinations, or business process rules; alternatively, another large language model can be used to simulate and generate diverse user questions and expected system responses based on the functional description of the target intelligent agent and historical dialogue data, thereby forming multiple sets of representative simulated dialogue data. This step aims to provide comprehensive and targeted test samples for subsequent simulation testing.

[0066] The simulation test results, obtained by simulating multiple sets of dialogue data to test the target prompt, involve inputting the generated sets of simulated dialogue data into a target agent equipped with the target prompt, and observing and recording the agent's response behavior. For example, automated test scripts can be used to batch execute the simulated dialogue data and automatically evaluate and score the agent's output (such as response accuracy, fluency, and compliance with business logic); alternatively, human testers can review and judge the agent's performance on the simulated dialogue data one by one to obtain simulation test results regarding the target prompt's performance in a controlled environment. This step aims to preliminarily screen for potential problems with the target prompt, ensuring it meets basic functional and quality requirements.

[0067] Receiving a preset proportion of real user traffic, conducting traffic tests based on this traffic, and collecting user feedback data during the traffic testing process refers to deploying the target prompt word to the production environment after it passes simulation testing, but only responding to a small portion (i.e., a preset proportion) of real user requests. For example, A / B testing or canary release methods can be used to route a portion of user traffic to the agent version using the target prompt word, while simultaneously collecting explicit feedback (such as user ratings, comments, satisfaction surveys) and implicit feedback (such as dialogue completion rate, user dwell time, subsequent actions, etc.) generated during these user interactions with the agent. This step aims to obtain actual performance and user experience data of the target prompt word in real, complex user scenarios.

[0068] The verification result is obtained by comparing the simulated test results and user feedback data with preset quality standards. This involves comprehensively analyzing performance data from the simulated test phase and user feedback data from the real traffic test phase, and comparing them with predefined quality standards. For example, preset quality standards may include quantitative indicators such as response accuracy, user satisfaction, task completion rate, and error rate, as well as qualitative requirements such as the professionalism, compliance, and friendliness of the response content. By comparing the test data from both phases with these standards, a comprehensive assessment can be made as to whether the target prompt keyword meets the launch requirements, thereby obtaining the final verification result (such as "verification passed" or "verification failed"). This step aims to provide an objective and comprehensive decision-making basis for whether the target prompt keyword can be fully launched.

[0069] S6: If the verification result is successful, then update the prompt word content of the corresponding prompt word module according to the configuration identifier.

[0070] Specifically, if the effectiveness verification shows that the target prompt can effectively solve the problem, the system administrator will find the corresponding prompt record in the system's configuration database based on the configuration identifier provided in the location report, and replace the original prompt with the new target prompt to complete the update of the prompt content.

[0071] Please see Figure 8 , Figure 8 A specific implementation of step S6 is shown below: S61: If the verification result is successful, locate the corresponding prompt word record in the configuration database according to the configuration identifier, archive the original prompt word, and update it to the current version with the target prompt word, incrementing the version number. S62: Associate and archive the defective dialogue case, the location result, the optimization direction information, the original prompt word, the target prompt word, and the verification result. S63: Add the successfully optimized case to the preset domain case library and update the recognition rules for each agent.

[0072] The configuration identifier is a unique code or string used to identify a specific prompt word module. It can be a globally unique identifier (UUID) automatically generated by the system, or a structured identifier combining the agent ID and module name (e.g., "Financial Agent_Financial Consulting_Intent Recognition"). The configuration database is a persistent storage system used to store and manage all prompt word modules and their related metadata. It can be a relational database, a document database, or a configuration management system with integrated version control. The archiving operation of original prompt words aims to back up and store the currently used prompt word content to preserve its historical versions. This can be achieved by setting status fields (such as "active" or "archived") in the configuration database or by migrating old version data to a history table. Updating the target prompt word refers to replacing the corresponding original prompt word content in the configuration database with the optimized and validated new prompt word content, making it the currently effective version. Version number incrementing typically uses semantic versioning (such as major version number.minor version number.revision number) or a simple integer increment method to clearly identify different iterations of the prompt word, facilitating traceability and management.

[0073] Linked archiving refers to the unified storage and association of all key data involved in a complete prompt word optimization process. This data includes the defective dialogue cases that triggered the optimization, the problem localization report generated during the problem localization phase, the optimization direction information generated during the defect extraction and analysis phase, the original prompt words before optimization, the target prompt words after optimization, and the final effect verification results. Linked archiving can be achieved by creating a unified optimization record table in a database and using foreign keys to link to specific records in each independent data table; alternatively, all relevant information can be packaged into a structured data object (such as JSON or XML), stored in a document database or object storage, and indexed by a unique optimization session ID.

[0074] The pre-defined domain case library is a knowledge base specifically designed to store and manage successful optimization cases. This library can be a structured database table containing fields such as case ID, defect type, pre-optimization prompt words, post-optimization prompt words, optimization direction, and domain; or it can be an unstructured collection of documents, searchable by tags and keywords. Its purpose is to provide reference and guidance for subsequent prompt word optimization. Updating the recognition rules of each agent refers to adjusting the rule set or model parameters used by the agent to recognize user intent, classify defect types, or make other decisions based on the experience of this successful optimization. For example, if this optimization solved a recognition error for a specific intent, the training data or rule base of the intent recognition model can be updated; if it solved a misjudgment of a defect type, the feature weights or decision boundaries of the defect classification model can be updated. This can be achieved by retraining the machine learning model, modifying the rule engine's configuration file, or updating the entity relationships in the knowledge graph.

[0075] This application effectively solves the problems of chaotic version management, difficulty in information traceability, and difficulty in accumulating and reusing optimization experience in the closed-loop process of prompt word optimization. Specifically, by locating and archiving the original prompt words based on configuration identifiers, and updating the current version with the target prompt word and incrementing the version number, fine-grained management of prompt word versions is achieved. This not only ensures that each update has a clear historical record for backtracking, avoiding version confusion, but also enables rapid location of the specific version when new problems occur, greatly improving the efficiency of troubleshooting and problem reproduction. In addition, the defective dialogue cases, location results, optimization direction information, original prompt words, target prompt words, and verification results of this optimization are linked and archived to construct a complete optimization process chain. This allows the entire process information from defect discovery to final verification to be completely preserved, providing comprehensive data support for subsequent review analysis, effect evaluation, and prevention of similar problems, significantly enhancing the traceability and transparency of the optimization process.

[0076] In one specific embodiment, suppose in a financial intelligent dialogue system, user A inquires with an intelligent assistant about detailed information on "a certain high-yield financial product". In its reply, the intelligent assistant incorrectly describes "historical annualized rate of return" as "expected annualized rate of return" and fails to mention the product's risk level, causing user A to give negative feedback that the reply is "inaccurate".

[0077] First, the system acquires and preprocesses the defective dialogue case to generate standardized defective dialogue cases. Specifically, the system collects dialogue sessions marked by user A through the user feedback entry point and identifies them as defective dialogue cases. Then, the system preprocesses the data of the defective dialogue case, cleaning and standardizing the raw dialogue data. Next, the system extracts the user's question content ("a high-yield financial product"), the dialogue flow path (e.g., user question, intent recognition module, product information introduction module, response generation module), the interaction records of each agent's prompt word module, the final response content (including incorrect statements), and the user feedback tag ("inaccurate") from the preprocessed defective dialogue case, and assembles this extracted information into standardized defective dialogue cases. This process achieves automated collection and structuring of defective cases, avoiding the inefficiency of manual screening and organization.

[0078] Next, the system locates the problem in the standardized defective dialogue case to identify the target agent, the problem prompting word module, and the configuration identifier, and generates a location report. The system inputs the user's question content, dialogue flow path, and final response content from the standardized defective dialogue case into an intent distribution and fault analysis model for analysis. This model, by analyzing the dialogue context and response content, outputs that the target agent to which the problem belongs is the "financial product consultation agent." Subsequently, based on the functional division and collaboration logic of the various prompting word modules within the "financial product consultation agent" (e.g., the product information introduction module is responsible for providing product details), the system identifies the problem prompting word module directly related to the standardized defective dialogue case as the "product information introduction prompting word module." Simultaneously, the system retrieves the unique configuration identifier of this "product information introduction prompting word module" from the configuration database. Finally, the system generates a location report based on the "financial product consultation agent," the "product information introduction prompting word module," and this unique configuration identifier. This automated location process significantly improves the efficiency and accuracy of fault diagnosis, solving the problems of time-consuming manual troubleshooting and reliance on experience in existing technologies.

[0079] Then, the system extracts and analyzes standardized defect dialogue cases to generate optimization direction information. The system inputs the standardized defect dialogue cases and location reports into a large defect classification model. This model identifies defect types including "technical terminology errors" (confusing "historical annualized rate of return" with "expected annualized rate of return") and "information gaps" (failure to mention risk level). Based on these defect types and pre-defined financial industry standards (e.g., financial product introductions must clearly state risks), the system determines the corresponding optimization directions as "clarifying the definition of rate of return terms" and "supplementing risk warning information." Simultaneously, the system queries the historical defect case database for the frequency of defects in the "product information introduction prompt word module" and calculates the optimization priority score and level based on a pre-defined defect severity score (e.g., financial information errors are considered high severity), determining that this module has a "high" optimization priority. Finally, the system generates optimization direction information based on the optimization direction and priority. This step achieves automated defect classification and priority assessment, ensuring the targeted and timely nature of optimization work.

[0080] Subsequently, the system updates the original prompts in the prompt word module based on the optimization direction information and the positioning report, generating target prompts. The system calls upon the prompt word optimization rule base (e.g., rules including "clearly distinguishing between historical and expected returns" and "mandatory addition of risk warnings") and a pre-set financial field case library (containing compliant financial product introduction examples) to process the optimization direction information and the original prompts in the "product information introduction prompt word module," generating a preliminary rewriting scheme. For example, the rewriting scheme might suggest adding a clear explanation of the return rate and a risk warning template to the prompts. The system performs semantic reconstruction and verification on this preliminary rewriting scheme to ensure its grammatical correctness, semantic clarity, and compliance with financial compliance requirements, obtaining candidate prompts. The system pre-evaluates the candidate prompts; if they meet the preset basic quality requirements (e.g., passing grammar checks, fact-checking, and compliance reviews), they are used as target prompts. If not, the system adjusts the optimization strategy and regenerates candidate prompts until a target prompt that meets the preset basic quality requirements is output. This automated prompt word generation and iteration process overcomes the shortcomings of existing technologies that rely on manual prompt word modification and struggle to quickly respond to batch issues.

[0081] Next, the system verifies the effectiveness of the target prompts and obtains the verification results. Based on the functional scenario of a "financial product consultation intelligent agent," the system generates multiple sets of simulated dialogue data (e.g., simulating users inquiring about different types of yield rates, risk levels, etc.), and uses this simulated dialogue data to conduct simulated tests on the target prompts, obtaining simulation test results (e.g., response accuracy rate, risk warning coverage rate). Simultaneously, the system receives a preset proportion of real user traffic (e.g., 5% of users) and conducts traffic tests based on this real user traffic, collecting user feedback data during the traffic test process (e.g., user satisfaction rating, repeat question rate). The system compares the simulated test results and user feedback data with preset quality standards (e.g., response accuracy rate must reach above 98%, user satisfaction rate must be no less than 90%, and there must be no compliance errors) to obtain the verification results. This verification mechanism, combining simulated testing and real traffic testing, provides a comprehensive and reliable effect evaluation, ensuring the quality of the optimized prompts.

[0082] Finally, if the verification result is successful, the system will update the prompt word content of the corresponding prompt word module according to the configuration identifier. Specifically, if the verification result is successful, the system locates the corresponding prompt word record in the configuration database according to the configuration identifier, archives the original prompt word, and updates it to the current version with the target prompt word, incrementing the version number (e.g., from V1.0 to V1.1). At the same time, the system will associate and archive the defective dialogue cases, location results, optimization direction information, original prompt words, target prompt words, and verification results of this optimization to form a complete optimization history. In addition, the successful cases of this optimization are added to the preset financial field case library to enrich the knowledge base and update the recognition rules of each agent to adapt to the new prompt word content. This closed-loop update and knowledge accumulation mechanism realizes automated and continuous iteration from defect discovery to problem solving, effectively solving the problems of fragmented links and long iteration response cycles in existing technologies, thereby continuously ensuring the quality of the dialogue system.

[0083] Please refer to Figure 9 As a response to the above Figure 2 The implementation of the method shown in this application provides an embodiment of a prompt word closed-loop optimization device based on defect cases. This device embodiment is similar to... Figure 2 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0084] like Figure 9 As shown, the prompt word closed-loop optimization device based on defect cases in this embodiment includes: a case acquisition module 71, a problem location module 72, a defect analysis module 73, a prompt word update module 74, an effect verification module 75, and a prompt word content update module 76, wherein: The case acquisition module 71 is used to acquire defect dialogue cases and preprocess the defect dialogue cases to generate standardized defect dialogue cases. Problem localization module 72 is used to localize problems in the standardized defect dialogue case, to determine the target agent corresponding to the standardized defect dialogue case, the problem prompt word module and configuration identifier in the target agent, and to generate a localization report; Defect analysis module 73 is used to extract and analyze defects in the standardized defect dialogue cases and generate optimization direction information; The prompt word update module 74 is used to update the original prompt words in the problem prompt word module based on the optimization direction information and the positioning report, and generate target prompt words; Effect verification module 75 is used to verify the effect of the target prompt words and obtain the verification result; The prompt word content update module 76 is used to update the prompt word content of the corresponding prompt word module according to the configuration identifier if the verification result is successful.

[0085] Furthermore, the case acquisition module 71 includes: The dialogue record collection unit is used to collect dialogue records marked by users through the user feedback entry, and to identify dialogue records that exceed the threshold through preset quality monitoring indicators, thereby obtaining the defective dialogue cases. The preprocessing unit is used to preprocess the defective dialogue cases to obtain preprocessed defective dialogue cases. The case assembly unit is used to extract the user question content, dialogue flow path, interaction records of each agent prompt word module, final response content and user feedback tags from each preprocessed defective dialogue case, and assemble the extracted information into the standardized defective dialogue case.

[0086] Furthermore, the problem localization module 72 includes: The target agent determination unit is used to input the user's question content, dialogue flow path and final response content in the standardized defect dialogue case into the intent distribution and fault analysis model for analysis, so as to output the target agent to which the problem belongs; The problem prompt word module determination unit is used to identify the problem prompt word module directly related to the standardized defect dialogue case based on the functional division of labor and cooperation logic of each prompt word module within the target intelligent agent, and to obtain the unique configuration identifier of the problem prompt word module from the configuration database. The location report generation unit is used to generate the location report based on the target intelligent agent, the question prompt word module, and the unique configuration identifier.

[0087] Furthermore, the defect analysis module 73 includes: The defect type identification unit is used to input the standardized defect dialogue case and the location report into the defect classification model to identify the defect type, wherein the defect type includes answers that deviate from the requirements, incorrect professional terminology, unclear process guidance, missing information or logical contradictions. An optimization direction determination unit is used to determine the optimization direction corresponding to the defect type based on the defect type and a preset domain specification. The defect frequency query unit is used to query the defect occurrence frequency of the problem prompt word module from the historical defect case database, and calculate the optimization priority score and level according to the preset defect severity score; The optimization priority determination unit is used to determine the optimization priority of the problem prompt word module based on the optimization priority score and level, and to generate the optimization direction information based on the optimization direction and the optimization priority.

[0088] Furthermore, the prompt word update module 74 includes: The preliminary rewriting scheme generation unit is used to call the prompt word optimization rule library and the preset domain case library to process the optimization direction information and the original prompt words to generate a preliminary rewriting scheme; The candidate prompt word generation unit is used to perform semantic reconstruction and verification on the preliminary rewriting scheme to obtain candidate prompt words; The target prompt word determination unit is used to pre-evaluate the candidate prompt words. If the candidate prompt words meet the preset basic quality requirements, the candidate prompt words are used as the target prompt words. If the candidate prompt words do not meet the preset basic quality requirements, the optimization strategy is adjusted and the candidate prompt words are regenerated until a target prompt word that meets the preset basic quality requirements is output.

[0089] Furthermore, the effect verification module 75 includes: The simulation test unit is used to generate multiple sets of simulated dialogue data based on the functional scenarios of the target intelligent agent, and to conduct simulation tests on the target prompt words through the multiple sets of simulated dialogue data to obtain simulation test results. The user feedback data collection unit is used to receive a preset proportion of real user traffic, conduct traffic tests based on the real user traffic, and collect user feedback data during the traffic test process. The verification result generation unit is used to compare the simulated test results and the user feedback data with preset quality standards to obtain the verification results.

[0090] Furthermore, the prompt content update module 76 includes: The version update unit is used to locate the corresponding prompt word record in the configuration database according to the configuration identifier if the verification result is successful, archive the original prompt word, update the current version with the target prompt word, and increment the version number; The associated archiving unit is used to associate and archive the defective dialogue cases, the location results, the optimization direction information, the original prompt words, the target prompt words, and the verification results of this optimization. The case addition unit is used to add the successfully optimized cases to the preset domain case library and update the recognition rules of each agent.

[0091] It should be noted that any AI models, software tools, or components not belonging to this company appearing in the embodiments of this application are merely illustrative examples and do not represent actual use. All user personal information involved in the embodiments of this application has been obtained by authorized entities (who have known and consented) or fully authorized by all parties through various legal and compliant means. The collection, storage, use, processing, transmission, provision, and disclosure of the information, data, and signals involved all comply with relevant laws and regulations and do not violate public order and good morals.

[0092] To address the aforementioned technical problems, embodiments of this application also provide a computer device. Please refer to [link / reference needed]. Figure 10 , Figure 10 This is a basic structural block diagram of the computer device in this embodiment.

[0093] Computer device 8 includes a memory 81, a processor 82, and a network interface 83 that are interconnected via a system bus. It should be noted that... Figure 10 Only a computer device 8 with three components—memory 81, processor 82, and network interface 83—is shown. It should be understood that implementing all shown components is not required; more or fewer components may be implemented alternatively. Those skilled in the art will understand that this computer device is a device capable of automatically performing numerical calculations and / or information processing according to pre-set or stored instructions. Its hardware includes, but is not limited to, microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), and embedded devices.

[0094] Computer devices can include desktop computers, laptops, handheld computers, and cloud servers. These devices allow for human-computer interaction with users through keyboards, mice, remote controls, touchpads, or voice-activated devices.

[0095] The memory 81 includes at least one type of readable storage medium, including flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 81 may be an internal storage unit of the computer device 8, such as the hard disk or memory of the computer device 8. In other embodiments, the memory 81 may also be an external storage device of the computer device 8, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the computer device 8. Of course, the memory 81 may also include both internal storage units and external storage devices of the computer device 8. In this embodiment, the memory 81 is typically used to store the operating system and various application software installed on the computer device 8, such as the program code of the closed-loop optimization method for prompt words based on defect cases. In addition, the memory 81 can also be used to temporarily store various types of data that have been output or will be output.

[0096] In some embodiments, processor 82 may be a central processing unit (CPU), controller, microcontroller, microprocessor, or other data processing chip. This processor 82 is typically used to control the overall operation of the computer device 8. In this embodiment, processor 82 is used to run program code stored in memory 81 or process data, for example, to run the program code of the above-described defect case-based prompt word closed-loop optimization method to implement various embodiments of the defect case-based prompt word closed-loop optimization method.

[0097] The network interface 83 may include a wireless network interface or a wired network interface, which is typically used to establish a communication connection between the computer device 8 and other electronic devices.

[0098] This application also provides another implementation, namely, a computer-readable storage medium storing a computer program that can be executed by at least one processor to cause the at least one processor to perform the steps of the above-described defect case-based prompt word closed-loop optimization method.

[0099] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods of the various embodiments of this application.

[0100] Obviously, the embodiments described above are merely some embodiments of this application, not all embodiments. The accompanying drawings show preferred embodiments of this application, but do not limit the scope of this application. This application can be implemented in many different forms; rather, these embodiments are provided to provide a more thorough and comprehensive understanding of the disclosure of this application. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing specific embodiments, or make equivalent substitutions for some of the technical features. Any equivalent structures made using the content of this application's specification and drawings, directly or indirectly applied to other related technical fields, are similarly within the scope of protection of this application.

Claims

1. A closed-loop optimization method for prompt words based on defect cases, characterized in that, include: Obtain defective dialogue cases and preprocess them to generate standardized defective dialogue cases; The standardized defect dialogue cases are used to locate problems, thereby determining the target agent corresponding to the standardized defect dialogue cases, the problem prompt word module and configuration identifier within the target agent, and generating a location report; Defects are extracted and analyzed from the standardized defect dialogue cases to generate optimization direction information; Based on the optimized direction information and the positioning report, the original prompt words in the problem prompt word module are updated to generate target prompt words; The effectiveness of the target prompt words was verified, and the verification results were obtained. If the verification result is successful, the prompt word content of the corresponding prompt word module is updated according to the configuration identifier.

2. The closed-loop optimization method for prompt words based on defect cases as described in claim 1, characterized in that, The process of acquiring defective dialogue cases and preprocessing them to generate standardized defective dialogue cases includes: The defective dialogue cases are obtained by collecting user-marked dialogue records through the user feedback portal and identifying dialogue records that exceed the threshold through preset quality monitoring indicators. The defective dialogue cases are preprocessed to obtain preprocessed defective dialogue cases; Extract the user question content, dialogue flow path, interaction records of each agent prompt word module, final reply content, and user feedback tags from each preprocessed defective dialogue case, and assemble the extracted information into the standardized defective dialogue case.

3. The closed-loop optimization method for prompt words based on defect cases as described in claim 1, characterized in that, The process of locating problems in the standardized defect dialogue cases, determining the target agent corresponding to the standardized defect dialogue case, the problem prompt word module and configuration identifier within the target agent, and generating a location report includes: The user's question, dialogue flow path, and final response in the standardized defect dialogue case are input into the intent distribution and fault analysis model for analysis, so as to output the target intelligent agent to which the problem belongs; Based on the functional division and collaboration logic of each prompt word module within the target intelligent agent, the problem prompt word module directly related to the standardized defect dialogue case is identified, and the unique configuration identifier of the problem prompt word module is obtained from the configuration database. The location report is generated based on the target intelligent agent, the question prompt word module, and the unique configuration identifier.

4. The closed-loop optimization method for prompt words based on defect cases as described in claim 1, characterized in that, The step of extracting and analyzing defects from the standardized defect dialogue cases to generate optimization direction information includes: The standardized defect dialogue cases and the location reports are input into the defect classification model to identify the defect types, which include answers that deviate from the requirements, incorrect technical terms, unclear process guidance, missing information, or logical contradictions. Based on the defect type and the preset domain specifications, determine the optimization direction corresponding to the defect type; The frequency of defects in the problem prompt word module is queried from the historical defect case database, and the optimization priority score and level are calculated based on the preset defect severity score. The optimization priority of the problem prompt word module is determined based on the optimization priority score and level, and the optimization direction information is generated based on the optimization direction and the optimization priority.

5. The closed-loop optimization method for prompt words based on defect cases as described in claim 1, characterized in that, The step of updating the original prompt words in the problem prompt word module based on the optimization direction information and the positioning report to generate target prompt words includes: The system calls upon the prompt word optimization rule base and the preset domain case base to process the optimization direction information and the original prompt words, generating a preliminary rewriting scheme. The preliminary rewriting scheme is semantically reconstructed and verified to obtain candidate prompt words; The candidate prompt words are pre-evaluated. If the candidate prompt words meet the preset basic quality requirements, they are used as the target prompt words. If the candidate prompt words do not meet the preset basic quality requirements, the optimization strategy is adjusted and the candidate prompt words are regenerated until the target prompt words that meet the preset basic quality requirements are output.

6. The closed-loop optimization method for prompt words based on defect cases as described in claim 1, characterized in that, The process of validating the effect of the target prompt words and obtaining the validation results includes: Based on the functional scenarios of the target intelligent agent, multiple sets of simulated dialogue data are generated, and the target prompt words are simulated and tested using the multiple sets of simulated dialogue data to obtain simulation test results. Receive a preset proportion of real user traffic, conduct traffic tests based on the real user traffic, and collect user feedback data during the traffic test process; The simulation test results and user feedback data are compared with preset quality standards to obtain the verification results.

7. The prompt word closed-loop optimization method based on defect cases as described in any one of claims 1 to 6, characterized in that, If the verification result is successful, the prompt word content of the corresponding prompt word module is updated according to the configuration identifier, including: If the verification result is successful, the corresponding prompt word record in the configuration database is located according to the configuration identifier, the original prompt word is archived, and the current version is updated with the target prompt word, incrementing the version number. The defective dialogue cases, the localization results, the optimization direction information, the original prompt words, the target prompt words, and the verification results of this optimization will be associated and archived. Add the successful cases from this optimization to the preset domain case library and update the recognition rules for each agent.

8. A closed-loop optimization device for prompt words based on defect cases, characterized in that, include: The case acquisition module is used to acquire defect dialogue cases and preprocess the defect dialogue cases to generate standardized defect dialogue cases. The problem localization module is used to localize the problems in the standardized defect dialogue cases, so as to determine the target agent corresponding to the standardized defect dialogue cases, the problem prompt word module and configuration identifier in the target agent, and generate a localization report; The defect analysis module is used to extract and analyze defects from the standardized defect dialogue cases and generate optimization direction information. The prompt word update module is used to update the original prompt words in the problem prompt word module based on the optimization direction information and the positioning report, and generate target prompt words; The effect verification module is used to verify the effect of the target prompt words and obtain the verification results. The prompt word content update module is used to update the prompt word content of the corresponding prompt word module according to the configuration identifier if the verification result is successful.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the prompt word closed-loop optimization method based on defect cases as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the prompt word closed-loop optimization method based on defect cases as described in any one of claims 1 to 7.

Images