Knowledge question and answer method and device

Abstract

The embodiment of the invention provides a knowledge question and answer method and device. The method comprises the following steps: checking a target question and answer text in a question and answer knowledge base by using a knowledge base checking agent to obtain a checking result, and sending the checking result to a maintainer, so that the maintainer corrects the content of the question and answer knowledge base; analyzing a question and answer text in the question and answer system by using a question and answer case learning agent, and generating a question and answer case library according to an analysis result; wherein the question and answer case library comprises question and answer error cases and question and answer correct cases; when a question text input by a user is received, searching a target question and answer case from the question and answer case library, wherein the similarity between the target question and answer case and the question text is higher than a similarity threshold value; and optimizing the question-answering system according to the system optimization item parameters in the target question-answering case, and generating an answer corresponding to the question text according to the question-answering knowledge base and the target question-answering case after optimization.

Description

Technical Field

[0001] This application relates to the field of knowledge question answering technology, and in particular to a knowledge question answering method and apparatus. Background Technology

[0002] With the popularization of mobile internet and the rapid development of artificial intelligence technology, customers' demands on bank customer service are also increasing, expecting their questions to be answered 24 / 7, instantly, and accurately.

[0003] Current bank Q&A systems generally lack effective optimization and knowledge iteration capabilities. Most systems only record negative feedback from users (such as clicking "not helpful" or giving a rating), and then these negative feedback cases are manually analyzed periodically to try to find the root cause of the problem and fix it. This process is not only very manpower-intensive, but the accuracy of root cause identification and the effectiveness of the corrective measures also depend on individual abilities.

[0004] Supervised fine-tuning training can improve the question-answering capabilities of large-scale models in vertical domains. Some existing technologies utilize large-scale models to automatically extract supervised fine-tuning data from existing business documents and historical dialogue records. However, both the generation of training data and the supervised fine-tuning training of large-scale models place high demands on the hardware investment of banks (such as GPU server resources). Limited GPU server resources, frequent changes in business knowledge, and frequent updates to the large-scale model's base model all contribute to the high cost of incremental training to improve the business knowledge question-answering capabilities of large-scale models. Therefore, the solution of automatically generating incremental training data for large-scale models from historical system data and performing incremental training is impractical for most bank question-answering scenarios. Summary of the Invention

[0005] To address the aforementioned technical problems, this application provides a knowledge-based question-and-answer method and apparatus.

[0006] In a first aspect, embodiments of this application provide a knowledge question-answering method, the method comprising: The knowledge base inspection agent is used to inspect the target question and answer text in the question and answer knowledge base, obtain the inspection results, and send the inspection results to the maintenance personnel so that the maintenance personnel can correct the content of the question and answer knowledge base. The question-and-answer example learning agent analyzes the question-and-answer text in the question-and-answer system and generates a question-and-answer example library based on the analysis results; wherein, the question-and-answer example library includes: incorrect question-and-answer cases and correct question-and-answer cases; When a user inputs a question text, the system searches the question-and-answer example database for a target question-and-answer example whose similarity to the question text is higher than a similarity threshold. The question-and-answer system is optimized based on the system optimization parameters in the target question-and-answer example, and the answer corresponding to the question text is generated based on the question-and-answer knowledge base and the target question-and-answer example after optimization.

[0007] Secondly, embodiments of this application provide a knowledge question-answering device, the device comprising: The inspection result acquisition module is used to use a knowledge base inspection agent to inspect the target question and answer text in the question and answer knowledge base, obtain the inspection result, and send the inspection result to the maintenance personnel so that the maintenance personnel can correct the content of the question and answer knowledge base. The case library generation module is used to analyze the question and answer text in the question-and-answer system using a question-and-answer example learning agent, and generate a question-and-answer example library based on the analysis results; wherein, the question-and-answer example library includes: incorrect question-and-answer cases and correct question-and-answer cases; The target case search module is used to search for target question and answer examples with a similarity higher than a similarity threshold from the question and answer example library when receiving question text input by the user. The answer generation module is used to optimize the question-and-answer system based on the system optimization parameters in the target question-and-answer example, and after optimization, generate the answer corresponding to the question text based on the question-and-answer knowledge base and the target question-and-answer example.

[0008] Thirdly, embodiments of this application provide an electronic device, including: A processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements the knowledge question-answering method described above.

[0009] Fourthly, embodiments of this application provide a computer-readable storage medium that, when the instructions in the storage medium are executed by a processor of an electronic device, enables the electronic device to perform any of the knowledge question-answering methods described above.

[0010] Compared with the prior art, the embodiments of this application have the following advantages: In this embodiment, a knowledge base checking agent automatically checks the question-and-answer knowledge base, replacing traditional manual, piecemeal analysis and reducing manpower costs. Simultaneously, the question-and-answer example learning agent accumulates "correct and incorrect questions and answers," constructing a bidirectional case library. This avoids the bias of relying solely on negative feedback, and the agent executes according to preset rules, eliminating the influence of individual ability on root cause localization and remediation effectiveness, thus achieving automated and standardized knowledge iteration. The RAG algorithm, combined with the question-and-answer example library, generates answers without requiring incremental training or supervised fine-tuning of large models, completely eliminating dependence on high-configuration graphics card servers and reducing hardware investment costs. Furthermore, knowledge updates only require adjusting the knowledge base and updating the case library through the agent, without retraining the model. This allows for rapid adaptation to frequent changes in business knowledge and model foundations, solving the problem of high and impractical incremental training costs. By combining the reuse of positive and negative experiences from the case library with the precise retrieval of the RAG algorithm, the illusion of large models is effectively reduced, and the accuracy of answers is improved. At the same time, the closed-loop mechanism of intelligent inspection, case accumulation and answer generation allows the system to flexibly adapt to business changes, ultimately achieving the optimization goals of low cost and automation, which meets the compliance and practicality requirements of bank question-and-answer systems.

[0011] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0012] Figure 1 A flowchart illustrating the steps of a knowledge question-answering method provided in this application embodiment; Figure 2 A schematic diagram illustrating a system architecture and data flow process provided in an embodiment of this application; Figure 3 This is a schematic diagram of the structure of a knowledge question-answering device provided in an embodiment of this application; Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0013] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0014] The terminology used in the embodiments of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. The singular forms “a,” “the,” and “the” used in the embodiments of this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0015] Example 1 Reference Figure 1The flowchart illustrates the steps of a knowledge question-answering method provided in an embodiment of this application, such as... Figure 1 As shown, this knowledge-based question-and-answer method may include: Step 101: Use the knowledge base inspection agent to inspect the target question and answer text in the question and answer knowledge base, obtain the inspection results, and send the inspection results to the maintenance personnel so that the maintenance personnel can correct the content of the question and answer knowledge base.

[0016] In this embodiment, the knowledge base inspection agent refers to an intelligent module that has the ability to automatically check the quality of question-and-answer knowledge base content. It has built-in preset inspection rules and algorithms, and can perform automated verification on dimensions such as accuracy, consistency, completeness, and compliance of question-and-answer text, outputting clear problem location and adjustment suggestions to assist maintenance personnel in optimizing the knowledge base.

[0017] In this example, both the knowledge base checking agent and the question-and-answer example learning agent in the following steps are large agent / reasoning models with deep research capabilities. This deep research capability is the Goal-driven Autonomous Cognitive Workflow System, which is not essentially about "smarter answers," but rather about encapsulating multi-step research tasks into a repeatable, verifiable, and interruptible-recoverable procedural execution flow. Core capabilities consist of four essential indicators: 1. Task abstraction and schematic decomposition: Transforming vague requirements into executable operation sequences and establishing dependency graphs. For example, GLM-4.5's DeepResearch achieves a 90.6% tool invocation success rate, underpinned by the stability of task decomposition—not a single-prompt word engineering, but a built-in planning pattern. 2. Tool-Augmented Reasoning Chain: The model autonomously decides which tool to invoke at which stage of reasoning and seamlessly embeds the unstructured data returned by the tool into the reasoning context. This differs from the "search first, then summarize" model of general models. 3. Long-range memory and backtracking verification: Maintaining consistency of research objectives through an average of 20-50 rounds of interaction, supporting self-questioning and revision of intermediate conclusions. For example, DeepSeek-R1's "slow thinking" mode essentially involves explicitly maintaining a stack of hypotheses to be verified. 4. Output quality gatekeeping mechanism: Performing structured checks before final delivery: citation integrity, logical closure, and factual cross-validation. General models rely on post-processing prompts, while proxy models internalize this as part of the execution pipeline.

[0018] A question-and-answer knowledge base is a structured database that stores business knowledge in the banking field. It contains a large number of standardized question-and-answer pairs (question-answer), business document fragments, rule descriptions, and other content. It is the core knowledge source for generating answers in question-and-answer systems and supports operations such as retrieval, updating, and maintenance.

[0019] The target question-answer text refers to the object of inspection by the knowledge base inspection agent. It refers to the question-answer pairs, document fragments or rule entries to be verified in the question-answer knowledge base. It can be determined by full traversal or sampling selection, covering high-frequency business scenarios, newly added knowledge entries or historical question entries.

[0020] The inspection result refers to the output product of the knowledge base inspection agent, which includes the quality judgment conclusion of the target question and answer text (qualified / unqualified), the problem type (such as factual error, logical contradiction, missing information, compliance risk, etc.), the specific problem location (such as document number, segment number), and targeted adjustment suggestions (such as supplementing information, correcting the expression, deleting redundancy, etc.).

[0021] In practical implementation, a knowledge base inspection agent can be used to inspect target question-and-answer texts within the question-and-answer knowledge base, obtain inspection results, and send these results to maintenance personnel for adjustments to the question-and-answer knowledge base. Specifically, a general-purpose large model capable of private deployment by banks, possessing reasoning capabilities and the ability to process extremely long texts, can be used to construct an automatic knowledge base inspection agent. This agent requires no quality control labeling and automatically and periodically "inspects" the system's knowledge base (document fragment library, question-and-answer pair library) based on the frequency of questions asked in the system, using high-frequency questions and answers as anchors. Based on the system's high-frequency question-and-answer retrieval knowledge base, the large model performs the following checks on all relevant information returned by the retrieval: 1. Identification of contradictory statements; 2. Expired content identification; 3. Identification of ambiguous / vague expressions that may cause misunderstanding; 4. Knowledge gap identification in the knowledge base.

[0022] For the identified issues, the knowledge base maintainers are reminded to correct the problematic content, and suggestions for correcting the content generated by the large model are provided.

[0023] The detailed process of the inspection can be described in detail below in conjunction with the specific implementation methods.

[0024] In one specific implementation of this application, step 101 may include: Sub-step A1: Use the knowledge base to check the agent and determine whether it meets the checking conditions of the question-and-answer knowledge base.

[0025] In this embodiment, the inspection conditions refer to the preset rules that trigger the knowledge base inspection agent to perform the inspection, including but not limited to timed triggering (such as a full inspection every morning), high-frequency question and answer threshold triggering (such as a question and answer text interaction frequency reaching a threshold), and triggering after knowledge base update (such as adding more than a preset number of knowledge entries).

[0026] A knowledge base checking agent can be used to determine whether the checking conditions of the question-and-answer knowledge base have been met. Specifically, the knowledge base checking agent can monitor trigger signals in real time and make a judgment against preset checking conditions (such as timing period, frequency threshold, and knowledge base update volume). If any checking condition is met (such as the number of high-frequency question-and-answer texts reaching a certain threshold within a certain time period, or reaching a preset checking time point), it is determined that the "checking condition has been met," and the subsequent checking process is initiated; if the condition is not met, monitoring continues.

[0027] Sub-step A2: When it is determined that the check conditions of the question-and-answer knowledge base have been met, the target question-and-answer text is obtained from the historical question-and-answer text of the question-and-answer system through the question acquisition tool; the target question-and-answer text refers to the question-and-answer text in the historical question-and-answer text of the question-and-answer system whose question-and-answer frequency is greater than a first frequency threshold; wherein, the first frequency threshold is dynamically set according to the total amount of knowledge base, device capacity and task processing time.

[0028] The first frequency threshold refers to the quantitative standard used to filter target question and answer texts. It refers to the preset critical value of question and answer text interaction frequency (such as the average number of questions and answers per day ≥ 5 times). This first frequency threshold can be dynamically set according to the total amount of knowledge base, device capabilities, and task processing time.

[0029] The question acquisition tool refers to the retrieval and extraction module built into the knowledge base inspection agent. It supports rules such as frequency filtering and keyword matching to accurately extract target question and answer texts that meet the conditions from the question and answer knowledge base.

[0030] The methods for determining whether a question-and-answer knowledge base (QA) has met its inspection criteria can include: 1. Meeting the QA QA criteria when the interval between inspections reaches a set duration. 2. Meeting the QA QA criteria when the number of newly added error cases in the QA QA case library reaches a certain threshold. Newly added error cases refer to those added to the QA QA QA knowledge base since the last inspection. 3. Meeting the QA QA criteria when the change in a pre-stored list of specified questions exceeds a certain threshold. The specified question list contains questions with a frequency greater than a second frequency threshold.

[0031] When determining if the check criteria for the question-and-answer knowledge base have been met, a question retrieval tool can be used to obtain target question-and-answer texts from the historical question-and-answer texts of the question-and-answer system. Target question-and-answer texts refer to question-and-answer texts in the knowledge base whose frequency exceeds a first frequency threshold. Specifically, the knowledge base checking agent invokes the question retrieval tool to filter high-interaction-frequency question-and-answer texts from the historical question-and-answer texts of the system, based on the rule that the frequency exceeds the first frequency threshold. Simultaneously, the question retrieval tool statistically analyzes the interaction counts of historical question-and-answer texts (e.g., user inquiries, agent calls), compares this data to the first frequency threshold, and extracts all question-and-answer texts that meet the criteria, identifying them as target question-and-answer texts.

[0032] Sub-step A3: For each target question and answer text, use a knowledge base traversal tool to traverse the question and answer knowledge base to obtain target information fragments associated with the target question and answer text, and construct a temporary sub-knowledge base based on the target information fragments.

[0033] Knowledge base traversal tools refer to tool modules used for related information mining. Through algorithms such as keyword matching and semantic association analysis, they traverse the question-and-answer knowledge base to mine all knowledge fragments (such as documents and association rule descriptions related to the target question-and-answer text) related to the same business topic.

[0034] Target information fragments refer to knowledge base content fragments that have business or logical connections with the target question and answer text, including but not limited to fragments of relevant business rule documents, similar question and answer pairs, supplementary explanation items, etc.

[0035] A temporary sub-knowledge base refers to a temporary structured database built based on target information fragments. It contains only knowledge content related to the current target question and answer text and is used for focused and targeted detection to avoid redundant interference from full knowledge base detection.

[0036] For each target question-and-answer text, a knowledge base traversal tool is used to traverse the question-and-answer knowledge base, obtaining target information fragments associated with the target question-and-answer text, and constructing a temporary sub-knowledge base based on these target information fragments. Specifically, for each target question-and-answer text, the agent invokes the knowledge base traversal tool, using keyword matching (e.g., extracting core words such as "credit card" and "annual fee" from the target question-and-answer text) and semantic association algorithms to traverse the entire question-and-answer knowledge base. Then, it mines and extracts all target information fragments related to the target question-and-answer text (such as rule documents related to the same business topic, question-and-answer pairs in similar scenarios, and supplementary explanations). Finally, the extracted target information fragments are encapsulated in a unified data format (e.g., topic classification, sorting by association strength) to construct a temporary sub-knowledge base focused solely on the current target question-and-answer text, reducing interference from irrelevant information.

[0037] Sub-step A4: Perform multi-dimensional detection on the temporary sub-knowledge base to obtain multi-dimensional detection results.

[0038] Multi-dimensional detection refers to the quality verification dimensions of temporary sub-knowledge bases, including accuracy (consistency with authoritative business rules), consistency (unified expression of knowledge on the same topic), completeness (no missing key information), compliance (meeting regulatory requirements), and relevance (logical matching with the target question and answer text).

[0039] After constructing the temporary sub-knowledge base, multi-dimensional checks can be performed on the temporary sub-knowledge base to obtain multi-dimensional check results. The implementation process of multi-dimensional checks can be described in detail below with reference to the specific implementation method.

[0040] In another specific implementation of this application, sub-step A4 may include: Sub-step B1: Perform contradiction statement detection on the information in the temporary sub-knowledge base, identify mutually contradictory information, and associate and record the source information and complete content information of each contradictory information to obtain the contradiction statement detection result.

[0041] In this embodiment, contradictory statement detection refers to the logical consistency verification of information in the temporary sub-knowledge base, which is a detection process that uses algorithms to identify factual conflicts, rule contradictions, or contradictory statements between different information fragments.

[0042] After obtaining the temporary sub-knowledge base, all information fragments can be extracted from it and grouped and clustered according to business themes and core keywords (e.g., fragments related to "credit card annual fee waiver" are grouped into one category). Semantic similarity comparison algorithms and rule matching algorithms are used to compare the core facts and rule expressions of information fragments within the same group pairwise, identifying conflicting content (e.g., fragment A "Annual fee of 3000 or more waives the next year's fee" versus fragment B "Annual fee of 5000 or more waives the next year's fee," etc.). The source information (document number, fragment sequence number) and complete content of each pair of contradictory information are linked and recorded, forming a structured result for contradictory statement detection.

[0043] Sub-step B2: Detect expired content in the information containing time-sensitive identifiers in the temporary sub-knowledge base, compare the time-sensitive identifiers with the current date, filter and record expired information, and obtain the expired content detection results.

[0044] Timeliness identifiers refer to time-related marks attached to knowledge base information, including but not limited to effective date, expiration date, update date, and applicable period, used to clarify the time constraints of the information.

[0045] Expired content detection refers to the verification of information validity based on the time dimension. It is a process of filtering out information that has exceeded its usage period or has expired by comparing the timeliness identifier with the current date.

[0046] After obtaining the temporary sub-knowledge base, iteratively traverse the temporary sub-knowledge base, filter out information fragments with time-sensitive identifiers, extract key time parameters such as effective date and expiration date, and automatically compare the extracted time parameters with the current date to determine whether the information is expired (e.g., the expiration date is earlier than the current date, or the applicable period has expired). Filter and record the complete content, time-sensitive identifier details, and source information of all expired information, generating expired content detection results.

[0047] Sub-step B3: Perform text representation detection on the information in the temporary sub-knowledge base from a preset user perspective to obtain representation detection results.

[0048] The preset user perspective refers to simulating the cognitive level and information needs of the target user (such as ordinary bank customers or users without professional financial knowledge background) to determine whether the information is easy to understand and unambiguous.

[0049] Textual descriptions can include two types: vague descriptions and ambiguous descriptions. Vague descriptions refer to expressions with unclear meanings and ambiguous boundaries, such as vague terms like "recently" or "appropriate amount," which prevent users from accurately obtaining key information. Ambiguous descriptions refer to expressions with unclear meanings and ambiguous boundaries, such as vague terms like "recently" or "appropriate amount," which prevent users from accurately obtaining key information.

[0050] After obtaining the temporary sub-knowledge base, the system can switch to a preset user perspective (ordinary customer perspective) and invoke natural language processing tools to parse the information in the temporary sub-knowledge base sentence by sentence for text identification detection, including: 1. Identifying ambiguous expressions: using keyword matching (such as "recently," "approximately," "appropriate") and numerical missing value detection (such as no specific amount or deadline) to mark content with unclear meaning. 2. Identifying ambiguous expressions: using polysemous word detection and sentence structure analysis (such as subject-verb-object ambiguity) to determine whether the information has multiple possible interpretations, marking ambiguous points and potential directions of misunderstanding. The system then summarizes the location, content, and problem type of ambiguous and ambiguous expressions to form the expression detection results.

[0051] Sub-step B4: When the information in the temporary sub-knowledge base does not meet the response requirements of the target question and answer text, the missing key information of the temporary sub-knowledge base output by the preset model is obtained, and the missing key information is used as the knowledge gap detection result.

[0052] The pre-set model refers to a semantic understanding and demand matching model that has been pre-trained and configured, and has the ability to analyze the response requirements of the target question and answer text and evaluate the completeness of information in the temporary sub-knowledge base.

[0053] The response requirement refers to the user's core information needs corresponding to the target question and answer text, including but not limited to fact query, process consultation, rule confirmation, etc., which are extracted by the preset model based on the semantics of the question.

[0054] Missing key information refers to the core content that is not covered in the temporary sub-knowledge base but is necessary to meet the response requirements, such as key conditions, numerical standards, process nodes, etc.

[0055] After obtaining the temporary sub-knowledge base, the target question-and-answer text can be input into a preset model. The model will parse and extract the user's core answer requirements (such as "inquiry about the specific conditions and procedures for credit card annual fee waivers"). The preset model can traverse and evaluate the information in the temporary sub-knowledge base to determine whether the existing information can completely and accurately meet the answer requirements. If it is determined that it does not meet the requirements, the model will automatically output the missing key information that has not been covered (such as missing "application submission channels for fee waivers" and "review period"), and record the content, type, and impact on the completeness of the answer of the missing key information, generating a knowledge gap detection result.

[0056] Sub-step B5: The contradictory statement detection result, the expired content detection result, the statement detection result, and the knowledge gap detection result are used as the multi-dimensional detection result.

[0057] After obtaining the above detection results, the results of contradictory statements, expired content, statements, and knowledge gaps can be used as multi-dimensional detection results.

[0058] This application embodiment achieves comprehensive and accurate verification of the information quality of temporary sub-knowledge base through targeted detection in four dimensions: contradictory statements, expired content, textual descriptions, and knowledge gaps. It not only accurately identifies various knowledge defects and clarifies their sources, but also locates the core information needs that have not been covered, providing maintenance personnel with targeted optimization basis, greatly improving the comprehensiveness and accuracy of knowledge base inspection, reducing the difficulty of manual inspection, and ensuring the accuracy, timeliness, understandability, and completeness of question and answer information.

[0059] Sub-step A5: Summarize the multi-dimensional detection results and generate an inspection result that includes a problem summary, a knowledge problem list, and corresponding optimization suggestions. The knowledge problem list includes information related to the source of the problem.

[0060] A problem summary is a concise summary of the knowledge problems discovered during the detection process, including key information such as the core contradictions of the problem and its relevance to business scenarios.

[0061] A knowledge problem list is a structured list that summarizes all detected problems, clearly defining the type, severity, and scope of impact of each problem.

[0062] Problem source association information refers to the specific location information corresponding to the knowledge problem, including the knowledge base document number, information fragment sequence number, and associated question-and-answer pair ID, which supports maintenance personnel in quickly locating the source of the problem.

[0063] After obtaining the multi-dimensional detection results, these results can be aggregated to generate an inspection result containing a problem summary, a knowledge problem list, and corresponding optimization suggestions. The knowledge problem list includes information about the source of the problems. Specifically, the agent aggregates the multi-dimensional detection results of all target question-and-answer texts, extracts the core contradictions of each problem, and generates a concise problem summary. Problems are categorized by type (e.g., accuracy issues, completeness issues) to form a knowledge problem list with information about the source of the problems (document number, segment number, etc.). For each problem, based on business rules and knowledge characteristics, actionable optimization suggestions are generated (e.g., "Supplement the processing time description for XX business" or "Revise the interest rate value to the latest standard"). The problem summary, knowledge problem list, and optimization suggestions are integrated to generate a structured inspection result, which is then sent to maintenance personnel.

[0064] This application focuses on high-frequency question-and-answer texts and related knowledge fragments to construct temporary sub-knowledge bases and performs multi-dimensional precise detection, achieving targeted mining and precise positioning of knowledge base issues. This not only improves the efficiency and relevance of knowledge checks, but also provides maintenance personnel with a list of issues with source location and actionable optimization suggestions, significantly reducing manual investigation costs and ensuring the accuracy, consistency, and compliance of high-frequency business question-and-answer knowledge.

[0065] The following is a general description of the knowledge base inspection agent: In this embodiment, the triggering condition for the knowledge base inspection agent can be: automatic triggering at a preset period (such as weekly), or triggering when the system's high-frequency question list changes significantly.

[0066] The task is to identify contradictions, outdated information, ambiguous information, and knowledge gaps in the knowledge base, and generate inspection reports for maintenance personnel to refer to.

[0067] The tools used may include: 1. High-frequency question acquisition tool: Extracts the top N (N is a positive integer) questions with the highest frequency of questions asked within a certain period of time (such as the past week) from the question database.

[0068] 2. Knowledge base traversal tool: This tool aims to retrieve as many relevant information fragments as possible from the knowledge base for a given question, rather than just the Top K most similar results, and return all relevant information in batches.

[0069] The execution process may include: 1. Anchor point determination: The agent first calls the high-frequency problem acquisition tool to obtain a batch of the most concerned problems as the "anchor points" for inspection.

[0070] 2. Related Information Retrieval: For each high-frequency question, the agent invokes the knowledge base traversal tool to conduct a thorough search in the document fragment library and question-answer pair library, collecting all content related to the high-frequency question to form a temporary "sub-knowledge base".

[0071] 3. Contradictory Statement Detection: Examine the sub-knowledge base to identify and list all contradictory statements regarding high-frequency issues. For each contradiction, indicate the source document and specific text.

[0072] 4. Expired Content Detection: Check all information in the sub-knowledge base that contains time-sensitive information such as date and activity validity period, and compare it with the current date. List all content that may have expired.

[0073] 5. Ambiguous / Fuzzy Expression Detection: The large model is then subjected to the perspective of an ordinary customer unfamiliar with banking operations, reviewing the sub-knowledge base. Any terms that may cause misunderstanding, are poorly defined, or are overly technical are identified and listed, with explanations provided for their ambiguity.

[0074] 6. Knowledge Gap Analysis: Using a large model, determine whether the content in the sub-knowledge base is sufficient to completely and clearly answer the original high-frequency questions. If not, clearly identify the missing key information, which constitutes the knowledge gap in the knowledge base.

[0075] 7. Inspection Report Generation: The agent summarizes all detection results and generates a "Knowledge Base Inspection Report" for each high-frequency question. The report includes: Problem Summary: The current investigation revolves around high-frequency system issues.

[0076] Knowledge Issues List: This list details every knowledge contradiction, outdated, ambiguous issue, or knowledge gap discovered, along with links to the source documents and excerpts of the issue content.

[0077] Optimization suggestion: Leverage the generative capabilities of the large model to provide specific and actionable correction suggestions for each identified knowledge problem. For example: "Suggestion: Please confirm the upper limit for credit card limit increases and unify the wording in document A and question-and-answer pair B." Step 102: Use a question-answering example learning agent to analyze the question-answering text in the question-answering system, and generate a question-answering example library based on the analysis results; wherein, the question-answering example library includes: question-answering error cases and question-answering correct cases.

[0078] A question-and-answer example learning agent is an intelligent module with the ability to mine and accumulate case studies. It can analyze the interaction effects (correct / incorrect) of question-and-answer texts in the adjusted knowledge base, extract key features and optimization experience, and generate structured question-and-answer examples to support subsequent question-and-answer optimization.

[0079] A question and answer example database refers to a database that stores structured question and answer examples, including two categories: incorrect question and answer examples and correct question and answer examples. Incorrect question and answer examples record the text of incorrect questions and answers, the error type, root cause analysis, and optimization strategies. Correct question and answer examples record the text of correct questions and answers, key success factors, and reusable strategies (such as question rewriting, retrieval parameter configuration, etc.).

[0080] Simultaneously, a question-and-answer example learning agent can be used to analyze the question-and-answer text in the question-and-answer system and generate a question-and-answer example library based on the analysis results. This library can include both incorrect and correct question-and-answer examples. In a specific implementation, the question-and-answer example learning agent can acquire question-and-answer texts (including correct and incorrect interaction records) that have undergone interactive verification (such as user feedback or quality control annotation).

[0081] Case feature extraction: For correct question-and-answer text: analyze key success factors (such as question rewriting methods, search parameters, and context combination rules) and extract reusable strategies.

[0082] For erroneous question and answer text: Locate the error type (such as knowledge base defects, retrieval failure, generation bias) and analyze the root cause (such as knowledge gaps, unreasonable parameters).

[0083] Structured case generation: The agent encapsulates case information (question text, answer text, interaction results, key factors / root causes, optimization strategies) in a unified format and categorizes them into "correct question and answer cases" or "incorrect question and answer cases".

[0084] Case library update: The generated structured cases are stored in the question and answer case library and indexed (such as question keywords and business scenario tags) to support subsequent fast retrieval.

[0085] In this embodiment, the question-and-answer example learning agent may include: a wrong example learning agent and a right example learning agent. The learning process for wrong and right question-and-answer examples can be described in detail in conjunction with the following specific implementation methods.

[0086] In one specific implementation of this application, step 102 may include: Sub-step C1: Using the error case learning agent, perform error mining and analysis on the erroneous question and answer text, the error reason annotation information of the erroneous question and answer text, and the correct answer annotation information to obtain question and answer error cases.

[0087] In this embodiment, the error learning agent focuses on intelligent modules that extract root causes and optimization solutions from error question-and-answer texts. It has the ability to analyze error types, locate key problems, and extract improvement strategies to generate structured question-and-answer error cases.

[0088] Error question and answer text refers to question and answer interaction records (including the original question and the incorrect answer) in the question and answer text of the question and answer system that have been identified by quality inspection as not meeting the requirements of the correct answer, containing logical errors or information deviations.

[0089] Error reason annotation information refers to the explanation of the core root cause of the error in the question and answer text, which is marked by quality inspectors or intelligent agents. This includes, but is not limited to, knowledge base defects, retrieval failures, and generation logic deviations.

[0090] Correct answer annotation information refers to the authoritative correct answer to the incorrect question and answer text, or the standard response content confirmed by business operations, which serves as a benchmark for error analysis and verification of improvement strategies.

[0091] Error cases refer to structured cases generated by the error learning agent, which include error question and answer text, error reason annotation information, correct answer annotation information, and corresponding improvement points, so as to avoid similar errors in the future.

[0092] In a question-and-answer system, a fault-based learning agent can be used to analyze and mine faulty question-and-answer texts, error reason annotations, and correct answer annotations to identify faulty cases and areas for improvement.

[0093] In its implementation, the error-learning agent extracts all error-marked question-and-answer texts from the historical question-and-answer texts of the question-and-answer system, simultaneously acquiring corresponding error reason annotations and correct answer annotations. Deep error cause analysis: The agent combines the error question-and-answer texts, annotations, and correct answers to dissect the entire error-generating process (such as knowledge base content, retrieval process, and generation logic), identifying the error cause type (such as knowledge gaps, unreasonable parameters, or flawed prompt words). Improvement point mining: Based on the error cause analysis results, targeted improvement strategies are refined (such as supplementing missing knowledge base content, adjusting the top_k (k is a positive integer) retrieval value, and optimizing the constraints of generated prompt words). Structured case encapsulation: Error question-and-answer texts + error reasons + correct answers + improvement points are integrated in a unified format to generate question-and-answer error cases. This implementation process can be described in detail below with reference to the specific implementation methods.

[0094] In another specific implementation of this application, the fault-based learning agent includes: a master coordinating agent, a research agent, and a regression testing agent, and the aforementioned sub-step C1 may include: Sub-step D1: The main coordinating agent determines the error type corresponding to the error question and answer text based on the error reason annotation information, the associated interaction information of the error question and answer text, and the correct answer annotation information.

[0095] In this embodiment, the fault-based learning agent includes: a main coordinating agent, a research agent, and a regression testing agent.

[0096] The master coordinating agent is responsible for formulating the analysis plan, assigning tasks to the sub-agents, managing information flow, and dynamically adjusting the plan based on new findings until the problem is ultimately solved. The master coordinating agent is implemented using a privately deployable agent-based large model (such as Tongyi DeepResearch) combined with customized code and prompts. The specific responsibilities of the master coordinating agent are as follows: 1. Task initialization (large model native capabilities plus prompt words project): Receive initial information (user questions, incorrect answers, system search context, quality control labels, etc.).

[0097] Create an internal "state dashboard" to track the entire execution process. This state will record all known information, executed steps, and discoveries made by each sub-agent, etc.

[0098] 2. Dynamic Programming and Strategy Formulation (Combining native capabilities of large models with prompts and pre-defined programs): The pre-defined rule codes and prompts explicitly restrict large proxy models to only calling the aforementioned research agent tools, ignoring the native tools of other large proxy models. Furthermore, the pre-defined rule codes and prompts provide clear and concise descriptions of the research agent tools' functions and invocation instructions.

[0099] Develop a research strategy based on preset rule codes and prompts, combined with your own planning abilities.

[0100] 3. Agent scheduling and task allocation (engineering with native capabilities of large models plus prompt words): The corresponding sub-agents are invoked according to the established strategy.

[0101] Extract and package the input information required by the sub-agent.

[0102] Generate task instructions for the sub-agent, such as: "Task: Try to find a more relevant context for 'A marketing campaign achievement criteria' by rewriting the question." 4. Status Management and Information Summarization (Native Capabilities of Large Models plus Hint Word Project): When a sub-agent completes a task, it receives the returned result.

[0103] Update the status notepad with this new discovery (such as a valid rewrite problem, a more relevant context).

[0104] Transform the output of one agent into the input of another agent.

[0105] 5. Looping, conditional statements, and termination: The execution process is usually not a one-step process. The main coordinating agent needs to decide whether to continue the investigation or whether human experts need to be involved.

[0106] Loop: If adjusting the top_k parameter fails (not all relevant information is retrieved), the main coordinating agent may decide to let it try again, but this time the task may be "trying to rewrite the problem".

[0107] Judgment: The main coordinating AI will assess whether the currently collected evidence is sufficient to form a complete conclusion. For example, if it has been determined that adding top_k can find the missing key information and generate a large model answer consistent with the correct answer labeled by the quality inspector, the main coordinating AI will determine that there is no need to further analyze the generation process.

[0108] Human expert intervention: When all investigation paths have been tried but the root cause of the question-and-answer error still cannot be found, the main coordinating AI will pause the investigation loop and generate questions requiring assistance, waiting for a response from a human expert.

[0109] After receiving a response from human experts, the main coordinating AI will restart dynamic programming and strategy formulation (step two).

[0110] Termination: When the main coordinating agent determines that the root cause of the question-and-answer error has been found, and the system successfully generates an answer that matches the correct answer marked by the quality inspector after applying the corresponding optimization strategy, the investigation loop terminates.

[0111] 6. Result Generation: After the study is completed, the main coordinating AI will generate the structured final output of the error learning module based on all the key findings, evidence chains, failed attempts and successful breakthroughs on the "state notepad" and according to the prompts.

[0112] In this embodiment, the main coordinating agent first determines the error type of the erroneous question-and-answer text based on error cause annotation information, related interaction information of the erroneous question-and-answer text, and correct answer annotation information. This identifies the basic type of question-and-answer error and guides subsequent in-depth research. This step directly uses the error causes provided by quality inspectors instead of relying on a large model for self-determination, which significantly improves the efficiency and accuracy of in-depth research and reduces the requirements for the parameter levels of large models.

[0113] Sub-step D2: Using the research agent as a standard, the error type is analyzed in the error question and answer text to obtain the error cause analysis results.

[0114] After obtaining the error type, the agent can analyze the error type against the error question and answer text by studying the correct answer annotation information as the standard, and obtain the error cause analysis results.

[0115] In this example, the research agents may include: a knowledge base content review agent, a retrieval process simulation and analysis agent, and a generation process analysis agent. The specific error analysis process can be described in detail below with reference to the specific implementation methods.

[0116] In another specific implementation of this application, the above sub-step D2 may include: Sub-step E1: Invoke the knowledge base content review agent to review the content in the question-and-answer knowledge base when the answer corresponding to the question-and-answer text is incorrectly labeled, and obtain the content review result.

[0117] When performing error analysis, a knowledge base content review agent can be invoked. When the answer corresponding to the question-and-answer text is marked as incorrect, the agent reviews the content in the question-and-answer knowledge base to obtain the content review results. Specifically, when the answer corresponding to the question-and-answer text is marked as incorrect, the agent extracts the target knowledge point from the context information of the answer, invokes the knowledge base retrieval tool, searches the question-and-answer knowledge base using the target knowledge point as the search term, obtains the corresponding search results, compares the target knowledge point with the search results, identifies contradictory information, associates and records the source information of each contradictory piece of information, and generates an error analysis conclusion and corresponding contradictory explanation information based on the contradictory information and the source information. The error analysis conclusion, contradictory explanation information, and each contradictory piece of information are used as the content review results.

[0118] In practical implementation, the trigger condition for the knowledge base content review agent is: a preliminary diagnosis suspects that there is a problem with the context (the answer is based on the context but is labeled incorrectly). Common problems include inconsistent / contradictory content, unclear references, ambiguous expressions, or grammatical errors.

[0119] The task is to thoroughly examine the accuracy of the facts retrieved from the context.

[0120] Tools used include: knowledge base retrieval tools (searching for relevant or contradictory information in a knowledge base).

[0121] Execution process: 1. Extract key information from the context. For example, the context might say: "Points cannot be redeemed for airline miles."

[0122] 2. Use the knowledge base search tool to search for "Can't points be redeemed for airline miles?".

[0123] 3. Analyzing the search results, we found that document A in the knowledge base clearly states that "XZ level customers' points can be redeemed for airline miles".

[0124] 4. Error analysis conclusion and evidence generation: "The content of the 17th fragment of document B in the context reference knowledge base contradicts the content of document A, and there may be an error." Sub-step E2: Invoke the retrieval process simulation and analysis agent. When the context of the answer corresponding to the question and answer text is incomplete or irrelevant to the question and answer text, analyze the retrieval system corresponding to the erroneous question and answer text to obtain the analysis and inspection results.

[0125] The retrieval process simulation and analysis agent is invoked. When the context of the answer corresponding to the question-and-answer text is incomplete or irrelevant, the retrieval system corresponding to the erroneous question-and-answer text is analyzed to obtain analysis and inspection results. Specifically, the retrieval process simulation and analysis agent can be invoked. When the context of the answer corresponding to the question-and-answer text is incomplete or irrelevant, a question rewriting tool is invoked to rewrite the original question text corresponding to the question-and-answer text. A knowledge base retrieval tool is then used to simulate a retrieval of the rewritten question text to obtain simulated retrieval results. The knowledge base retrieval tool is used to retrieve the original question text using different retrieval parameters to obtain multiple retrieval results. Based on the simulated retrieval results and multiple retrieval results, the error cause analysis results and the corresponding descriptive information are determined. The error cause analysis results and descriptive information are used as analysis and inspection results. If the simulated retrieval results and multiple retrieval results do not meet the requirements of the original question text, a knowledge base traversal tool is used to retrieve the original question text. Based on the content returned by the retrieval, the missing information associated with the original question text in the question-and-answer knowledge base is determined to obtain analysis and inspection results.

[0126] In practical applications, the triggering conditions for retrieval process simulation and analysis of intelligent agents can be: preliminary diagnosis reveals that the context is irrelevant or incomplete.

[0127] The task is to investigate why the retrieval system did not return all relevant context.

[0128] The tools used may include: 1. Problem rewriting tool (large model hints): generating various rewrites of the original problem (such as concretization, abstraction, sub-problem decomposition, etc.). 2. Knowledge base retrieval tool: re-querying the knowledge base with different query parameters (such as top_k, weights of mixed search, similarity threshold). 3. Knowledge base traversal tool: this tool aims to retrieve as many potentially relevant information fragments as possible from the knowledge base for a given problem, rather than just the Top K most similar results, and return all relevant information in batches.

[0129] The execution process can be: 1. Problem rewriting experiment: The original question was rewritten using a question rewriting tool, and these rewritten queries were then used to simulate a search using a knowledge base retrieval tool. The quality of the search results from different rewritten versions was compared. If a particular rewritten version could retrieve more relevant information, it indicates that the original question formulation was the bottleneck preventing the search from succeeding.

[0130] 2. Experiment on adjusting search parameters: Keeping the original question unchanged, use the knowledge base retrieval tool and adjust the retrieval parameters. For example, adjust top_k: increase top_k from 10 to 20. Check if the newly recalled documents (ranked 11-20) contain the missing key information. If so, the reason is that the top_k value is too small.

[0131] 3. Error analysis conclusions and evidence generation: "The original question was poorly worded; rewriting it as '...' can significantly improve recall quality." or "This type of question belongs to the summary category, and the top_k should be appropriately increased."

[0132] 4. Knowledge gap identification: If the experiment ultimately fails to retrieve all the context required to answer the original question, a knowledge base traversal tool is invoked to analyze the returned content in batches to determine if it is sufficient to completely answer the original question. If not, it identifies the missing key information, which represents the knowledge gaps in the knowledge base.

[0133] Sub-step E3: Call the generation process analysis agent. When the error type is a large model error type or an answer synthesis error type, perform generation process analysis on the large model to obtain the generation process analysis results.

[0134] When the error type is either a large model error or an answer synthesis error, the generation process analysis agent can be invoked to perform generation process analysis on the large model, obtaining the generation process analysis results. Specifically, when the error type is either a large model error or an answer synthesis error, the generation process analysis agent can use a prompt word analysis tool to analyze the prompt words in the generation process of the original question text, obtaining the prompt word analysis results. Then, the large model analysis tool can be used to analyze the model reasoning process of the large model, obtaining the model capability analysis results. Finally, based on the prompt word analysis results and the model capability analysis results, the generation process analysis results are determined.

[0135] In this example, the triggering conditions for generating the link analysis agent are: a preliminary diagnosis of a large model illusion, or the context is correct but the answer synthesis is incorrect.

[0136] The task is to analyze why a large model generates incorrect answers when given the correct context.

[0137] The tools used may include: 1. Prompt word analysis tool: Check the prompt words in the generation process to see if there are any guiding or ambiguous instructions.

[0138] 2. Large Model Thinking Process Analysis Tool: (Only effective for question-and-answer responses from thinking large models) Analyzes the internal reasoning steps of the large model when generating answers.

[0139] The execution process can be as follows: 1. Instruction compliance analysis: Check the constraints in the prompt words for the large model's question and answer. If the large model's answer does not follow the constraints, it means that the expression of the constraints needs to be adjusted, or that the large model has difficulty following the constraints.

[0140] 2. Information Synthesis Ability Analysis: If the context contains related, repetitive, or contradictory information from multiple fragments, has the large model correctly correlated, ignored, or summarized it? If not, it indicates that the large model's information synthesis ability is insufficient, and additional cleaning of the search results is needed during the retrieval process, or more explicit instructions should be given in the generated prompts.

[0141] 3. Error Analysis Conclusions and Evidence Generation: "Insufficient constraint of prompts in the generation process leads the model to tend to hallucinate when contextual information is incomplete." or "The model failed to correctly synthesize information from fragments 2 and 3, resulting in a one-sided answer."

[0142] Sub-step E4: The content review result, the analysis and inspection result, and the generation process analysis result are determined as the error cause analysis result.

[0143] After obtaining the content review results, analysis and inspection results, and generation process analysis results, these results can be used as the error cause analysis results.

[0144] Sub-step D3: Invoke the regression testing agent, using the correct answer annotation information as the test evaluation benchmark, to test the optimization strategy in the error cause analysis results, and generate an evaluation report containing the strategy effect and risk assessment results.

[0145] After obtaining the error cause analysis results, a regression testing agent can be invoked. Using the correct answer annotation information as the test evaluation benchmark, the optimization strategies in the error cause analysis results are tested, generating an evaluation report that includes the strategy's effectiveness and risk assessment results. Specifically, the regression testing agent can obtain the optimization strategy to be tested, which is the strategy in the incorrect question answer example. Each question in the test set is simultaneously input into two parallel question-answering systems. In one question-answering system, the optimization strategy to be tested is applied, while the other question-answering system does not apply the optimization strategy to be tested. The output results of the two question-answering systems for each question are obtained and compared with the standard answers in the test set to obtain the strategy effectiveness of the optimization strategy to be tested. Based on the comparison results, a risk assessment is performed on the optimization strategy to be tested, obtaining the risk assessment results. Based on the strategy effectiveness and risk assessment results, an evaluation report is generated.

[0146] In this example, the regression test agent is triggered as follows: the agent is not started after each fix, but is automatically or manually triggered by the main coordinating agent when the system is about to undergo a change with potentially wide-ranging impact.

[0147] The core task is to conduct a comprehensive A / B comparison test on the system performance before and after the change in an isolated and controlled simulation environment, and produce a detailed quantitative evaluation report.

[0148] The tools used may include: 1. Regression test set generation tool: Automatically samples and generates a question and answer set containing both correct and incorrect results from the original system.

[0149] 2. Automated Evaluation Tool: Automatically compares answers in the question-and-answer set (correct answers provided by quality inspectors or answers in standard question-and-answer pairs) with the system's responses before and after the change. Comparison methods include semantic similarity and large model judgment.

[0150] The execution process can be as follows: 1. Initialization and Loading: The agent receives the optimization strategy to be tested (e.g., "increase the vector retrieval weight of all questions containing 'credit card' and 'annual fee' from 0.5 to 0.7").

[0151] At the same time, it calls the regression test set generation tool to obtain the regression test set.

[0152] 2. Parallel comparison execution: The agent simultaneously inputs each question from the test set into two parallel question-answering systems: System A (control group): The system logic running the current online version.

[0153] System B (Experimental Group): The system logic that applies the new optimized strategy runs in the "Strategy Execution Sandbox".

[0154] 3. Automatic result evaluation: For each question, the evaluation engine automatically collects the outputs of A and B and compares them with the "standard answer" in the regression test set.

[0155] The test result for each question is categorized into one of the following four states: Improvement: A's answer is incorrect, B's answer is correct.

[0156] Regression: A's answer is correct, B's answer is incorrect.

[0157] Keep it right: Both A and B answered correctly.

[0158] Keep it wrong: Both A and B answered incorrectly.

[0159] 4. Report generation: The agent summarizes the evaluation results of all problems and calculates key indicators such as error correction rate, regression rate, and overall accuracy change.

[0160] The final output will be a decision report containing the following information: Summary of key metrics: Changes in key metrics.

[0161] List of Remediation Cases: List the specific issues that were successfully resolved by the new strategy.

[0162] Risk checklist: List the specific problems and erroneous outputs that may have been "outsmarted" by the new strategy, for manual review.

[0163] Final recommendation: Based on preset rules, provide a conclusion of "recommend deployment" or "reject deployment".

[0164] Sub-step D4: Invoke the main coordinating agent, based on the correct answer annotation information, summarize the error cause analysis results and the evaluation report to obtain improvement suggestion information, and integrate the improvement suggestion information and the erroneous question and answer text to obtain the question and answer error case.

[0165] Finally, the main coordinating agent can be invoked to summarize the error analysis results and evaluation report based on the correct answer annotation information, obtain improvement suggestions, and integrate the improvement suggestions with the erroneous question-and-answer text to obtain question-and-answer error cases. In other words, after collecting the system-level optimization suggestions generated by the research agent, the main coordinating agent invokes the regression testing agent to test the new strategy using historical questions from a large question bank, generating an impact assessment report that clearly indicates which problems each new strategy can fix, but which existing correct answers it might negatively impact. Optimization strategies showing negative impacts in the report are only adopted in subsequent steps after manual review. This ensures the safety and robustness of system optimization, avoids the "whack-a-mole" problem, and makes each system evolution more robust and reliable.

[0166] Sub-step C2: Using the positive example learning agent, analyze the correctly labeled question and answer text in the question and answer text to obtain correct question and answer cases.

[0167] A positive example learning agent is an intelligent module that extracts successful experiences and reusable strategies from correct question-and-answer texts. It is used to discover the key factors for successful question-and-answer and generate structured correct question-and-answer cases.

[0168] Correct question-answering cases refer to structured cases generated by the positive example learning agent, which include correct question-answer text, key success factors (such as question rewriting methods, retrieval parameter configuration, etc.) and reusable optimization strategies for subsequent reuse of successful experiences.

[0169] When performing positive example learning, a positive example learning agent is used to analyze the correct question and answer texts labeled in the question and answer text of the question and answer system to obtain correct question and answer cases.

[0170] In its implementation, the positive example learning agent extracts all correctly labeled question-and-answer texts (including the original question and standard answer) from the historical question-and-answer texts of the question-and-answer system. Success factor extraction involves analyzing the processing chain of the correct question-and-answer texts to identify key success factors (such as question rewriting strategies, retrieval parameter configurations, and context combination methods). Reusable strategy refinement involves transforming the key success factors into standardized, reusable strategies. Finally, the correct question-and-answer texts, key success factors, and reusable strategies are integrated in a unified format to generate correct question-and-answer examples.

[0171] During positive example learning, a positive example learning agent is used to analyze the correctly labeled question-and-answer text in the question-and-answer system to obtain correct question-and-answer cases. Specifically, the implementation process may include: using the positive example learning agent to perform policy stripping tests and / or replacement comparison tests on the labeled correct question-and-answer text, obtaining test results; analyzing the test results to obtain successful attribution conclusion information corresponding to the correct question-and-answer text; generating optimization methods based on the successful attribution conclusion information; and generating correct question-and-answer cases based on the original question text of the correct question-and-answer text and the optimization method.

[0172] In its implementation, the positive example learning module (which can be a positive example learning agent) aims to automatically analyze and extract reusable success paradigms from question-and-answer interactions marked as correct by quality inspectors. It complements the error example learning module, together forming the "case learning agent." This module is also a system comprised of multiple specialized sub-agents working together.

[0173] Its input is: 1. The user's original question; 2. The system answer marked as correct; 3. The context retrieved during the system's question-and-answer interaction (document fragments and question-and-answer pairs); 4. The complete strategy chain used in the system's question-and-answer interaction (e.g., whether question rewriting was used, the specific rewriting results, search parameters, context combination methods, prompt word templates, etc.).

[0174] The output includes: 1. Success attribution type (e.g., appropriate question rewriting, effective retrieval strategy, etc.); 2. Success attribution analysis conclusions and evidence (e.g., 'Question rewritten A' retrieved more key fragments X than 'original question'); 3. Reusable success paradigm (including triggering conditions and optimization methods).

[0175] The trigger condition for the positive example learning agent is: receiving a "correct" case marked by the quality inspector.

[0176] The task is to accurately identify and analyze the core reasons for the success of this Q&A session, and generate a reusable success paradigm.

[0177] Tools that can be used include: Strategy stripping / replacement comparative experiment tools: In a sandbox environment, remove or replace a certain strategy component and observe whether the question-and-answer results worsen, thereby verifying the effectiveness of the strategy.

[0178] The execution process may include: 1. Hypothesis and Validation Loop: The agent sequentially performs "hypothesis-validation" on key links in the policy chain (such as question rewriting, retrieval parameters, and context combination).

[0179] Assumption Example 1: Problem rewriting is key. The agent invokes a policy stripping tool to skip the problem rewriting stage and directly use the original problem for subsequent processes. If the quality of the generated answer significantly decreases (e.g., incomplete information or off-topic), it preliminarily confirms that "problem rewriting" is one of the key factors for success.

[0180] Assume Example 2: Specific search weights are key. The agent invokes a policy replacement tool to replace the currently used mixed search weights (e.g., vector 0.5, BM25 0.5) with default weights (e.g., 0.6, 0.4). If the quality of the retrieved documents decreases, it confirms that the "specific search weights" were crucial for success.

[0181] 2. Attribution Conclusion Generation: Based on the results of all comparative experiments, the agent generates structured attribution conclusions for success. For example: "The success of this question answering is mainly attributed to refining the vague question 'Query the returns of XX wealth management product' into 'Query the latest net asset value of XX wealth management product' through a question rewriting strategy, thereby accurately hitting the target document fragment in the knowledge base." 3. Success Paradigm Extraction: Using a large model, generate "triggering conditions" based on the original question and "optimization methods" based on the success attribution conclusions, combine them into a complete success paradigm (i.e., a correct question and answer example), and store it in the "question and answer example library".

[0182] Sub-step C3: Generate the question and answer example library based on the question and answer error cases and the question and answer correct cases.

[0183] After obtaining the incorrect and correct question-and-answer examples, a question-and-answer example library can be generated based on these examples.

[0184] This application embodiment uses a fault-based learning agent to discover the root causes of errors and improvement points, and a positive-based learning agent to extract successful experiences and reusable strategies, constructing a question-and-answer example library containing both positive and negative cases. This not only provides a basis for error avoidance in subsequent question-and-answer sessions, but also allows for the reuse of successful experiences, helping the system to optimize quickly and improve the accuracy and iteration efficiency of question-and-answer sessions.

[0185] Step 103: Upon receiving the question text input by the user, search the question and answer example library for target question and answer examples that have a similarity higher than the similarity threshold with the question text.

[0186] Target question answer examples refer to cases retrieved from the question answer example library that have a similarity to the user's question text that exceeds a preset threshold. The optimization strategies contained therein can be directly adapted to the question answering process of the current question.

[0187] The similarity threshold refers to the preset standard used to determine the matching degree of cases (such as semantic similarity ≥ 85%), which is determined by the business scenario and model performance to ensure that the retrieved target cases have practical reference value.

[0188] Upon receiving a new question text input by the user, the system can search the question-and-answer example database for target question-and-answer examples with a similarity score higher than a similarity threshold. Specifically, a hybrid retrieval algorithm (such as BM25 sparse retrieval + vector dense retrieval) combined with a re-ranking algorithm can be used to retrieve cases semantically similar to the user's question text from the question-and-answer example database. Then, a matching determination is performed: the similarity score between the retrieved cases and the user's question is calculated, and cases with a similarity score higher than a preset threshold are selected as target question-and-answer examples. If no matching cases are found, the system proceeds to the regular RAG process.

[0189] The implementation process can be described in detail below with reference to the specific implementation methods.

[0190] In one specific implementation of this application, step 103 may include: Sub-step F1: Upon receiving the question text, use a hybrid retrieval algorithm and a reordering algorithm to retrieve and reorder the question text to obtain the standard answer corresponding to the question text.

[0191] In this embodiment, upon receiving user-inputted question text, a hybrid retrieval algorithm and a reordering algorithm can be used to retrieve and reorder the question text to obtain the corresponding standard answer. Specifically, customer questions and agent questions from the question-and-answer phase, as well as standard questions from the knowledge maintenance phase, are directed to a unified question database. The question database is responsible for recording various types of questions and forwarding them to the question-and-answer system. Upon receiving a new question, the question-and-answer system uses a hybrid retrieval algorithm (such as sparse retrieval using BM25 combined with dense retrieval using vector retrieval) combined with a reordering algorithm (Cross Encoder model) to retrieve standard questions and answers similar to the new question within the FAQ (Frequently Asked Questions) system.

[0192] Sub-step F2: When the standard answer does not meet the answer requirements of the question text, search the question answer example library for a target question answer example with a similarity higher than the similarity threshold with the question text.

[0193] When the standard answer does not meet the requirements of the question text, a target question-and-answer example with a similarity higher than a similarity threshold can be searched from the question-and-answer example database. Specifically, if a standard question-and-answer pair that meets the preset condition of high similarity exists (e.g., exceeding the preset similarity threshold, or with very small text editing distance), the standard answer is returned directly to the user. If a standard question-and-answer pair that meets the preset condition of relatively similarity exists, the top N standard question-and-answer pairs that meet the condition are returned for the user to choose from. If none of these exist, or if the user reports an incorrect answer in real time, the question is transferred to the RAG question-and-answer system.

[0194] Step 104: Optimize the question-answering system according to the system optimization parameters in the target question-answer example, and after optimization, generate the answer corresponding to the question text based on the question-answering knowledge base and the target question-answer example.

[0195] Retrieval-enhanced generation algorithms are hybrid algorithms that combine knowledge base retrieval with large model generation. They first retrieve knowledge related to the question from the knowledge base, and then use the retrieval results as context input to the large model to generate answers that are factual and logically coherent, thus avoiding the illusion of a large model.

[0196] After obtaining the target question and answer example, the question and answer system can be optimized based on the system optimization parameters in the target question and answer example. After optimization, the answer corresponding to the question text can be generated based on the question and answer knowledge base and the target question and answer example.

[0197] The implementation process can be described in detail below with reference to the specific implementation methods.

[0198] In one specific implementation of this application, step 104 may include: Sub-step G1: When the target question answer example is a question answering error case, optimize the system modules in the question answering system that match the system optimization item parameters according to the system optimization item parameters in the target question answer example; wherein, the system modules include at least one of: retrieval engine, large model and knowledge base.

[0199] In this embodiment, when the target question-and-answer example is a question-and-answer error case, the system modules in the question-and-answer system that match the system optimization parameters in the target question-and-answer example are optimized. The system modules may include at least one of the following: a search engine, a large model, and a knowledge base.

[0200] Sub-step G2: When the question text is a real-time question, the user's intent is identified based on the user profile and the question text. An initial answer is generated based on the user intent, the search results of the question-and-answer knowledge base, and the reference information in the target question and answer example. The initial answer is then processed in conjunction with the user profile to generate the target answer for the question text.

[0201] When the question text is a real-time question, the user's intent is identified based on the user profile and the question text. An initial answer is generated based on the user intent, the search results of the question-and-answer knowledge base, and the reference information in the target question and answer examples. The initial answer is then processed in conjunction with the user profile to generate the target answer for the question text.

[0202] During the specific retrieval process, the Retrieval-Augmented Generation (RAG) algorithm is used. In the RAG question-answering system, similar cases to the question-answer example database for a new question are first retrieved. If cases meeting preset high similarity criteria exist, optimization strategies corresponding to those cases are applied in the normal RAG system workflow. These strategies include: Question rewriting: rewriting the original question by making it more specific or abstract, or decomposing it into sub-questions; Retrieval parameter adjustment: adjusting, for example, the weights of top k, sparse, and dense searches in mixed searches; Context combination: adjusting the way relevant document fragments and question-answer pairs are combined into a reference context (e.g., format, order); Prompt word adjustment: adjusting the conditional constraints on the question and answer in the prompt words; and Adding error examples to the prompt word context: adding error warning modules to the context of the prompt words, placing incorrect questions and answers, and providing root cause analysis. If no similar cases exist, the normal RAG question-answering workflow continues.

[0203] Sub-step G3: When the question text is a non-real-time question, an initial answer is generated based on the question text, the search results of the question-and-answer knowledge base, and the reference information in the target question and answer examples. The large model generates an answer tracing report based on the process information of answer generation. The initial answer and the answer tracing report are integrated into the final answer. The answer tracing report is used for post-quality inspection.

[0204] When the question text is a non-real-time question, an initial answer is generated based on the question text, the search results from the question-and-answer knowledge base, and optimization strategies using the target question-and-answer examples. The large model then generates an answer tracing report based on the information from the answer generation process. The initial answer and the answer tracing report are integrated into the final answer, which is used for later quality control. Specifically, for non-real-time request questions (new standard questions), since there is no requirement for response latency, the question-and-answer system generates an additional "answer tracing report" for each answer based on information generated during the answer generation process (e.g., user question, retrieved document fragments or question-and-answer pairs, model inference process, model answer). This report helps quality control personnel efficiently determine the correctness of the system's answers. The report includes: Direct evidence: clearly listing which specific fragments from which documents in the knowledge base the answer references. Inference path: concisely demonstrating how the large model derives the final answer based on information from retrieved document fragments or question-and-answer pairs and the user question. Confidence rating: providing a confidence rating and explaining the basis for the rating.

[0205] In this embodiment, cluster analysis can also be performed on the question-and-answer error cases in the question-and-answer example library. This implementation process can be described in detail below with reference to the specific implementation method.

[0206] In one implementation of this application, the method further includes: Step S1: When the clustering analysis conditions of the question-and-answer example library are met, the question-and-answer example learning agent is used to cluster the question-and-answer error cases in the question-and-answer example library to obtain the clustered error cases of the question-and-answer error cases.

[0207] In this embodiment, determining the clustering analysis conditions for the question and answer example library can be done by: pre-setting the clustering analysis period, and performing clustering analysis on the question and answer example library when the conditions for the clustering analysis period are met.

[0208] Once the clustering analysis conditions for the question-and-answer example library are determined, a question-and-answer example learning agent can be used to cluster the question-and-answer error cases within the library, resulting in clustered error cases. In other words, similar question-and-answer error cases in the question-and-answer example library are grouped into one category.

[0209] Step S2: Perform error commonality analysis on the clustering error cases to obtain the error commonality features of the clustering error cases.

[0210] Then, common error analysis can be performed on clustering error cases to obtain the common error characteristics of clustering error cases.

[0211] Step S3: Based on the common characteristics of the errors, generate a general solution corresponding to the clustering error cases.

[0212] Finally, a general solution for clustering error cases can be generated based on the common characteristics of the errors.

[0213] In the above scheme, the question-and-answer example learning agent not only learns from individual error examples marked by quality inspectors, but also periodically performs cluster analysis and mining on all error examples in the "question-and-answer example library." The goal is not only to easily fix "one" problem, but also to learn general solutions for "a class" of problems. By analyzing similar errors, hidden common patterns are discovered.

[0214] The knowledge-based question-answering method provided in this application automatically checks the question-answering knowledge base through a knowledge base checking agent, replacing traditional manual, piecemeal analysis and reducing manpower costs. Simultaneously, the question-answering example learning agent accumulates "correct and incorrect cases" to build a bidirectional case library, avoiding the bias of relying solely on negative feedback. Furthermore, the agent executes according to preset rules, eliminating the influence of individual ability on root cause localization and repair effectiveness, thus achieving automated and standardized knowledge iteration. The method uses the RAG algorithm combined with the question-answering example library to generate answers, eliminating the need for incremental training or supervised fine-tuning of large models, completely eliminating dependence on high-configuration graphics card servers, and reducing hardware investment costs. Moreover, knowledge updates only require adjusting the knowledge base and updating the case library through the agent, without retraining the model, allowing for rapid adaptation to frequent changes in business knowledge and model foundation, solving the problem of high and impractical incremental training costs. By combining the reuse of positive and negative experiences from the case library with the precise retrieval of the RAG algorithm, the illusion of large models is effectively reduced, and the accuracy of answers is improved. At the same time, the closed-loop mechanism of intelligent inspection, case accumulation and answer generation allows the system to flexibly adapt to business changes, ultimately achieving the optimization goals of low cost and automation, which meets the compliance and practicality requirements of bank question-and-answer systems.

[0215] Example 2 Reference Figure 2 This diagram illustrates a system architecture and data flow process provided in an embodiment of this application. Figure 2 As shown, the data flow process can follow the logic of: input collection → knowledge matching → output and quality inspection → content maintenance → case accumulation → closed-loop optimization. The implementation process can include: Step 1, Question Input and Collection: Customer questions generated from new customer conversations, agent questions entered by agents, and standard questions added by Q&A maintenance personnel will be uniformly collected into the question database; at the same time, the content of new customer conversations will be synchronously stored in the historical conversation database to retain interaction records.

[0216] Step 2: Resource matching and output of the knowledge question answering system: The question database transmits questions to the knowledge question answering system, which then coordinates with the three types of resource databases to complete the matching and output. 1. Match "question-related standard question-answer pairs" from the standard question-answer pair database; 2. Match "problem-related document fragments" from the document fragment library; 3. Combining the examples in the question and answer example library (called through "System Optimization Strategy"), generate two types of results: "Business Problem System Q&A" for business scenarios and "Customer Problem Agent Answers" for customer inquiries.

[0217] Step 3, Quality Inspection and Annotation: Quality inspectors annotate the output of the knowledge Q&A system: 1. For "question, search results and answer content", complete "question, search and answer (annotation)"; 2. For "question, search results and document fragments", complete "question, search and document fragment (annotation)"; Both types of annotation results are synchronized to the "system content and quality inspection follow-up" stage.

[0218] Step 4, Knowledge Content Maintenance and Correction: Based on the quality inspection results and daily maintenance needs, carry out precise corrections: Q&A maintenance personnel perform "Q&A Pair Content Correction" for the standard Q&A pair library; additional maintenance personnel perform "Document Content Correction" for the document fragment library to ensure the accuracy of knowledge resources.

[0219] Step 5: Question and Answer Example Library Construction and Intelligent Optimization: The results of "System Content and Quality Inspection Follow-up" are passed to the question and answer learning agent. The agent combines the "system optimization strategy" to mine cases from "system question and answer related standard question and answer pairs" and "system question and answer related document fragments" to generate a question and answer example library. At the same time, the knowledge system knowledge configuration agent will identify erroneous knowledge interfaces and issue reminders to promote the precise optimization of knowledge resources.

[0220] Step 6: High-Frequency Question and Answer Accumulation and Closed-Loop Iteration: The frequently generated "high-frequency system questions and answers" will be accumulated back into the standard question and answer database to continuously enrich standard knowledge resources; at the same time, the question and answer example database will provide feedback on optimization strategies to the "knowledge question and answer system", ultimately forming a business closed loop of "question input → knowledge matching → output quality inspection → content maintenance → case accumulation → iterative optimization".

[0221] In this embodiment, a "quality control and annotation integrated tool" is provided for question-and-answer quality inspectors, combining routine question-and-answer quality control with model training data annotation. In addition to providing quality inspectors with functions such as annotating the correctness of agent and system answers, explaining the reasons for errors, and annotating correct answers, the tool also supports quality inspectors in labeling the document fragments and question-and-answer pairs retrieved by the question-and-answer system during the process as relevant / weakly relevant / irrelevant. These annotations, as high-quality training data, can be used for the continuous optimization of the retrieval models (such as vector models) embedded in the question-and-answer system.

[0222] Question-and-answer learning examples can include: I. Learning from Mistakes

[0223] By utilizing a general-purpose large-scale model with reasoning capabilities that can be privately deployed by banks and an agent-type large-scale model with Deep Research capabilities (including task decomposition and planning, tool invocation, and conclusion integration), a "question-answer example learning agent" is constructed to analyze and mine experiential knowledge from question-answer examples.

[0224] The annotations of quality inspectors on the reasons for errors in questions and answers, along with the marking of correct answers, serve as task planning clues and goal achievement signals for the intelligent agent. The agent then autonomously analyzes and identifies the causes of errors and areas for improvement. Furthermore, when the agent encounters difficulties during in-depth research, it automatically seeks human intervention from business experts for guidance, ensuring the quality of the research.

[0225] The results of the mining include: 1. Categorize the causes of errors in Q&A.

[0226] 2. Root cause analysis of incorrect questions and answers.

[0227] 3. Strategies for rewriting incorrect examples.

[0228] 4. Optimization strategies for retrieving incorrect examples.

[0229] 5. Optimization strategy for synthesizing faulty instances based on their reference context.

[0230] 6. Strategies for adjusting prompts for incorrect examples.

[0231] 7. Document snippets or Q&A sections provide suggestions for identifying and correcting errors.

[0232] The above results, along with incorrect answers, correct answers, and quality control comments, constitute a set of question and answer error examples, which are recorded in the "Question and Answer Example Library".

[0233] For the system optimization strategies generated by the agent, the agent automatically tests the new strategies using historical questions from a large question bank, generating an impact assessment report. The report clearly indicates which problems the new strategy can fix, but also which existing correct answers might be negatively affected. Optimization strategies showing negative impacts in the report must be reviewed by system maintenance personnel before being added to the "question-answer example library." This ensures the security and robustness of system optimization, avoiding the "one problem solved, another arises" issue, and making system evolution more stable and reliable.

[0234] If the agent discovers errors or knowledge gaps in document fragments or question-and-answer pairs in the knowledge base during error analysis, it prompts the knowledge base maintainers to correct the errors and provides suggestions for correcting the errors generated by the large model.

[0235] This intelligent agent not only learns from individual error examples marked by quality inspectors, but also periodically performs cluster analysis and mining on all error examples in the "question and answer example library." The goal is not only to easily fix "one" problem, but also to learn general solutions for "a class" of problems. By analyzing similar errors, it discovers hidden common patterns and extracts them into generalizable system optimization strategies.

[0236] II. Learning from positive examples.

[0237] For questions and answers correctly marked by quality inspectors, the AI ​​agent analyzes the reasons for their success. Successful strategies such as "question rewriting," "context combination strategies," or "prompt word templates" are extracted to form reusable success paradigms, which are then recorded as positive examples in the "question and answer example library."

[0238] III. Automatic knowledge base check.

[0239] An intelligent agent for automatic knowledge base inspection is constructed using a general-purpose large-scale model that can be privately deployed by the bank and possesses reasoning capabilities and the ability to process extremely long texts. This agent requires no quality control labeling and automatically and periodically "inspects" the system's knowledge base (document fragment library and question-answer pair library) based on the frequency of questions asked in the system, using high-frequency questions and answers as anchors. Based on the system's high-frequency question-answer retrieval knowledge base, the large-scale model performs the following checks on all relevant information returned by the retrieval: Identification of contradictory statements; Expired content identification; Identification of ambiguous / vague expressions that may lead to misunderstanding; Knowledge gap identification in the knowledge base.

[0240] For the identified issues, the knowledge base maintainers are reminded to correct the problematic content, and suggestions for correcting the content generated by the large model are provided.

[0241] IV. Adaptive and proactive optimization.

[0242] Whether it's a customer or agent question during the Q&A phase, or a newly created standard question during the knowledge maintenance phase, the Q&A system generates answers and records all types of questions in the "question database." Upon receiving a new question, in addition to following the system's standard RAG (Related Information Grouping) process, the Q&A system attempts to search for similar cases in the question example database and applies optimization strategies corresponding to these similar cases in the RAG's question processing, retrieval, contextual synthesis, and prompt word optimization stages.

[0243] For non-real-time request types of questions (such as newly created standard questions during question-answer pair maintenance), when generating answers, the question-answering system generates an additional "answer traceability report" based on the process information generated by the system when generating answers (such as questions, retrieved document fragments or question-answer pairs, and model reasoning processes). This helps quality inspectors efficiently determine whether the system's answers are correct or not.

[0244] For real-time request-based questions (such as customer inquiries or agent-relayed questions), the question-answering system performs additional customer profile enhancement when generating answers. Specifically, it combines customer profile tags with other anonymized information (such as: car enthusiasts, new customers within 90 days) with the current question to form a richer context, helping the system better understand customer intent. This information is also provided to the large model when generating answers, allowing the large model to perform appropriate stylistic polishing of the answers, making the generated answers better aligned with the customer's reading habits.

[0245] This application's embodiments integrate the capabilities of large-scale intelligent agents and professionals throughout the entire process of question-answering system optimization by constructing a human-machine collaborative case learning loop. By establishing a closed-loop learning mechanism of "question-answering-quality inspection-analysis-repair-question-answering," quality inspectors can efficiently assist the question-answering system in learning from labeled cases while performing question-answering quality inspection work, achieving continuous optimization of question-answering capabilities. This invention has the following three advantages: Case Experience Accumulation: During the initial quality control phase, quality inspectors annotated the correctness and errors of the question-and-answer system's results and processes. Using these high-quality annotations as clues, the large-scale intelligent agent automatically conducts in-depth research, accurately tracing back and diagnosing the root causes of errors or the reasons for the success of correct examples. For knowledge errors, the large-scale intelligent agent prompts knowledge maintainers and provides correction suggestions; for retrieval-related issues, it generates optimization strategies such as "question rewriting strategies" or "retrieval strategy adjustments," while automatically conducting question-and-answer regression testing to verify the effects and impact; for effective strategies in correct cases, the large-scale intelligent agent compiles reusable successful paradigms. This allows the question-and-answer system to continuously learn and accumulate experience from the cases annotated during quality control.

[0246] Improved accuracy: The question-and-answer example database records the error analysis results and optimization strategies of the system and agents' incorrect question-and-answer responses, as well as successful paradigms for correct answers. When handling similar questions in the future, the question-and-answer system will prioritize querying the "question-and-answer example database." This is equivalent to establishing a continuously updated, real-time callable "system optimization guide," proactively avoiding known "traps" or performing fine-tuning before the system generates answers. This will continuously improve the accuracy of the question-and-answer system's responses and alleviate the large model illusion problem caused by RAG question answering.

[0247] Knowledge quality improvement: Based on the high-frequency questions and answers recorded by the question-and-answer system, the knowledge base is automatically analyzed to identify contradictory knowledge, invalid knowledge, ambiguous knowledge, and knowledge blind spots. The system prompts knowledge base maintenance personnel to make corrections and generates correction suggestions to continuously improve the knowledge quality of the question-and-answer system's knowledge base.

[0248] Example 3 Reference Figure 3 The diagram shows a structural schematic of a knowledge question-answering device provided in an embodiment of this application, such as... Figure 3 As shown, the knowledge question-answering device 300 may include the following modules: The inspection result acquisition module 310 is used to use a knowledge base inspection agent to inspect the target question and answer text in the question and answer knowledge base, obtain the inspection result, and send the inspection result to the maintenance personnel so that the maintenance personnel can correct the content of the question and answer knowledge base. The case library generation module 320 is used to analyze the question and answer text in the question-and-answer system using a question-and-answer example learning agent, and generate a question-and-answer example library based on the analysis results; wherein, the question-and-answer example library includes: incorrect question-and-answer cases and correct question-and-answer cases; The target case search module 330 is used to search for target question and answer examples with a similarity higher than a similarity threshold from the question and answer example library when receiving question text input by the user; The answer generation module 340 is used to optimize the question-and-answer system according to the system optimization parameters in the target question-and-answer example, and after optimization, generate the answer corresponding to the question text according to the question-and-answer knowledge base and the target question-and-answer example.

[0249] Optionally, the inspection result acquisition module includes: The inspection condition determination unit is used to inspect the intelligent agent using the knowledge base to determine whether the inspection conditions of the question-and-answer knowledge base are met. The target text extraction unit is used to extract target question-and-answer text from the historical question-and-answer text of the question-and-answer system using a question acquisition tool when the check conditions of the question-and-answer knowledge base are determined to be met. The target question-and-answer text refers to the question-and-answer text in the historical question-and-answer text of the question-and-answer system whose question-and-answer frequency is greater than a first frequency threshold. The first frequency threshold is dynamically set based on the total amount of knowledge base, device capacity, and task processing time. The sub-knowledge base acquisition unit is used to traverse the question-and-answer knowledge base using a knowledge base traversal tool for each target question-and-answer text, obtain target information fragments associated with the target question-and-answer text, and construct a temporary sub-knowledge base based on the target information fragments. The detection result acquisition unit is used to perform multi-dimensional detection on the temporary sub-knowledge base and obtain multi-dimensional detection results. The inspection result generation unit is used to summarize the multi-dimensional detection results and generate inspection results that include a problem summary, a knowledge problem list and corresponding optimization suggestions. The knowledge problem list includes problem source association information.

[0250] Optionally, the inspection condition determination unit includes: The first condition determination subunit is used to determine the inspection condition of the question-and-answer knowledge base when the interval between checking the question-and-answer knowledge base reaches a set time. The second condition determination subunit is used to determine the inspection condition of the question-and-answer knowledge base when the number of newly added error cases in the question-and-answer knowledge base reaches a certain threshold; the newly added error cases refer to the error cases added to the question-and-answer knowledge base since the last inspection. The third condition determination subunit is used to determine the check condition of the question-and-answer knowledge base when the change range of the specified question list pre-stored in the system is greater than the range threshold; the specified question list records questions with a question-and-answer frequency greater than the second frequency threshold.

[0251] Optionally, the detection result acquisition unit includes: The first result acquisition subunit is used to perform contradiction statement detection on the information in the temporary sub-knowledge base, identify mutually contradictory information, and associate and record the source information and complete content information of each contradictory information to obtain the contradiction statement detection result; The second result acquisition subunit is used to detect expired content in the information containing time-sensitive identifiers in the temporary sub-knowledge base, compare the time-sensitive identifiers with the current date, filter and record expired information, and obtain the expired content detection result. The third result acquisition subunit is used to perform text representation detection on the information in the temporary sub-knowledge base from a preset user perspective, and obtain the representation detection result. The fourth result acquisition subunit is used to acquire the missing key information of the temporary sub-knowledge base output by the preset model when the information in the temporary sub-knowledge base does not meet the response requirements of the target question and answer text, and to use the missing key information as the knowledge gap detection result. The detection result acquisition subunit is used to take the contradictory statement detection result, the expired content detection result, the statement detection result, and the knowledge gap detection result as the multi-dimensional detection result.

[0252] Optionally, the question-and-answer learning agent includes: a wrong-example learning agent and a positive-example learning agent. The case library generation module includes: The error case acquisition unit is used to use the error case learning agent to perform error mining and improvement point analysis on the error question and answer text, the error reason annotation information of the error question and answer text, and the correct answer annotation information of the error question and answer text to obtain error cases of question and answer; The correct case acquisition unit is used to analyze the correct question and answer text annotated in the question and answer text using the positive example learning agent to obtain correct question and answer cases; The case library generation unit is used to generate the question and answer case library based on the question and answer error cases and the question and answer correct cases.

[0253] Optionally, the fault-based learning agent includes: a master coordinating agent, a research agent, and a regression testing agent. The error case acquisition unit includes: The error type determination subunit is used to determine the error type corresponding to the error question and answer text by means of the main coordinating agent, based on the error cause annotation information, the associated interaction information of the error question and answer text, and the correct answer annotation information; The error cause analysis result acquisition subunit is used by the research agent to analyze the error question and answer text based on the error type using the correct answer annotation information as a standard, and obtain the error cause analysis result. The evaluation report generation subunit is used to call the regression test agent, use the correct answer annotation information as the test evaluation benchmark, test the optimization strategy in the error cause analysis results, and generate an evaluation report containing the strategy effect and risk assessment results. The error case acquisition subunit is used to call the main coordinating agent, based on the correct answer annotation information, to summarize the error cause analysis results and the evaluation report, obtain improvement suggestion information, and integrate the improvement suggestion information and the error question and answer text to obtain the question and answer error case.

[0254] Optionally, the research agent includes: a knowledge base content review agent, a retrieval process simulation and analysis agent, and a generation process analysis agent. The sub-unit for obtaining the error cause analysis results includes: The review result acquisition subunit is used to call the knowledge base content review agent to review the content in the question-and-answer knowledge base when the answer corresponding to the question-and-answer text is marked incorrectly, and obtain the content review result; The analysis and inspection result acquisition subunit is used to call the retrieval process simulation and analysis agent to analyze the retrieval system corresponding to the erroneous question and answer text when the context of the answer corresponding to the question and answer text is incomplete or irrelevant to the question and answer text, and obtain the analysis and inspection result; The generation process analysis result acquisition subunit is used to call the generation process analysis agent to perform generation process analysis on the large model when the error type is the large model error type or the answer synthesis error type, and obtain the generation process analysis result. The error cause analysis result determination subunit is used to determine the content review result, the analysis and inspection result, and the generation process analysis result as the error cause analysis result.

[0255] Optionally, the review result acquisition subunit includes: The target knowledge point extraction subunit is used to call the knowledge base content review agent to extract the target knowledge point from the context information corresponding to the answer when the answer to the question and answer text is marked as incorrect. The search results acquisition sub-unit is used to call the knowledge base retrieval tool, use the target knowledge point as the search term to search the question and answer knowledge base, and obtain the corresponding search results; The information acquisition subunit is used to compare the target knowledge point with the search results, identify contradictory information, and associate and record the source information of each contradictory piece of information. The explanatory information generation subunit is used to generate error cause analysis conclusions and corresponding explanatory information based on the contradictory information and the source information; The content review result acquisition subunit is used to take the error cause analysis conclusion, the contradiction explanation information and each contradiction information as the content review result.

[0256] Optionally, the subunit for obtaining the analysis and inspection results includes: The simulation result acquisition subunit is used to call the retrieval process simulation and analysis agent. When the context of the answer corresponding to the question and answer text is incomplete or irrelevant to the question and answer text, the question rewriting tool is called to rewrite the original question text corresponding to the question and answer text, and the knowledge base retrieval tool is used to simulate the retrieval of the rewritten question text to obtain the simulation retrieval result. The multiple search result acquisition subunit is used to retrieve the original question text using the knowledge base retrieval tool with different search parameters to obtain multiple search results; The first analysis result acquisition subunit is used to determine the error cause analysis result and the description information corresponding to the error cause analysis result based on the simulated search result and the multiple search results, and to use the error cause analysis result and the description information as the analysis check result; The second analysis result acquisition subunit is used to retrieve the original question text using a knowledge base traversal tool if the simulated retrieval results and the multiple retrieval results do not meet the requirements of the original question text, and to determine the missing information associated with the original question text in the question-and-answer knowledge base based on the retrieved content, so as to obtain the analysis and inspection results.

[0257] Optionally, the subunit for obtaining the analysis results of the generation process includes: The prompt word analysis result acquisition subunit is used to call the generation process analysis agent. When the error type is a large model error type or an answer synthesis error type, the prompt word analysis tool is used to analyze the prompt words in the generation process of the original question text to obtain the prompt word analysis result. The capability analysis result acquisition sub-unit is used to analyze the model reasoning process of the large model using large model analysis tools to obtain model capability analysis results; The generation process analysis result determination subunit is used to determine the generation process analysis result based on the prompt word analysis result and the model capability analysis result.

[0258] Optionally, the evaluation report generation subunit includes: The strategy acquisition subunit is used to call the regression test agent to acquire the optimization strategy to be tested; the optimization strategy to be tested is the strategy in the error question example. The question input subunit is used to input each question in the test set into two parallel question-answering systems simultaneously; wherein, one question-answering system applies the optimization strategy to be tested, and the other question-answering system does not apply the optimization strategy to be tested. The strategy effect acquisition subunit is used to acquire the output results of the two question-answering systems corresponding to each question, and compare them with the standard answers in the test set to obtain the strategy effect of the optimization strategy to be tested. The risk assessment result acquisition subunit is used to perform risk assessment on the optimization strategy to be tested based on the comparison results, and obtain the risk assessment result; The evaluation report generation subunit is used to generate the evaluation report based on the effectiveness of the strategy and the risk assessment results.

[0259] Optionally, the device further includes: The clustering processing module is used to use the question-and-answer example learning agent to cluster the question-and-answer error cases in the question-and-answer example library when the clustering analysis conditions of the question-and-answer example library are determined to be met, so as to obtain the clustered error cases of the question-and-answer error cases; The common feature acquisition module is used to perform error commonality analysis on the clustering error cases to obtain the error common features of the clustering error cases; The general solution generation module is used to generate a general solution corresponding to the clustering error cases based on the common error characteristics.

[0260] Optionally, the correct case acquisition unit includes: The test result acquisition subunit is used to use the positive example learning agent to perform policy stripping test and / or replacement comparison test on the labeled correct question and answer text to obtain the test results; The attribution conclusion acquisition subunit is used to analyze and obtain the successful attribution conclusion information corresponding to the correct question and answer text based on the test results, and generate an optimization method based on the successful attribution conclusion information. The correct case generation subunit is used to generate the correct question and answer case based on the original question text of the correct question and answer text and the optimization method.

[0261] Optionally, the target case search module includes: The standard answer acquisition unit is used to retrieve and reorder the question text using a hybrid retrieval algorithm and a reordering algorithm when the question text is received, so as to obtain the standard answer corresponding to the question text. The target case search unit is used to search for target question and answer examples from the question and answer example library when the standard answer does not meet the answer requirements of the question text. The similarity between the target question and answer example and the question text is higher than the similarity threshold.

[0262] Optionally, the answer generation module includes: The system optimization unit is used to optimize the system modules in the question-answering system that match the system optimization parameters in the question-answering system when the target question-answering example is a question-answering error case; wherein the system modules include at least one of: a retrieval engine, a large model, and a knowledge base; The first answer generation unit is used to identify the user's user intent based on the user profile and the question text when the question text is a real-time question text, generate an initial answer based on the user intent, the retrieval results of the question-and-answer knowledge base and the reference information in the target question and answer example, and process the initial answer in combination with the user profile to generate the target answer of the question text; The second answer generation unit is used to generate an initial answer based on the question text, the search results of the question-and-answer knowledge base, and the reference information in the target question-and-answer example when the question text is a non-real-time question text. The large model generates an answer tracing report based on the process information of answer generation. The initial answer and the answer tracing report are integrated into the final answer. The answer tracing report is used for post-quality inspection.

[0263] The knowledge-based question-answering device provided in this application automatically checks the question-answering knowledge base through a knowledge base checking agent, replacing traditional manual, piecemeal analysis and reducing manpower costs. Simultaneously, the question-answering example learning agent accumulates "correct and incorrect cases" to build a bidirectional case library, avoiding the bias of relying solely on negative feedback. Furthermore, the agent executes according to preset rules, eliminating the influence of individual ability on root cause localization and repair effectiveness, thus achieving automated and standardized knowledge iteration. The device uses the RAG algorithm combined with the question-answering example library to generate answers, eliminating the need for incremental training or supervised fine-tuning of large models, completely eliminating dependence on high-configuration graphics card servers and reducing hardware investment costs. Moreover, knowledge updates only require adjusting the knowledge base and updating the case library through the agent, without retraining the model. This allows for rapid adaptation to frequent changes in business knowledge and model foundations, solving the problem of high and impractical incremental training costs. By combining the reuse of positive and negative experiences from the case library with the precise retrieval of the RAG algorithm, the illusion of large models is effectively reduced, and the accuracy of answers is improved. At the same time, the closed-loop mechanism of intelligent inspection, case accumulation and answer generation allows the system to flexibly adapt to business changes, ultimately achieving the optimization goals of low cost and automation, which meets the compliance and practicality requirements of bank question-and-answer systems.

[0264] This application also provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the above-described knowledge question-and-answer method.

[0265] Figure 4 A schematic diagram of the structure of an electronic device 400 according to an embodiment of the present invention is shown. Figure 4 As shown, the electronic device 400 includes a central processing unit (CPU) 401, which can perform various appropriate actions and processes according to computer program instructions stored in read-only memory (ROM) 402 or loaded from storage unit 408 into random access memory (RAM) 403. The RAM 403 may also store various programs and data required for the operation of the electronic device 400. The CPU 401, ROM 402, and RAM 403 are interconnected via a bus 404. An input / output (I / O) interface 405 is also connected to the bus 404.

[0266] Multiple components in electronic device 400 are connected to I / O interface 405, including: input unit 406, such as keyboard, mouse, microphone, etc.; output unit 407, such as various types of monitors, speakers, etc.; storage unit 408, such as disk, optical disk, etc.; and communication unit 409, such as network card, modem, wireless transceiver, etc. Communication unit 409 allows electronic device 400 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0267] The various processes and handling described above can be executed by processing unit 401. For example, the methods of any of the above embodiments can be implemented as computer software programs tangibly contained in a computer-readable medium, such as storage unit 408. In some embodiments, part or all of the computer program can be loaded and / or installed on electronic device 400 via ROM 402 and / or communication unit 409. When the computer program is loaded into RAM 403 and executed by CPU 401, one or more actions of the methods described above can be performed.

[0268] Additionally, embodiments of this application also provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the aforementioned knowledge-based question-and-answer method.

[0269] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal that includes said element.

Claims

1. A knowledge-based question-and-answer method, characterized in that, The method includes: The knowledge base inspection agent is used to inspect the target question and answer text in the question and answer knowledge base, obtain the inspection results, and send the inspection results to the maintenance personnel so that the maintenance personnel can correct the content of the question and answer knowledge base. The question-and-answer example learning agent analyzes the question-and-answer text in the question-and-answer system and generates a question-and-answer example library based on the analysis results; wherein, the question-and-answer example library includes: incorrect question-and-answer cases and correct question-and-answer cases; When a user inputs a question text, the system searches the question-and-answer example database for a target question-and-answer example whose similarity to the question text is higher than a similarity threshold. The question-and-answer system is optimized based on the system optimization parameters in the target question-and-answer example, and the answer corresponding to the question text is generated based on the question-and-answer knowledge base and the target question-and-answer example after optimization.

2. The method according to claim 1, characterized in that, The knowledge base inspection agent checks the target question-and-answer text within the question-and-answer knowledge base to obtain the inspection results, including: The intelligent agent is checked using the knowledge base to determine whether it meets the checking conditions of the question-and-answer knowledge base. When the check conditions of the question-and-answer knowledge base are met, the target question-and-answer text is obtained from the historical question-and-answer text of the question-and-answer system using a question acquisition tool; the target question-and-answer text refers to the question-and-answer text in the historical question-and-answer text of the question-and-answer system whose question-and-answer frequency is greater than a first frequency threshold; wherein, the first frequency threshold is dynamically set according to the total amount of knowledge base, device capacity, and task processing time; For each target question and answer text, a knowledge base traversal tool is used to traverse the question and answer knowledge base to obtain target information fragments associated with the target question and answer text, and a temporary sub-knowledge base is constructed based on the target information fragments; The temporary sub-knowledge base is subjected to multi-dimensional detection to obtain multi-dimensional detection results; The multi-dimensional detection results are summarized to generate an inspection result that includes a problem summary, a knowledge problem list, and corresponding optimization suggestions. The knowledge problem list includes information related to the source of the problem.

3. The method according to claim 2, characterized in that, The determination of the check conditions for meeting the question-and-answer knowledge base includes: If the interval between checking the question-and-answer knowledge base reaches a set time, it is determined that the check condition of the question-and-answer knowledge base has been met. If the number of newly added error cases in the question-and-answer knowledge base reaches a certain threshold, the inspection condition of the question-and-answer knowledge base is determined to be met; the newly added error cases refer to the error cases added to the question-and-answer knowledge base since the last inspection. If the change in the specified question list pre-stored in the question-answering system exceeds a threshold, the check condition of the question-answering knowledge base is determined to be met; the specified question list records questions with a question-answering frequency greater than a second frequency threshold.

4. The method according to claim 2, characterized in that, The multi-dimensional detection of the temporary sub-knowledge base, to obtain multi-dimensional detection results, includes: The information in the temporary sub-knowledge base is subjected to contradiction statement detection to identify mutually contradictory information, and the source information and complete content information of each contradictory information are associated and recorded to obtain the contradiction statement detection result; The information containing time-sensitive identifiers in the temporary sub-knowledge base is checked for expired content. The time-sensitive identifiers are compared with the current date, and expired information is filtered and recorded to obtain the expired content detection results. The information in the temporary sub-knowledge base is subjected to textual expression detection from a preset user perspective to obtain expression detection results; When the information in the temporary sub-knowledge base does not meet the response requirements of the target question and answer text, the missing key information of the temporary sub-knowledge base output by the preset model is obtained, and the missing key information is used as the knowledge gap detection result. The results of the contradictory statement detection, the results of the expired content detection, the results of the statement detection, and the results of the knowledge gap detection are used as the multi-dimensional detection results.

5. The method according to claim 1, characterized in that, The question-and-answer example learning agent includes: a wrong example learning agent and a positive example learning agent. The method of using a question-and-answer example learning agent to analyze the question-and-answer text in the question-and-answer system and generate a question-and-answer example library based on the analysis results includes: Using the error case learning agent, the error question and answer text, the error reason annotation information of the error question and answer text, and the correct answer annotation information are analyzed to mine and analyze the error cause and improvement point of the question and answer text, and to obtain the question and answer error cases. Using the positive example learning agent, the correct question and answer text labeled in the question and answer text is analyzed to obtain correct question and answer cases; The question and answer example library is generated based on the incorrect and correct question and answer examples.

6. The method according to claim 5, characterized in that, The error-learning agent includes: a main coordinating agent, a research agent, and a regression testing agent. The error-learning agent uses the aforementioned error case learning mechanism to perform error mining and improvement point analysis on the erroneous question-and-answer text, the error reason annotation information of the erroneous question-and-answer text, and the correct answer annotation information of the question-and-answer text, to obtain question-and-answer error cases, including: The main coordinating agent determines the error type corresponding to the error question and answer text based on the error reason annotation information, the associated interaction information of the error question and answer text, and the correct answer annotation information. Using the research agent and the correct answer annotation information as a standard, the error type is analyzed in the error question and answer text to obtain the error cause analysis results. The regression testing agent is invoked, and the correct answer annotation information is used as the test evaluation benchmark to test the optimization strategy in the error cause analysis results, and an evaluation report containing the strategy effect and risk assessment results is generated. The main coordinating agent is invoked to summarize the error analysis results and the evaluation report based on the correct answer annotation information, obtain improvement suggestion information, and integrate the improvement suggestion information with the erroneous question and answer text to obtain the question and answer error case.

7. The method according to claim 6, characterized in that, The research agents include: a knowledge base content review agent, a retrieval process simulation and analysis agent, and a generation process analysis agent. The research agent analyzes the error question-and-answer text based on the correct answer annotation information to obtain error cause analysis results, including: The knowledge base content review agent is invoked to review the content in the question-and-answer knowledge base when the answer corresponding to the question-and-answer text is incorrectly labeled, and the content review result is obtained. The retrieval process is invoked to simulate and analyze the intelligent agent. When the context of the answer corresponding to the question and answer text is incomplete or irrelevant to the question and answer text, the retrieval system corresponding to the erroneous question and answer text is analyzed to obtain the analysis and inspection results. The generation process analysis agent is invoked to perform generation process analysis on the large model when the error type is a large model error type or an answer synthesis error type, and the generation process analysis results are obtained. The content review results, the analysis and inspection results, and the generation process analysis results are determined as the error cause analysis results.

8. The method according to claim 7, characterized in that, The intelligent agent that invokes the knowledge base for content review reviews the content in the question-and-answer knowledge base when the answer corresponding to the question-and-answer text is incorrectly labeled, and obtains the content review result, including: The knowledge base content review agent is invoked to extract target knowledge points from the context information corresponding to the answer when the answer to the question and answer text is marked as incorrect. The knowledge base retrieval tool is invoked to search the question-and-answer knowledge base using the target knowledge point as the search term, and the corresponding search results are obtained. By comparing the target knowledge points with the search results, contradictory information is identified, and the source information of each contradictory piece of information is associated and recorded. Based on the contradictory information and the source information, an error cause analysis conclusion and corresponding contradictory explanation information are generated; The error analysis conclusions, the contradiction explanations, and the contradiction information are used as the content review results.

9. The method according to claim 7, characterized in that, The analysis of the retrieval system corresponding to the erroneous question-and-answer text yields the following analysis and inspection results: The retrieval process is simulated and analyzed by an intelligent agent. When the context of the answer corresponding to the question and answer text is incomplete or irrelevant to the question and answer text, the question rewriting tool is invoked to rewrite the original question text corresponding to the question and answer text, and the knowledge base retrieval tool is used to simulate the retrieval of the rewritten question text to obtain the simulated retrieval results. Using the knowledge base retrieval tool with different retrieval parameters, the original question text was retrieved, yielding multiple retrieval results; Based on the simulated search results and the multiple search results, the error cause analysis results and the corresponding descriptive information are determined, and the error cause analysis results and the descriptive information are used as the analysis and inspection results. If the simulated search results and the multiple search results do not meet the requirements of the original question text, then the knowledge base traversal tool is used to search the original question text, and the missing information associated with the original question text in the question-and-answer knowledge base is determined based on the content returned by the search, so as to obtain the analysis and inspection results.

10. The method according to claim 7, characterized in that, The process of calling the generation process analysis agent, when the error type is a large model error type or an answer synthesis error type, performs generation process analysis on the large model to obtain generation process analysis results, including: The generation process analysis agent is invoked. When the error type is a large model error type or an answer synthesis error type, the prompt word analysis tool is used to analyze the prompt words in the generation process of the original question text to obtain the prompt word analysis results. The model inference process of the large model is analyzed using a large model analysis tool to obtain the model capability analysis results; Based on the results of the prompt word analysis and the results of the model capability analysis, the analysis results of the generation process are determined.

11. The method according to claim 6, characterized in that, The process involves invoking the regression testing agent, using the correct answer annotation information as the testing evaluation benchmark, to test the optimization strategies in the error cause analysis results, and generating an evaluation report containing strategy effectiveness and risk assessment results, including: The regression testing agent is invoked to obtain the optimization strategy to be tested; the optimization strategy to be tested is the strategy in the erroneous question answer example. Each question in the test set is simultaneously input into two parallel question-answering systems; wherein, one question-answering system applies the optimization strategy to be tested, and the other question-answering system does not apply the optimization strategy to be tested. Obtain the output results of the two question-answering systems corresponding to each question, and compare them with the standard answers in the test set to obtain the strategy effect of the optimization strategy to be tested; Based on the comparison results, a risk assessment is performed on the optimization strategy to be tested, and the risk assessment results are obtained. The assessment report is generated based on the effectiveness of the strategy and the results of the risk assessment.

12. The method according to claim 1, characterized in that, The method further includes: When the clustering analysis conditions of the question-and-answer example library are met, the question-and-answer example learning agent is used to cluster the question-and-answer error cases in the question-and-answer example library to obtain the clustered error cases of the question-and-answer error cases; The common error characteristics of the clustering error cases are obtained by performing error commonality analysis on the clustering error cases; Based on the common characteristics of the errors, a general solution is generated for the clustering error cases.

13. The method according to claim 5, characterized in that, The step of using the positive example learning agent to analyze the correctly labeled question-and-answer text in the question-and-answer text to obtain correct question-and-answer cases includes: Using the positive example learning agent, the labeled correct question and answer text is subjected to policy stripping test and / or replacement comparison test to obtain test results; Based on the test results, the successful attribution conclusion information corresponding to the correct question and answer text is analyzed, and an optimization method is generated based on the successful attribution conclusion information. The correct question and answer case is generated based on the original question text and the optimization method.

14. The method according to claim 1, characterized in that, The step of searching for target question-and-answer examples with a similarity higher than a similarity threshold from the question-and-answer example database when receiving question text input by the user includes: Upon receiving the question text, a hybrid retrieval algorithm and a reordering algorithm are used to retrieve and reorder the question text to obtain the standard answer corresponding to the question text. When the standard answer does not meet the requirements of the question text, a target question-answer example with a similarity higher than the similarity threshold is searched from the question-answer example library.

15. The method according to claim 1, characterized in that, The step of optimizing the question-answering system based on the system optimization parameters in the target question-answer example, and generating the answer corresponding to the question text based on the question-answering knowledge base and the target question-answer example after optimization, includes: When the target question answer example is a question answering error case, the system modules in the question answering system that match the system optimization item parameters are optimized according to the system optimization item parameters in the target question answer example; wherein, the system modules include at least one of: a retrieval engine, a large model, and a knowledge base; When the question text is a real-time question, the user's intent is identified based on the user profile and the question text. An initial answer is generated based on the user intent, the search results of the question-and-answer knowledge base, and the reference information in the target question and answer example. The initial answer is then processed in conjunction with the user profile to generate the target answer for the question text. When the question text is a non-real-time question, an initial answer is generated based on the question text, the search results of the question-and-answer knowledge base, and the reference information in the target question and answer examples. The large model then generates an answer tracing report based on the process information of answer generation. The initial answer and the answer tracing report are integrated into the final answer. The answer tracing report is used for subsequent quality inspection.

16. A knowledge-based question-and-answer device, characterized in that, The device includes: The inspection result acquisition module is used to use a knowledge base inspection agent to inspect the target question and answer text in the question and answer knowledge base, obtain the inspection result, and send the inspection result to the maintenance personnel so that the maintenance personnel can correct the content of the question and answer knowledge base. The case library generation module is used to analyze the question and answer text in the question-and-answer system using a question-and-answer example learning agent, and generate a question-and-answer example library based on the analysis results; wherein, the question-and-answer example library includes: incorrect question-and-answer cases and correct question-and-answer cases; The target case search module is used to search for target question and answer examples with a similarity higher than a similarity threshold from the question and answer example library when receiving question text input by the user. The answer generation module is used to optimize the question-and-answer system based on the system optimization parameters in the target question-and-answer example, and after optimization, generate the answer corresponding to the question text based on the question-and-answer knowledge base and the target question-and-answer example.

Images