Multimodal-based knowledge base vertical field intelligent question answering engineering method

Abstract

The application discloses a multi-modal-based knowledge base vertical field intelligent question answering engineering method and belongs to the technical field of intelligent question answering.The application solves the problems of the existing method, such as confusion of answer sources, lack of credible reference annotation, lack of statistical reasoning, imperfect self-learning of a knowledge base, error-prone long answer output and the like, and the problems are difficult to meet the credibility and engineering requirements of vertical field question answering.Through unified analysis on multi-modal input questions of a user, context is supplemented and an intention is recognized, knowledge is recalled from a knowledge base and scored to determine whether evidence is sufficient; according to a judgment result, an answer is generated according to a knowledge base credible answer or a network backup; for statistical reasoning type problems, automatic conversion to a query plan is completed to calculate and determine, corresponding formats are output according to types, long texts are guaranteed to be complete through an outline, a state machine and breakpoint continuation writing; and after candidate knowledge is collected and audited, the knowledge base is updated in a versioned manner, and a full-process traceable vertical field intelligent question answering closed loop is formed.

Description

Technical Field

[0001] This invention relates to the field of intelligent question answering technology, specifically to an engineering method for intelligent question answering in vertical domains based on a multimodal knowledge base. Background Technology

[0002] Currently, there are three main types of intelligent question-answering technologies in vertical fields, all of which have obvious shortcomings.

[0003] The first type involves directly calling a large language model for question answering. This approach can understand natural language and generate fluent text, but in vertical scenarios with significant constraints such as professionalism, timeliness, region, version, and rules, it is prone to generating incorrect content, creating illusions, and it is impossible to distinguish the source of the answer, making it difficult for users to judge its credibility.

[0004] The second category is traditional retrieval-enhanced question-answering systems. These systems generate answers by combining knowledge base retrieval with large models, which can reduce some illusions. However, they rely solely on semantic similarity for recall and do not comprehensively consider factors such as knowledge source, document version, applicable conditions, update time, business tags, and citation basis. This makes them prone to recalling incorrect knowledge or mixing knowledge from different conditions, resulting in insufficient answer accuracy.

[0005] The third type is a question-and-answer system with a fixed knowledge base or enterprise knowledge base. It can only answer existing questions and will directly refuse to answer or give a generalized answer to questions that are not covered. It lacks a clear mechanism to distinguish between knowledge base answers and online search answers and cannot achieve controlled fallback answers.

[0006] In summary, the existing technology has the following core defects: There is no clear mechanism to distinguish between knowledge base answers and online search answers, the sources of answers are confused, and their credibility is difficult to verify; The lack of a strict knowledge citation and annotation mechanism makes the answers untraceable, which is not conducive to manual verification and compliant use. When the local knowledge base fails to retrieve sufficient evidence to directly support the answer, the existing system lacks a controlled fallback process, which can easily lead to direct rejection of the answer, or generate uncertain or even fabricated answers in the absence of reliable evidence. It lacks knowledge-based statistical analysis and logical reasoning capabilities, and can only answer factual questions; The system lacks the ability to analyze and orchestrate multimodal inputs and outputs, and its interaction and presentation formats are limited. The knowledge base's self-learning and updating mechanism is imperfect, manual maintenance is inefficient, and direct writing can easily pollute the knowledge base. Long answers are prone to problems such as truncation, omissions, incorrect citations, and missing formatting, resulting in poor usability. Knowledge documents are segmented with fixed lengths, resulting in semantic fragmentation and low retrieval accuracy. Using a single search channel results in incomplete recall and is difficult to adapt to areas with high rule constraints, such as medical insurance.

[0007] Therefore, there is an urgent need to construct an integrated engineering technology solution that can achieve multimodal unified processing, evidence-driven credible answers, network fallback, statistical reasoning, adaptive segmentation, hybrid retrieval, knowledge graph reasoning, vertical domain compliance analysis, long answer integrity control, and secure self-learning updates. Summary of the Invention

[0008] The purpose of this invention is to provide an engineering method for intelligent question answering in vertical domains based on a multimodal knowledge base. This method involves uniformly parsing user multimodal input questions, completing the context, identifying intent, retrieving knowledge from the knowledge base, and scoring the sufficiency of evidence. Based on the judgment result, an answer is generated either as a reliable answer from the knowledge base or as a fallback answer from the internet. For statistical reasoning questions, the method automatically converts them into query plans to complete calculations and judgments, outputting corresponding formats according to type. Long texts are kept complete through outlines, state machines, and breakpoint continuation. Furthermore, candidate knowledge is collected, reviewed, and versioned to update the knowledge base, forming a fully traceable closed loop for intelligent question answering in vertical domains, thus solving the problems mentioned in the background technology.

[0009] To achieve the above objectives, the present invention provides the following technical solution: An engineering approach for intelligent question answering in vertical domains based on multimodal knowledge bases includes the following steps: S1: Receive multimodal questions input by users, transcribe speech into text, extract text and structure from images and documents, identify fields and relationships in tables, and unify input questions from different modalities into standardized question representations; S2: Read the content of historical dialogues in multi-turn dialogues, and filter, compress and resolve the historical context to filter key information in historical dialogues and form searchable questions; S3: Perform intent recognition on user questions, and classify questions into knowledge query, rule explanation, statistical analysis, logical reasoning, image understanding, table analysis, prescription analysis, expense list analysis, reimbursement judgment, online search and knowledge base update suggestions; based on the recognition results, the questions are routed to different processing links; S4: Perform adaptive segmentation of knowledge documents by type, bind metadata, and build a hybrid index that integrates vector library, full-text index, structured library, and knowledge graph; and collect, classify, and access non-public information such as internal regulations, corporate systems, and industry rules through browser plugin intelligent agents, local desktop OCR intelligent agents, and AnySearch tools to build a proprietary knowledge base that search engines cannot access, forming a collaborative collection architecture; the adaptive segmentation includes: parent-child segmentation, fixed-length segmentation, table row segmentation, semantic segmentation, structured segmentation, image compensation segmentation, question-answer pair segmentation, and case segmentation; the metadata includes document name, chapter, clause, version, effective time, expiration time, applicable region, applicable object, medical insurance code, and source level; S5: Generate multiple query variants based on user questions and execute multi-channel hybrid retrieval, deduplicatize, fuse, and multi-dimensionally score candidate knowledge to determine whether there is valid evidence to support the answer; the multi-channel hybrid retrieval includes: vector retrieval, keyword retrieval, full-text retrieval, structured query, rule matching, and knowledge graph multi-hop query; multi-dimensional scoring factors include semantic similarity, keyword matching degree, knowledge source level, update time, document version, applicable conditions, medical insurance code matching degree, knowledge graph path length, and contextual consistency; S6: Based on the identified entities, perform one-hop, two-hop, or multi-hop relationship queries in the knowledge graph to form entity relationship paths, related evidence, and reasoning basis; and integrate the graph reasoning results with the retrieved evidence to provide related support for subsequent compliance analysis, reimbursement judgment, and risk identification; S7: For scenarios involving contract compliance review, bidding compliance review, and public canteen procurement compliance review, relevant clauses from the internal rules knowledge base will be prioritized as the core basis; structured identification will be performed on images or documents such as prescriptions, medication lists, expense lists, invoices, examination reports, and test reports to extract key information such as drugs, items, amounts, medical insurance codes, diagnoses, and medical institutions; combining the medical insurance catalog, policy clauses, business rules, and knowledge graph relationships, the feasibility of reimbursement, the reasonableness of expenses, and medical risk analysis will be completed, and definitive conclusions, risk warnings, and cited evidence will be output; S8: Automatically adapts to multimodal output formats such as text, tables, charts, reference cards, and analysis reports based on question type and answer content.

[0010] Furthermore, it also includes the following steps: S9: For long text responses, first generate an outline. The response outline includes chapter titles, the response objectives for each chapter, the required references, the expected output format, and the completion status. The generation progress is tracked by a state machine, and the system automatically resumes writing at breakpoints when truncation or omission occurs, thus completing the integrity verification.

[0011] Furthermore, it also includes the following steps: S10: During operation, user feedback, manually corrected content, low-confidence issues, high-frequency online search issues, user satisfaction Q&A pairs, and newly added business data are collected and entered into the candidate knowledge pool. The candidate knowledge is subject to multiple reviews, and the candidate knowledge that passes the review is entered into the formal knowledge base. Once the candidate knowledge passes the review, a new knowledge version is generated, and the relevant indexes, citation numbers, and metadata are updated synchronously.

[0012] Furthermore, in S5, determining whether there is valid evidence to support the answer includes the following steps: If the evidence in the knowledge base is deemed sufficient, the system enters the knowledge base credible answer mode. In this mode, an answer is generated based on the knowledge base evidence, and the document name, clause, version, update time, and citation number are simultaneously annotated before the credible answer is output.

[0013] If the evidence in the knowledge base is deemed insufficient, no definitive conclusion is generated, and the system enters the fallback mode of online search. In the fallback mode of online search, external trusted sources or public network searches are called to generate online reference answers, clearly indicating the online search answer / external reference answer, outputting the reference answer and prompting that manual confirmation is required.

[0014] Furthermore, in S5, determining whether there is valid evidence to support the answer also includes the following steps: Perform task parsing on users' natural language questions to identify whether the current question belongs to the category of statistical calculation, trend analysis, condition filtering, rule judgment, or comprehensive reasoning. If the question pertains to statistical analysis or logical reasoning, the user's natural language question will be converted into a unified query plan. If the knowledge is structured, the unified query plan will be converted into a structured query statement; if the knowledge is graph-based, the unified query plan will be converted into a graph query statement. If the knowledge belongs to document type, the unified query plan will be converted into semantic search conditions, keyword search conditions, or fragment filtering conditions. Based on the unified query plan, statistical objects, statistical fields, time ranges, classification dimensions, and filtering conditions are identified from the knowledge base. The structured data, semi-structured data, document fragments, question and answer records, or extracted knowledge facts in the knowledge base are then calculated and analyzed to obtain statistical results.

[0015] Furthermore, in S5, determining whether there is valid evidence to support the answer also includes the following steps: If it is a reasoning problem, logical judgment is made based on the rules, clauses, knowledge graph relationships, time conditions, applicable objects, contextual evidence and business constraints in the knowledge base to generate a reasoning conclusion; When outputting statistical analysis results or inference conclusions, the knowledge sources, rule bases, statistical scope, and intermediate calculation results involved in the calculation or inference should be marked simultaneously. If the data required for statistics or reasoning is incomplete in the current knowledge base, a message will be displayed indicating that the current conclusion is generated only based on the existing knowledge base content, and the scope of missing data or the knowledge content that needs to be supplemented will be explained.

[0016] Furthermore, in S9, the generation progress is tracked through a state machine, including the following steps: When generating segments, maintain a global generation state machine or structured state queue to record the generation state of each segment, table, chart, reference, conclusion, and supplementary explanation. For long text responses, first generate a response outline including chapter titles, the target of each chapter, the required references, and the expected output format, and initialize a generation status queue including six states: not started, generating, completed, to be completed, to be verified, and abnormal interruption. After generating the answer content in segments and binding the references, a segment integrity check is performed. The segment integrity check verifies each item from five dimensions: whether the paragraph is missing, whether the table is complete, whether the reference is missing, whether the conclusion is missing, and whether the format is abnormal. If the verification result is found to be missing or interrupted, a continuation instruction will be automatically generated based on the breakpoint position recorded in the globally generated state machine or structured state queue. Once all segments are generated, the system performs a final integrity check. If the final verification still finds missing content, the completion process will be triggered again. If there are no missing or interrupted parts, proceed to the final integrity check. If the check passes, output the complete long answer.

[0017] Furthermore, the knowledge base can be one or more combinations of a document library, vector library, full-text index library, knowledge graph, rule library, structured database, question-answer pair library, and business dataset.

[0018] Compared with the prior art, the beneficial effects of the present invention are: 1. By first determining whether there is sufficient evidence in the current knowledge base before answering, if the knowledge base contains an answer, an accurate answer is given based on the knowledge base, and the source of the reference is marked, the credibility of vertical domain question answering is improved, avoiding the large language model from generating answers out of thin air. When the knowledge base does not contain an answer, the system triggers an online search as a fallback, and clearly marks the answer as either an online search answer or an external reference answer, preventing users from mistaking external reference content for confirmed knowledge base content, and achieving a clear distinction between knowledge base answers and online search answers.

[0019] 2. This invention can not only answer single knowledge query questions, but also perform statistical analysis, trend comparison, rule judgment and logical reasoning based on the existing knowledge base, thereby upgrading the knowledge base from a static database to an analyzable and reasonable knowledge service system, and enhancing the analytical and reasoning capabilities of the knowledge base.

[0020] 3. Through candidate knowledge pools, multi-level review, version management, and hot update mechanisms, the knowledge base can be continuously improved based on user feedback, manual correction, and online search results, while avoiding erroneous knowledge from directly polluting the official knowledge base, thus achieving secure self-learning and continuous updating of the knowledge base.

[0021] 4. By implementing outline planning, segmented generation, reference binding, and integrity verification mechanisms, it reduces issues such as mid-way truncation, paragraph omissions, incomplete tables, and incorrect references in long answers, thus solving the problem of incomplete output of long answers. It can be applied to various vertical industry scenarios such as government affairs, medical care, medical insurance, finance, education, enterprise knowledge management, industrial operation and maintenance, and customer service knowledge base.

[0022] 5. By employing various document segmentation methods such as parent-child segmentation, fixed-length segmentation, line segmentation, semantic segmentation, structured segmentation, and image recognition segmentation, policy clauses, medical insurance catalogs, reimbursement rules, expense items, and Q&A materials can be organized according to business semantics, avoiding contextual breaks caused by simple segmentation and improving the accuracy of knowledge retrieval and citation.

[0023] 6. By establishing relationships between symptoms, diseases, diagnoses, drugs, treatment items, medical insurance catalogs, payment categories, limited payment scope, insurance types, regional policies, and cost rules through a medical insurance policy knowledge graph, the system identifies prescription images, medication lists, cost details, and invoices uploaded by users. Combining the medical insurance reimbursement catalog, structured rules, vector retrieval results, and knowledge graph relationships, the system provides auxiliary analysis on whether drugs are reimbursable, whether there are limited payment conditions, whether there is duplicate charging, and the risk of abnormal costs or over-treatment. This enables the system to perform multi-hop expansion and associative reasoning based on single entities input by users, thereby answering extended questions related to medical insurance policies that are not explicitly expressed by users, improving the intelligent review capabilities of prescriptions, medication lists, and cost lists. Attached Figure Description

[0024] Figure 1 This is a diagram illustrating the overall architecture of the vertical domain trusted question answering and compliance analysis system of the present invention. Figure 2 This is a flowchart illustrating the process of separating knowledge base answers from online search answers in this invention. Figure 3 This is a diagram illustrating the knowledge base trusted reference and evidence constraint generation mechanism of the present invention; Figure 4 This is a flowchart illustrating the statistical analysis and associative reasoning process based on vector libraries, structured rules, and knowledge graphs of the present invention. Figure 5 This is a flowchart of the knowledge base self-learning review and version update process of the present invention; Figure 6 This is a flowchart illustrating the long answer integrity control process of the present invention. Figure 7 This is a flowchart illustrating the adaptive segmentation and hybrid indexing process for knowledge documents in this invention. Figure 8 This is a flowchart of the multimodal recognition and compliance analysis of medical insurance prescriptions, medication lists, and expense lists according to the present invention. Detailed Implementation

[0025] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0026] To address the technical challenges of existing technologies, such as obfuscated answer sources, lack of credible citation annotations, absence of controlled fallback mechanisms, insufficient statistical reasoning, inadequate multimodal capabilities, imperfect knowledge base self-learning, and susceptibility to errors in long answer output, thus failing to meet the credibility, engineering, and compliance requirements of question answering in vertical domains, please refer to [link to relevant documentation]. Figures 1-8 This embodiment provides the following technical solution: An engineering approach for intelligent question answering in vertical domains based on multimodal knowledge bases includes the following steps: S1: Receives multimodal questions from user input, including text, voice, images, screenshots, documents, tables, or combinations of multiple modalities; transcribes voice into text, extracts text and structure from images and documents, identifies fields and relationships in tables, and uniformly converts input questions from different modalities into standardized question representations.

[0027] S2: In multi-turn dialogues, the system reads historical dialogue content and filters, compresses, and dereferences the historical context. It filters key information from historical dialogues, eliminates interference from pronouns and ellipsis, and forms a searchable question. This step is not simply piecing together all historical records, but rather extracting key information relevant to the current question, identifying pronouns or ellipsis such as "this," "the one above," or "what was just said," thus forming a complete question expression and reducing interference from irrelevant context. For example: In the first round, a user uploads a prescription image; in the second round, they ask, "Are these medications reimbursable?"; and in the third round, they ask, "Are there any duplicate prescriptions?" The system can point "these medications" to the list of medications identified in the previous round and map "duplicate prescriptions" to a risk assessment of duplication between similar medications, medications with the same generic name, medications with the same efficacy, or medications with the same medical insurance code.

[0028] S3: Identify user intent in questions and categorize them into knowledge queries, rule explanations, statistical analysis, logical reasoning, image comprehension, table analysis, prescription analysis, expense list analysis, reimbursement judgment, online search, and knowledge base update suggestions. Based on the identification results, the questions are routed to different processing paths. For example, general knowledge queries are routed to the knowledge base retrieval path; statistical questions to the statistical analysis path; reasoning questions to the knowledge reasoning path; and questions not found in the knowledge base are routed to the online search fallback path. Specifically, intents related to contract compliance review, bidding compliance review, and public canteen procurement compliance review will be routed to a dedicated internal rule knowledge base processing path, generating answers based on internal regulations.

[0029] S4: The system first receives raw knowledge data, including nine categories of materials: policy documents, medical insurance drug catalogs, treatment item catalogs, medical consumable catalogs, reimbursement rule documents, Q&A materials, tables, scanned documents / images, and historical review cases. It then sequentially performs document type recognition, layout parsing, and content extraction. Layout parsing and content extraction include six processing steps: title recognition, chapter recognition, paragraph recognition, table recognition, OCR recognition, and field extraction. After content extraction, the system selects the corresponding segmentation strategy based on the document type. It then performs adaptive segmentation of the knowledge documents by type, binds metadata, and constructs a hybrid index integrating vector libraries, full-text indexes, structured libraries, and knowledge graphs. Furthermore, it is equipped with a browser plugin intelligent agent, a local desktop OCR intelligent agent, and the AnySearch tool to collect, classify, and reach non-public information such as internal regulations, corporate systems, and industry rules, building a proprietary knowledge base that search engines cannot access, forming a collaborative collection architecture. The browser plugin intelligent agent... It can silently monitor non-public web resources such as policy documents, detailed rules, and announcements on business web pages, internal office systems, and industry-specific network pages, automatically completing page content crawling, structured parsing, noise removal and cleaning, and incremental archiving, achieving seamless and continuous knowledge collection from intranets and closed business platforms; For local desktop OCR intelligent agents, it can transform scattered offline materials such as local paper documents, scanned copies, offline PDFs, official policy documents, and offline filing materials into standardized and readable documents through layout analysis, table recognition, text extraction, format restoration, and field structure extraction. The knowledge storage unit; the AnySearch tool targets various heterogeneous storage sources such as enterprise internal cloud drives, shared folders, confidential business databases, and hierarchical document warehouses, enabling cross-path, multi-format, batch intelligent retrieval and targeted crawling, automatically identifying and classifying valid policy documents, industry rules, and internal regulations; the three types of collection units work together and complement each other, specifically crawling enterprise-specific regulations, management systems, industry rules, and business specifications that cannot be crawled by general search engines, are not publicly disclosed, and complete automatic collection, intelligent classification, tagging, hierarchical archiving, and unification. Access management involves building a closed, dedicated private knowledge base; this hybrid indexing system is adaptable to a general knowledge management framework for various vertical compliance scenarios, such as cooperation compliance review knowledge bases, bidding compliance review knowledge bases, and public canteen procurement compliance review knowledge bases; the knowledge base comprises one or more combinations of document libraries, vector libraries, full-text index libraries, knowledge graphs, rule libraries, structured databases, question-answer pair libraries, and business datasets; metadata includes document name, clause number, page number, medical insurance code, region, insurance type, effective date, version number, and citation number; the adaptive segmentation strategy includes at least: First, parent-child segmentation. The system segments according to a hierarchical structure of "document-chapter-clause-paragraph-sentence". The parent block retains the complete policy context, while the child block is used for precise retrieval and citation. The parent block ensures the integrity of the clause context, while the child block improves the accuracy of vector recall.

[0030] Second, fixed-length segmentation. For descriptive text with strong continuity and unclear structure, the system segments it according to a preset character length or token length and sets overlapping windows to avoid semantic breaks.

[0031] Third, row-based or table-based segmentation. For tabular data such as the medical insurance drug catalog, treatment item catalog, consumable catalog, and price catalog, the system segments the data according to the relationships between table rows, fields, and cells, so that each row of catalog data forms an independent searchable unit.

[0032] Fourth, semantic segmentation. The system segments according to the semantic boundaries of the policy, keeping the applicable objects, applicable conditions, reimbursement ratios, payment categories, limited payment scope, and exceptions within the same semantic unit.

[0033] Fifth, structured segmentation. For medical insurance catalogs or business tables with fixed fields, the system extracts fields such as medical insurance code, item name, generic name, product name, specifications, payment category, payment scope, regional version, effective date, and expiration date to form structured knowledge units.

[0034] Sixth, image or scanned document compensation segmentation. For scanned documents, prescription images, expense list images, and other materials, the system first performs text recognition, table recognition, and layout structure recognition, and then segments them according to the recognized row and column relationships, item details, and field meanings. After segmentation, standardized knowledge fragments are generated. After metadata binding is completed, a hybrid index is built. The hybrid index corresponds to a vector library (storing semantic vectors), a full-text index library (storing keywords, clauses, and codes), a structured database (storing medical insurance catalogs and payment rules), and a knowledge graph (storing entities, attributes, and relationships), ultimately generating searchable, referable, and reasonable knowledge units.

[0035] Seventh, question-and-answer pair segmentation. For historical question-and-answer data, customer service knowledge base, and policy question-and-answer materials, the system can segment them according to the format of "question-answer-reference basis-applicable conditions," so that question-and-answer pairs can be directly used to recall similar questions.

[0036] Eighth, case segmentation. For historical audit cases, expense dispute cases, and reimbursement cases, the system can segment them according to the structure of "case facts - involved projects - applicable rules - audit conclusion - risk reasons" for subsequent matching of similar cases and risk warnings.

[0037] S5: Generates multiple query variants based on user questions and executes multi-channel hybrid retrieval, including original query, standardized query, rewritten query, keyword query, and graph entity query. Before implementing multi-dimensional comprehensive scoring after obtaining the candidate knowledge set through multi-channel retrieval, the system first performs a check on the completeness and recognition confidence of the local knowledge base. If the check determines that the item data is missing, key fields are incomplete, or the image and text recognition confidence is lower than the preset threshold, it directly outputs data supplementation prompts or manual review suggestions to guide users to supplement missing content such as diagnosis, insurance location, medical institution information, and clear document images. Such items will not enter the subsequent evidence scoring and two-branch response process. Different query variants can enter vector retrieval, full-text retrieval, structured query, and graph query channels respectively to improve recall coverage. The process involves deduplication, fusion, and multi-dimensional scoring of candidate knowledge. After deduplication and rearrangement, the system sequentially verifies the applicability of evidence and determines the timeliness of policy versions, eliminating invalid evidence that is outdated, outside the applicable region, or does not meet the eligibility criteria. Then, it combines multi-dimensional indicators to perform quantitative calculations of the confidence level of individual and overall answers, determining whether valid evidence exists to support the response. This includes the following steps: Multi-channel hybrid retrieval includes vector retrieval, keyword retrieval, full-text retrieval, structured query, rule matching, and knowledge graph multi-hop query. For example, if a user inputs "Is this headache medication covered by medical insurance?", the system can identify "headache" as a symptom entity, identify the drug entity in the prescription image, rewrite the question as "Query the medical insurance catalog payment categories and limited payment scope of prescription drugs related to headaches," and generate search conditions such as drug name, generic name, medical insurance code, symptoms, diagnosis, region, and eligibility type.

[0038] A comprehensive score is calculated by combining multiple scoring factors, including semantic similarity, keyword matching degree, knowledge source level, update time, document version, applicable conditions, medical insurance code matching degree, graph path length, and contextual consistency, to determine whether there is valid evidence to support the answer. The large model generates answers following three constraints: First, the generated content must not deviate from the five sources of retrieved evidence; second, the answer text must be linked to the corresponding original citation source; and third, a strict distinction must be made between definitive conclusions and risk warnings, and definitive judgments are prohibited from being output without sufficient evidence.

[0039] If the evidence in the knowledge base is deemed sufficient, the system enters the knowledge base trusted answer mode. In this mode, an answer is generated based on the knowledge base evidence, and the document name, clause, version, update time, and citation number are simultaneously annotated before the trusted answer is output, ensuring that the user can clearly know which part of the current knowledge base the answer comes from.

[0040] If the evidence in the knowledge base is deemed insufficient, no definitive conclusion is generated, and the system enters a fallback mode of online search. In this mode, external trusted sources or public network searches are invoked to generate an online reference answer. After generating the online reference answer, the system verifies the authority, consistency, and timeliness of the online results. If the online results are free from uncertainty, source conflicts, or timeliness risks, they are clearly labeled as "Online Search Answer / External Reference Answer" and output as the reference answer. If the online results have unclear sources, conflicting conclusions from different sources, are too old, lack timeliness, or are inconsistent with the current knowledge base evidence, they are clearly labeled as "Online Search Answer / External Reference Answer," and a risk explanation is output, prompting for manual confirmation. This mechanism enables the system to accurately answer and cite sources when an answer exists in the knowledge base, perform an online search and clearly explain the source of the answer when no answer exists in the knowledge base, and prompt for manual confirmation when online results have uncertainty, source conflicts, or timeliness risks.

[0041] Furthermore, a quantitative evaluation mechanism for evidence confidence is introduced. Based on multi-dimensional scoring factors, a confidence score between 0 and 1 is generated for each recalled candidate knowledge through weighted calculation, allowing the weights of different dimensions to be dynamically configured according to domain characteristics. For example, in the medical field, the weight of source level and version validity is much higher than that of semantic similarity; the opposite is true in the creative generation scenario. While calculating the confidence of a single piece of evidence, an overall consistency analysis is performed on the top-K recalled pieces of evidence. If multiple pieces of high-confidence evidence corroborate each other, the confidence of the overall answer is further improved. If there is a conflict between the evidence (e.g., different versions of policies), the conflict will be marked, and all conflicting evidence and their respective confidence scores will be retained, rather than being forcibly merged.

[0042] The beneficial effects achieved by the above content are as follows: Through the above method, the system determines whether there is sufficient evidence in the current knowledge base to support the answer, and outputs a comprehensive confidence score at the same time. This score will be used to determine whether to trigger a fallback online search, for example, when the A_score is lower than a preset threshold. It will also be used to show the credibility of the answer to the user in a visual way, such as a progress bar, star rating, and color coding, when outputting multimodal answers.

[0043] The system performs task parsing on users' natural language questions to identify whether the current question belongs to the category of statistical calculation, trend analysis, conditional filtering, rule judgment, or comprehensive reasoning.

[0044] If the problem pertains to statistical analysis or logical reasoning, the user's natural language question is converted into a unified query plan. A complete unified query plan includes six key components: statistical object, filtering conditions, time range, classification dimension, reasoning objective, and referencing constraints. The matching diagram plan constitutes the fields. The unified query plan includes statistical object, filtering conditions, time range, classification dimension, aggregation method, sorting rules, reasoning objective, and referencing constraints. If the problem pertains to structured knowledge, the unified query plan is converted into a structured query statement. For graph-based knowledge, the unified query plan is converted into a graph query statement. If the problem pertains to document-based knowledge, the unified query plan is converted into semantic retrieval conditions. The system includes keyword search criteria or fragment filtering criteria; based on a unified query plan, it identifies statistically significant objects, statistical fields, time ranges, classification dimensions, and filtering conditions from the knowledge base, and performs calculations and analyses on structured data, semi-structured data, document fragments, question-and-answer records, or extracted knowledge facts in the knowledge base to obtain statistical results; the statistical results include quantity, proportion, mean, maximum value, minimum value, ranking, trend, distribution, cross-comparison, and abnormal changes; after summarizing the data returned from multiple knowledge bases, it performs result fusion and inference calculations, sequentially executing six types of operation steps: aggregation statistics, condition matching, graph path analysis, rule judgment, conflict detection, and conclusion summarization.

[0045] If the problem is a reasoning problem, logical judgments are made based on rules, clauses, knowledge graph relationships, time conditions, applicable objects, contextual evidence, and business constraints in the knowledge base to generate a reasoning conclusion. The reasoning process includes condition matching, rule comparison, judgment of hierarchical concepts, judgment of time validity, judgment of applicable objects, conflict detection, and conclusion summarization. When outputting statistical analysis results or reasoning conclusions, the knowledge sources, rule basis, statistical scope, and intermediate calculation results involved in the calculation or reasoning are also marked. If the data required for statistics or reasoning is incomplete in the current knowledge base, it is indicated that the current conclusion is generated only based on the existing knowledge base content, and the missing data range or knowledge content that needs to be supplemented is explained.

[0046] The beneficial effects achieved by the above content are as follows: Through the above methods, the system can not only answer questions about single pieces of knowledge already existing in the knowledge base, but also perform statistical analysis, trend judgment, comparative analysis, and logical reasoning based on the current knowledge base, thereby enabling the knowledge base to have auxiliary analysis and decision-making capabilities.

[0047] S6: Based on the identified entities, perform one-hop, two-hop, or multi-hop relationship queries in the knowledge graph to form entity relationship paths, related evidence, and reasoning basis; and integrate the graph reasoning results with the retrieved evidence to provide related support for subsequent compliance analysis, reimbursement judgment, and risk identification; S7: For vertical scenarios involving internal corporate rules, such as contract compliance review, bidding compliance review, and public canteen procurement compliance, the system prioritizes retrieving and using the corresponding management systems, compliance clauses, process specifications, constraint details, and industry internal control requirements from the internal rules knowledge base as the core basis for judgment when conducting intelligent question answering and compliance assessment. This avoids interference from general online materials and external non-authoritative information, ensuring that the compliance review conclusions strictly follow the company's established internal system standards.

[0048] The system uses prescription images, medication lists, expense lists, payment receipts, examination reports, and discharge settlement statements as the original data uploaded by users. It then performs image preprocessing, including noise reduction, skew correction, sharpness detection, and region localization. Next, it conducts multimodal recognition, including OCR text recognition, table recognition, layout analysis, and field extraction. After recognition, a structured item list is generated, containing fields such as drug name, treatment item, consumable name, specifications, quantity, unit price, amount, diagnostic information, and candidate medical insurance code. Finally, it performs name standardization and code matching, with standardization including generic name unification, trade name mapping, abbreviation correction, unit conversion, medical insurance code matching, and regional version matching.

[0049] Simultaneously, structured recognition is performed on prescriptions, medication lists, expense lists, invoices, examination reports, and test reports (images or documents) to extract key information such as drugs, items, amounts, medical insurance codes, diagnoses, and medical institutions. A bidirectional mapping relationship between drug entities and diagnostic entities is constructed within a knowledge graph. Diagnostic information constrains the scope of applicable drugs, filtering out off-label use. Furthermore, the diagnostic information is used to reverse-verify the reported diagnostic content, achieving bidirectional cross-verification and assisting in the assessment of compliance risks related to limited payment, excessive medical treatment, and unreasonable prescribing. The four types of search results are then aggregated and fed into the medical insurance compliance analysis engine for subsequent assessments of reimbursement feasibility, limited payment conditions, and the risk of duplicate charges. The system assesses risks of prescription over-prescription and excessive medical treatment. When the engine determines sufficient evidence, it outputs definitive or conditional conclusions such as reimbursement eligibility, non-reimbursement eligibility, partial reimbursement eligibility, or eligibility subject to certain conditions. If the evidence is insufficient, it provides risk warnings and review suggestions, including the need for supplementary diagnosis, supplementary insurance region information, uploading clear images, and manual review. Finally, it integrates the identification results, catalog matching results, reimbursement judgment, risk warnings, policy basis, graph relationship paths, and cited sources to form a standardized analysis report. The standardized drug, treatment, consumable, and chargeable items are then simultaneously input into a vector library, full-text index library, structured medical insurance catalog library, and medical insurance policy knowledge graph for matching. 1. Match similar policy clauses, reimbursement rules, and historical Q&A using a vector library; 2. Match drug names, project names, medical insurance codes, and clause keywords using a full-text index database; 3. Accurately match the medical insurance drug catalog, treatment item catalog, consumable catalog, and payment rules through a structured catalog database; 4. Use knowledge graphs to query the relationships between drugs, treatments, diagnoses, symptoms, payment categories, restrictions, and risk rules.

[0050] The system performs medical insurance compliance analysis based on the matching results. This medical insurance compliance analysis includes at least: First, the system assesses the feasibility of reimbursement. It determines whether the medicine, treatment item, or consumable is covered by medical insurance, whether it falls under Category A, Category B, or is a self-paid item, and whether it is subject to restrictions based on region, type of insurance, hospital level, or payment limitations.

[0051] Second, the system assesses the risk of over-medicalization. Based on the relationship between diagnosis, symptoms, medications, examinations, treatments, and policies / rules, the system determines whether there are risks such as mismatches with the diagnosis, excessive procedures, abnormal frequency of examinations, abnormal medication courses, or overlapping use of similar drugs.

[0052] Third, the system assesses the risk of excessively high prescription costs. It compares the unit price, quantity, specifications, and total price of drugs or medications with prices listed in the medical insurance catalog, historical prices, payment limits, or local policies and rules to identify items that may have abnormal prices, excessively high costs, or exceed payment standards.

[0053] Fourth, the system assesses the risk of duplicate charges. It identifies items with the same medical insurance code, the same item name, similar items, the same type of medical treatment, or items that appear repeatedly within the same time period in the expense list to determine if there is a risk of duplicate billing, splitting charges, or overlapping charges for similar items.

[0054] Fifth, the system determines whether the drug or item in the medical insurance catalog has a limited scope of payment. The system considers factors such as diagnosis, indication, medical institution level, insurance type, and regional policies to determine if the drug meets the payment conditions.

[0055] Sixth, missing material assessment. If key fields such as diagnosis, region, insurance type, hospital level, medical insurance code, quantity, and unit price are missing from images or documents, the system will indicate that the current conclusion has limitations and explain the materials that need to be supplemented.

[0056] When outputting analysis results, the system should distinguish between definitive conclusions and risk warnings. For content directly supported by the medical insurance catalog, policy provisions, or structured rules, output conclusions such as "reimbursable," "not reimbursable," "partially reimbursable," or "requires certain conditions," along with the cited references. For content requiring manual review or with insufficient materials, output warnings such as "risk exists," "supplementary diagnostic information required," or "manual review required," avoiding treating model predictions as definitive conclusions.

[0057] Through this mechanism, the system can transform user-uploaded prescriptions, medication lists, and expense lists into searchable, matchable, and reasonable structured data, and combine it with medical insurance reimbursement catalogs, knowledge graphs, and large language models to complete reimbursement judgments, cost rationality assessments, and risk analyses.

[0058] S8: Automatically adapts to multimodal output formats such as text, tables, charts, reference cards, and analysis reports based on question type and answer content. Output formats include text descriptions, point-by-point answers, tables, process steps, comparison tables, statistical charts, trend charts, structured JSON, reference cards, report documents, or voice-over text. For example: for process-related questions, the system outputs a step-by-step process; for comparison-related questions, the system outputs a comparison table; for statistical questions, the system outputs a statistical table or chart; for knowledge base Q&A, the system outputs the answer text and cited evidence; for fallback answers based on online searches, the system outputs an online search identifier and external source hints. For example, outputting according to the following structure: 1. Identification Results: Lists the drugs, items, quantities, unit prices, amounts, and medical insurance code candidates identified from prescriptions, medication lists, or expense lists; 2. Matching Results: Lists the medical insurance catalog, payment category, limited payment scope, and policy basis for each item; 3. Analysis Conclusion: Indicate whether the expense is reimbursable, not reimbursable, partially reimbursable, requires certain conditions to be met, or cannot be determined; 4. Risk Warning: Explain whether there are risks such as duplicate charges, abnormal fees, excessive medical treatment, or missing data; 5. Citation Basis: Include policy documents, catalog documents, clause numbers, version numbers, and update times; 6. Graph Relationship Path: Displays the relationship links between drugs and diagnostics, and between fee items and payment rules; 7. Review suggestions: Provide prompts for low-confidence identifications, missing data, or content requiring manual judgment.

[0059] The final output fully covers six modules: statistical results, inference conclusions, relationship paths, policy basis, risk warnings, and explanations of missing data, and corresponds one-to-one with the output fields of the flowchart.

[0060] S9: The system receives a long answer generation task; for long text answers, it first generates an outline, which includes chapter titles, the answer objective for each chapter, required citations, expected output format, and completion status; it tracks the generation progress through a state machine, automatically resuming writing at breakpoints when truncation or omissions occur, and completing integrity verification; including the following steps: During segmented generation, a global generation state machine or structured state queue is maintained to record the generation status of each paragraph, table, chart, citation, conclusion, and supplementary explanation. For long text answers, an answer outline including chapter titles, answer objectives for each chapter, required citations, and expected output format is generated first. The generation state queue including six states—not started, generating, completed, pending completion, pending verification, and abnormal interruption—is initialized. After generating the answer content in segments and binding the citations, a single-segment integrity check is performed. The single-segment integrity check verifies each item from five dimensions: whether the paragraph is missing, whether the table is complete, whether the citation is missing, whether the conclusion is missing, and whether the format is abnormal.

[0061] If the verification result indicates a missing or interrupted section, a continuation instruction is automatically generated based on the breakpoint position recorded in the global state machine or structured state queue. The breakpoint position includes the paragraph number that failed the single-segment integrity check, the corresponding outline node, the missing type, the end of the generated content, the bound references, and the output format to be completed. The system generates a continuation instruction based on the breakpoint position, re-entering the completed content, the current breakpoint, the remaining outline, and reference constraints into the generation module. After the continuation content is generated, the system writes the completed text back to the corresponding outline node and re-enters the single-segment integrity check, repeatedly verifying for missing paragraphs, table integrity, missing references, missing conclusions, and formatting anomalies, until the node's status changes from "to be completed" or "abnormal interruption" to "completed." The completed content, current breakpoint, remaining outline, and citation constraints are then re-entered into the generation module, allowing the system to continue generation from the breakpoint instead of regenerating the entire answer. After the continuation is complete, the system returns to the single-segment integrity check step for repeated verification. Once all segments are generated, the system performs a final integrity check to confirm that all chapters in the answer outline are complete, all citations are bound, all tables or charts are output completely, and the conclusion is not missing. If the final check still finds missing content, the completion process is triggered again. If there are no missing or interrupted parts, the final integrity check is performed, and the complete long answer is output after the check passes.

[0062] The beneficial effects achieved by the above content are as follows: Through this mechanism, the system can reduce common problems in long answers, such as mid-way truncation, paragraph omission, incomplete tables, disordered citations, and missing conclusions. It is especially suitable for long text question-and-answer scenarios such as technical solution descriptions, policy explanations, system descriptions, statistical analysis reports, and knowledge base analysis reports.

[0063] S10: During operation, eight categories of candidate knowledge sources are collected: user feedback, manually corrected content, low-confidence issues, high-frequency online search issues, newly added policy documents, updated medical insurance catalog documents, prescription recognition correction results, and cost list review results. These, along with user satisfaction Q&A pairs and newly added business data, are added to the candidate knowledge pool. Candidate knowledge undergoes multiple reviews, including source verification, timeliness verification, duplication verification, conflict verification, security verification, field integrity verification, and manual review. Review methods include rule-based review, manual review, multi-model cross-review, or a combination thereof. If candidate knowledge fails review, it is returned for modification, discarded, or manually reviewed, and the corresponding entry is directly terminated from the database entry process. Candidate knowledge that passes review enters the knowledge database processing stage, sequentially completing document parsing and adaptive processing. The system employs five preprocessing steps: knowledge segmentation, metadata binding, entity relationship extraction, and citation number generation. After preprocessing, it synchronously updates six major knowledge repositories: document repository, vector index repository, full-text index repository, structured database, knowledge graph, and rule base. Once candidate knowledge passes review, a new knowledge version is generated, and related indexes, citation numbers, and metadata are updated simultaneously. The system retains historical versions for easy tracing, rollback, and comparison, and builds a new knowledge index in the background to ensure no impact on the current online Q&A service. After the new version is built and validated, if the validation is successful, a hot update switches to the new version, and the online Q&A service loads the new knowledge base. If the new version is faulty, the system can roll back to the previous stable version. Through this mechanism, the system achieves continuous learning, controllable updates, and uninterrupted service for the knowledge base.

[0064] It continuously collects questions deemed insufficient in the knowledge base and triggering online fallback, as well as questions with evidence in the knowledge base but low confidence in the final answer. It periodically performs clustering and structured analysis on these questions to achieve continuous diagnosis and proactive enhancement of the knowledge base; specifically including: Each question-and-answer interaction is recorded in a structured log. When an interaction triggers an online search as a fallback due to insufficient evidence in the knowledge base, or when the confidence of the final comprehensive answer generated is lower than a preset threshold, the question and its related context will be marked and stored in the gap analysis pool. Newly added questions in the gap analysis pool are processed in batches according to a preset cycle. Each question is converted into a high-dimensional vector using an embedding model, and semantically similar questions are aggregated into the same question cluster using a density-based clustering algorithm. The large language model analyzer is called to perform multi-dimensional inductive analysis on the questions within each question cluster. The analysis dimensions include the topic classification involved in the cluster, the type of missing knowledge (factual, process, or rule judgment), frequently occurring entity names or relationships, and frequently occurring constraints. The constraints specifically include regional constraints, time or version constraints, and applicable object or identity constraints. The current knowledge base is then re-searched based on the representative questions of the cluster to quantify the coverage and severity of the gap in the current knowledge base for that knowledge point.

[0065] Based on the above analysis results, a structured knowledge gap entry is automatically generated for each question cluster. Multiple entries are summarized to form a complete knowledge gap report. The report is presented in a structured form such as tables or key-value pairs, clearly indicating which areas and types of knowledge in the current knowledge base have systematic deficiencies, and includes fields such as representative user questions, suggested search keywords, and the severity of the gap.

[0066] Configure two complementary action paths for knowledge gap reports: The first path is decision support for knowledge base administrators, presenting the report to administrators through a visual interface, supporting filtering and sorting by topic and severity, and automatically generating a knowledge supplementation task for each gap for administrators to assign or handle directly.

[0067] The second path is automated proactive retrieval. For gaps marked as high severity and belonging to the factual or rule-based categories, multiple structured search queries are generated based on suggested search keywords and constraints. These queries are then targeted at pre-configured trusted external knowledge sources, and the retrieved original documents or fragments are directly written into the candidate knowledge pool, along with an automatic retrieval identifier and the corresponding gap number.

[0068] The automatically retrieved results and the results from user feedback and manual corrections written into the candidate knowledge pool all undergo multiple review processes, including source verification, timeliness verification, conflict detection, and logical consistency checks. In the review interface, the automatically retrieved items are marked as pending review and the corresponding knowledge gap report is displayed to assist reviewers in making quick judgments. Only candidate knowledge that passes the review is allowed to enter the version update process and is finally loaded into the official knowledge base through a hot update mechanism, thus filling the previously identified knowledge gaps in a closed loop.

[0069] The beneficial effects achieved by the above content are as follows: Through the above mechanism, the evolution from passive response to active learning is realized, forming a complete knowledge evolution closed loop in which the question-answering service generates log records, the log records drive gap analysis, the gap analysis triggers active retrieval, the retrieval results are reviewed and stored in the database, and the subsequent services are enhanced after storage.

[0070] Working principle: By uniformly parsing the user's multimodal input questions, completing the context and identifying the intent, it then retrieves knowledge from the knowledge base and scores the evidence to determine its sufficiency. Based on the judgment result, it generates an answer according to the knowledge base's credible answer or online fallback. For statistical reasoning questions, it automatically converts them into query plans to complete calculations and judgments, outputs the corresponding format according to the type, and ensures the completeness of long texts through outlines, state machines, and breakpoint continuation. It also collects candidate knowledge, reviews it, and updates the knowledge base in a versioned manner, forming a fully traceable closed loop for intelligent question answering in the vertical domain.

[0071] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, or article that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, or article.

[0072] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. An engineering method for intelligent question answering in vertical domains based on multimodal knowledge bases, characterized in that: Includes the following steps: S1: Receive multimodal questions input by users, transcribe speech into text, extract text and structure from images and documents, identify fields and relationships in tables, and unify input questions from different modalities into standardized question representations; S2: Read the content of historical dialogues in multi-turn dialogues, and filter, compress and resolve the historical context to filter key information in historical dialogues and form searchable questions; S3: Perform intent recognition on user questions and classify questions into knowledge query, rule explanation, statistical analysis, logical reasoning, image understanding, table analysis, prescription analysis, expense list analysis, reimbursement judgment, online search and knowledge base update suggestions; Based on the identification results, the problems are routed to different processing paths; S4: Perform adaptive segmentation of knowledge documents by type, bind metadata, and build a hybrid index that integrates vector library, full-text index, structured library, and knowledge graph; and collect, classify and reach non-public information such as internal regulations, corporate systems and industry rules through browser plugin intelligent agents, local desktop OCR intelligent agents and AnySearch tools, build a dedicated knowledge base that search engines cannot access, and form a collaborative collection architecture; S5: Generate multiple query variations based on user questions and perform multi-channel hybrid retrieval, deduplicatize, merge and multi-dimensionally score candidate knowledge, and determine whether there is valid evidence to support the answer; S6: Based on the identified entities, perform one-hop, two-hop, or multi-hop relationship queries in the knowledge graph to form entity relationship paths, related evidence, and reasoning basis; and integrate the graph reasoning results with the retrieved evidence to provide related support for subsequent compliance analysis, reimbursement judgment, and risk identification; S7: For scenarios involving contract compliance review, bidding compliance review, and public canteen procurement compliance review, relevant clauses from the internal rules knowledge base will be prioritized as the core basis; structured identification will be performed on images or documents such as prescriptions, medication lists, expense lists, invoices, examination reports, and test reports to extract key information such as drugs, items, amounts, medical insurance codes, diagnoses, and medical institutions; combining the medical insurance catalog, policy clauses, business rules, and knowledge graph relationships, the feasibility of reimbursement, the reasonableness of expenses, and medical risk analysis will be completed, and definitive conclusions, risk warnings, and cited evidence will be output; S8: Automatically adapts to multimodal output formats such as text, tables, charts, reference cards, and analysis reports based on question type and answer content.

2. The engineering method for intelligent question answering in vertical domains based on multimodal knowledge bases according to claim 1, characterized in that, It also includes the following steps: S9: For long text responses, first generate an outline. The response outline includes chapter titles, the response objectives for each chapter, the required references, the expected output format, and the completion status. The generation progress is tracked by a state machine, and the system automatically resumes writing at breakpoints when truncation or omission occurs, thus completing the integrity verification.

3. The engineering method for intelligent question answering in vertical domains based on multimodal knowledge bases according to claim 2, characterized in that, It also includes the following steps: S10: During operation, user feedback, manually corrected content, low-confidence issues, high-frequency online search issues, user satisfaction Q&A pairs, and newly added business data are collected and entered into the candidate knowledge pool. The candidate knowledge is subject to multiple reviews, and the candidate knowledge that passes the review is entered into the formal knowledge base. Once the candidate knowledge passes the review, a new knowledge version is generated, and the relevant indexes, citation numbers, and metadata are updated synchronously.

4. The engineering method for intelligent question answering in vertical domains based on multimodal knowledge bases according to claim 1, characterized in that, In S5, determining whether there is valid evidence to support the answer includes the following steps: If the evidence in the knowledge base is deemed sufficient, the system enters the knowledge base credible answer mode. In the knowledge base credible answer mode, an answer is generated based on the knowledge base evidence, and the document name, clause, version, update time, and citation number are simultaneously annotated before the credible answer is output. If the evidence in the knowledge base is deemed insufficient, no definitive conclusion is generated, and the system enters the fallback mode of online search. In the fallback mode of online search, external trusted sources or public network searches are called to generate online reference answers, clearly indicating the online search answer / external reference answer, outputting the reference answer and prompting that manual confirmation is required.

5. The engineering method for intelligent question answering in vertical domains based on multimodal knowledge bases according to claim 4, characterized in that, In S5, determining whether there is valid evidence to support the answer also includes the following steps: Perform task parsing on users' natural language questions to identify whether the current question belongs to the category of statistical calculation, trend analysis, condition filtering, rule judgment, or comprehensive reasoning. If the question pertains to statistical analysis or logical reasoning, the user's natural language question will be converted into a unified query plan. If the knowledge is structured, the unified query plan will be converted into a structured query statement; if the knowledge is graph-based, the unified query plan will be converted into a graph query statement. If the knowledge belongs to document type, the unified query plan will be converted into semantic search conditions, keyword search conditions, or fragment filtering conditions. Based on the unified query plan, statistical objects, statistical fields, time ranges, classification dimensions, and filtering conditions are identified from the knowledge base. The structured data, semi-structured data, document fragments, question and answer records, or extracted knowledge facts in the knowledge base are then calculated and analyzed to obtain statistical results.

6. The engineering method for intelligent question answering in vertical domains based on multimodal knowledge bases according to claim 5, characterized in that, In S5, determining whether there is valid evidence to support the answer also includes the following steps: If it is a reasoning problem, logical judgment is made based on the rules, clauses, knowledge graph relationships, time conditions, applicable objects, contextual evidence and business constraints in the knowledge base to generate a reasoning conclusion; When outputting statistical analysis results or inference conclusions, the knowledge sources, rule bases, statistical scope, and intermediate calculation results involved in the calculation or inference should be marked simultaneously. If the data required for statistics or reasoning is incomplete in the current knowledge base, a message will be displayed indicating that the current conclusion is generated only based on the existing knowledge base content, and the scope of missing data or the knowledge content that needs to be supplemented will be explained.

7. The engineering method for intelligent question answering in vertical domains based on multimodal knowledge bases according to claim 2, characterized in that, In S9, the generation progress is tracked through a state machine, including the following steps: When generating segments, maintain a global generation state machine or structured state queue to record the generation state of each segment, table, chart, reference, conclusion, and supplementary explanation. For long text responses, first generate a response outline including chapter titles, the target of each chapter, the required references, and the expected output format, and initialize a generation status queue including six states: not started, generating, completed, to be completed, to be verified, and abnormal interruption. After generating the answer content in segments and binding the references, a segment integrity check is performed. The segment integrity check verifies each item from five dimensions: whether the paragraph is missing, whether the table is complete, whether the reference is missing, whether the conclusion is missing, and whether the format is abnormal. If the verification result is found to be missing or interrupted, a continuation instruction will be automatically generated based on the breakpoint position recorded in the globally generated state machine or structured state queue. Once all segments are generated, the system performs a final integrity check. If the final verification still finds missing content, the completion process will be triggered again. If there are no missing or interrupted parts, proceed to the final integrity check. If the check passes, output the complete long answer.

8. The engineering method for intelligent question answering in vertical domains based on multimodal knowledge bases according to claim 1, characterized in that, The knowledge base is one or more combinations of document library, vector library, full-text index library, knowledge graph, rule library, structured database, question-answer pair library, and business dataset.

Images