Policy question and answer system, method and medium based on multi-model cooperation and atlas enhancement

Abstract

The application relates to the technical field of knowledge graph, in particular to a policy question and answer system and method based on multi-model cooperation and graph enhancement and a medium, the method comprising automatically generating a policy knowledge ontology structure, configuring a multi-model cooperative extraction engine, performing entity recognition and relationship extraction tasks on a standard policy corpus, and cross-verifying and aligning the extraction results of multiple large language models through a preset conflict resolution strategy to obtain a structured policy knowledge triple set; based on the policy knowledge ontology structure, mapping the structured policy knowledge triple set to a graph database to construct a dynamic policy knowledge graph. The application configures a multi-model cooperative extraction engine and cross-verifies and aligns the extraction results of multiple large language models through a conflict resolution strategy, thereby improving the accuracy of the structured policy knowledge triple and effectively correcting the semantic illusion or extraction deviation that may exist in a single model.

Description

Technical Field

[0001] This application relates to the field of knowledge graph technology, and in particular to a policy question answering system, method and medium based on multi-model collaboration and graph enhancement. Background Technology

[0002] In the current development of e-government, using artificial intelligence technology to provide users with accurate policy consultation services has become an important direction.

[0003] However, existing technologies still have limitations in practical applications. Policy texts are usually highly professional, logically rigorous, and have complex hierarchical nesting relationships. Traditional knowledge extraction methods often suffer from low extraction accuracy when faced with such complex unstructured texts.

[0004] Existing research (such as question-answering systems based on the traditional RAG framework and static knowledge graph question-answering methods) typically uses a single large language model for knowledge extraction, which carries the risk of single-point illusion. Existing knowledge graph ontology usually relies on manual pre-setting, making it difficult to adapt to the dynamic changes in the policy field. Some question-answering systems based on vector retrieval lack structured reasoning capabilities and cannot support multi-hop path queries. None of the above methods can simultaneously solve the three major problems of automated extraction, dynamic updating, and interpretable question answering of policy knowledge. Summary of the Invention

[0005] The objective of this application is to provide a policy question answering method based on multi-model collaboration and graph enhancement, comprising: acquiring multi-source unstructured policy text data through a data acquisition terminal and preprocessing the multi-source unstructured policy text data to construct a standard policy corpus; on a server configured with a GPU cluster, calling a large language model with no less than 30 billion parameters through an inference API, and performing a global scan of the standard policy corpus based on the semantic understanding capability of the large language model, automatically generating a policy knowledge ontology structure including policy entity types, attribute dimensions, and relational constraints by identifying core business terms, administrative level dimensions, and policy association logic in the policy text; configuring a multi-model collaborative extraction engine, using multiple large language models with different parameter scales or fine-tuning directions to perform entity recognition and relation extraction tasks on the standard policy corpus, and performing cross-validation and consistency alignment of the extraction results of multiple large language models through a preset conflict resolution strategy, wherein the conflict resolution strategy introduces an uncertainty estimation mechanism to quantify the confidence of the model output, and adopts adaptive integration to dynamically adjust the participating models according to the query difficulty. The system calculates the number of policy knowledge triples, obtains a set of structured policy knowledge triples, maps the set of structured policy knowledge triples to a graph database, and constructs a dynamic policy knowledge graph. Responding to user-inputted policy consultation commands, the system extracts semantic features and query intent from the policy consultation commands using a large language model. Based on these semantic features, it employs knowledge graph augmented retrieval technology to perform multi-hop path retrieval within the dynamic policy knowledge graph to obtain a deterministic knowledge subgraph related to the policy consultation command. The multi-hop path retrieval process includes: before performing vector retrieval, using a large language model to perform graph structure-aware query decomposition on the user query, identifying the entity types and relational path templates required for the query; using relational path templates to constrain the candidate node set by type, reducing invalid candidates; re-sorting the knowledge graph path score and vector similarity score according to learnable weights to return a more accurate knowledge subgraph; based on the structured facts provided by the deterministic knowledge subgraph, combined with the natural language generation capabilities of the large language model, constructing a response text with policy source information and outputting it to the user terminal.

[0006] By adopting the above technical solutions, a large language model is used to perform a global scan of the standard policy corpus and automatically generate a policy knowledge ontology structure. This enables automated modeling of complex knowledge systems in the policy domain, reducing the time for manual ontology construction from several weeks to several hours. This solves the technical bottleneck of low efficiency and difficulty in adapting to complex policy logic caused by reliance on manual labor. A multi-model collaborative extraction engine is configured, and a conflict resolution strategy is used to cross-validate and align the extraction results of multiple large language models, improving the accuracy of structured policy knowledge triples and effectively correcting semantic illusions or extraction biases that may exist in a single model. The set of structured policy knowledge triples is mapped to a graph database to construct a dynamic policy knowledge graph. Combined with the large language model, multi-hop path retrieval and natural language generation are performed, improving the accuracy of policy question answering. At the same time, the response generation time is reduced from minutes in traditional retrieval systems to seconds.

[0007] Optionally, the step of preprocessing the multi-source unstructured policy text data further includes a processing procedure for extremely long policy documents. This processing procedure includes: using an overlapping sliding window mechanism to semantically segment the extremely long policy document, ensuring that adjacent segments retain a preset proportion of overlapping content to maintain contextual coherence; extracting summaries from each semantic segment based on a large language model, and associating and storing the summary information with the original segmented text; wherein the summary information is used to provide global semantic guidance during subsequent knowledge extraction to avoid the loss of policy elements due to text length limitations.

[0008] By adopting the above technical solution, an overlapping sliding window mechanism is used to semantically segment ultra-long policy documents and retain a preset proportion of overlapping content. This enables long documents exceeding 100,000 words to be effectively processed without losing contextual coherence, avoiding the problem of losing key policy elements caused by traditional segmentation methods. Based on a large language model, a summary is extracted from each semantic segment and the summary information is associated with and stored with the original segmented text. This allows the subsequent knowledge extraction process to obtain global semantic guidance, improving the completeness of knowledge extraction from ultra-long documents.

[0009] Optionally, the step of cross-validating and aligning the extraction results of multiple large language models using a preset conflict resolution strategy includes: assigning a weight score based on the historical accuracy of each large language model participating in collaborative extraction; statistically analyzing the frequency of occurrence and corresponding weighted scores of extraction results from different large language models for the same policy entity or relationship; triggering a voting mechanism based on a confidence threshold in response to discrepancies in the extraction results among different large language models, and selecting the extraction result with the highest weighted score as the final structured output, wherein the weighted score combines historical accuracy weight and uncertainty estimation weight, and models with lower uncertainty are given higher decision weights; the uncertainty estimation weight is calculated by using semantic entropy or token probability variance as a quantitative indicator of confidence in a single extraction; distinguishing subtasks including entity recognition, relationship extraction, and attribute classification, and maintaining weights for each model on each type of subtask; when the discrepancy in the outputs of each model exceeds a threshold, prioritizing the model with higher accuracy as the referee.

[0010] By adopting the above technical solution, each large language model participating in collaborative extraction is assigned a weight score based on historical accuracy, and the extraction results of different models are weighted and statistically analyzed. This allows important models (i.e., models with high accuracy) to have a greater say in decision-making, improving the scientific nature of conflict resolution. In response to discrepancies in the extraction results among different models, a voting mechanism based on confidence threshold is triggered, selecting the extraction result with the highest weighted score as the final output, thereby improving the consistency alignment accuracy of entity recognition and relation extraction.

[0011] Optionally, the step of constructing a dynamic policy knowledge graph further includes a process for real-time updating of policy knowledge. This process includes: configuring a policy change monitoring plugin to monitor update callback signals from the policy release platform in real time; automatically extracting incremental policy text and triggering the multi-model collaborative extraction engine for real-time processing in response to detected real-time update events such as policy repeal, revision, or new issuance; and logically deleting or correcting attributes of invalid nodes and relationships in the dynamic policy knowledge graph based on the incremental extraction results. The real-time update process ensures low latency between the policy knowledge graph and the physical corpus. Synchronously, the real-time update process employs a streaming computing framework to achieve near real-time knowledge updates, uses incremental indexing technology to avoid full reconstruction, and implements an automatic knowledge conflict detection and resolution mechanism, supporting the tracing of historical knowledge versions. The tracing of historical knowledge versions is achieved by adding timestamp attributes of the start and end times of the validity period to each edge in the knowledge graph, constructing a temporal knowledge graph that supports policy history tracing queries. The automatic knowledge conflict detection and resolution mechanism automatically determines whether it is a full text replacement, partial revision, or parallel validity by semantically comparing the new and old clauses based on a large language model, and updates the relationship annotations in the graph accordingly.

[0012] By adopting the above technical solution, the policy change monitoring plugin is configured to monitor the update callback signals of the policy release platform in real time, enabling policy update events to be perceived in real time. This reduces the update delay of the policy knowledge base from several days in the traditional periodic updates to several hours. When a policy repeal, revision, or new event is detected, the multi-model collaborative extraction engine is automatically triggered for real-time processing, and logical deletion or attribute correction is performed on the dynamic policy knowledge graph. This reduces the synchronization delay between the knowledge graph and the physical corpus, ensuring the timeliness of the question answering system.

[0013] Optionally, the step of constructing a dynamic policy knowledge graph also includes a cross-document policy conflict detection process, which includes: performing cross-document entity constraint detection on new policies with the same applicable objects and the same policy fields; identifying exclusive constraint clauses and establishing explicit mutually exclusive edges in the knowledge graph; and proactively prompting users in the question-and-answer response to the existence of policy conflicts or to declare mutually exclusive restrictions.

[0014] Optionally, the step of constructing a response text with policy source information and outputting it to the user terminal includes: recording the policy source index tags associated with each retrieval path during the multi-hop path retrieval; converting the structured facts in the deterministic knowledge subgraph into logical deduction chains, and using the logical deduction chains as limiting constraints generated by the large language model; in the generated response text, corresponding the deduction conclusions with the policy source index tags through reference annotations; the step of constructing a response text with policy source information and outputting it to the user terminal further includes introducing an interpretability mechanism; wherein, the interpretability mechanism includes: introducing attention. The mechanism visualization showcases key information focused on by the model, progressively revealing the reasoning process to enhance user trust. A confidence scoring system quantifies the reliability of the answers, and interactive explanations allow users to inquire about the details of the reasoning. The step of constructing a response text with policy source information and outputting it to the user's terminal also includes a fact-checking process: extracting key facts containing numbers, dates, and organization names from the generated response; performing precise comparisons with the values ​​of corresponding nodes or attributes in the knowledge graph, including string matching and numerical range verification; automatically triggering regeneration or targeted correction if inconsistencies are found; and marking the verification status of each assertion in the response to enhance the user's judgment of the answer's credibility.

[0015] By adopting the above technical solution, the policy text index tags associated with each retrieval path are recorded during multi-hop path retrieval, so that every factual statement in the final response can be traced back to the specific policy article, realizing the traceability of the response content; the structured facts in the deterministic knowledge subgraph are transformed into logical deduction chains and used as limiting constraints for the generation of the large language model, which improves the logical consistency of the generated response and significantly reduces factual errors.

[0016] Optionally, the step of constructing a response text with policy source information and outputting it to the user terminal further includes an iterative retrieval and self-verification process. The iterative retrieval and self-verification process includes: decomposing complex policy consultation instructions into multiple sub-queries using a large language model; independently performing knowledge graph retrieval on each sub-query and aggregating the results of multiple rounds of retrieval into a structured context; generating a preliminary answer based on the deterministic knowledge subgraph, and then re-verifying the factual consistency of key assertions. If inconsistencies are found, regeneration or targeted correction is triggered; and the verification status of each assertion is marked in the response, including whether it has been verified or not.

[0017] Optionally, the step of performing multi-hop path retrieval in the dynamic policy knowledge graph based on the semantic features includes: calculating the cosine similarity between the semantic vector of the policy consultation instruction and the attributes of each node in the dynamic policy knowledge graph; dynamically delineating a candidate node set based on semantic relevance scores, and determining the retrieval depth based on intent recognition results. The retrieval process adopts an adaptive retrieval strategy, including query rewriting to convert the user's natural language query into a standard retrieval expression, a hybrid retrieval strategy combining keyword retrieval and semantic retrieval, dynamically selecting the optimal retrieval strategy based on query intent recognition, and continuously optimizing the retrieval results based on user feedback; and executing a graph traversal algorithm starting from the candidate node set to extract a highly relevant knowledge subgraph that can cover the user's potential intent.

[0018] By adopting the above technical solution, the cosine similarity between the semantic vector of the policy consultation instruction and the attributes of each node in the knowledge graph is calculated, and the candidate node set is dynamically delineated based on the semantic relevance score, which improves the retrieval accuracy and reduces the size of the candidate node set. The retrieval depth is determined according to the intent recognition result, and a graph traversal algorithm is executed to extract highly relevant knowledge subgraphs, which reduces the response time of complex multi-hop queries from minutes to sub-seconds for full graph traversal, while maintaining a high recall rate.

[0019] Optionally, the step of configuring the multi-model collaborative extraction engine further includes a scheduling process for heterogeneous model computing resources. The scheduling process for heterogeneous model computing resources includes: real-time collection of the memory occupancy rate and computing load status of each computing node in the server cluster; dynamic matching of large language models with corresponding parameter sizes based on the complexity score of the policy text to be processed; and distribution of extraction tasks to the optimal computing node using a load balancing gateway. The scheduling process achieves efficient utilization of model computing resources in multi-concurrency scenarios.

[0020] By adopting the above technical solution, the memory utilization rate and computing load status of each computing node in the server cluster are collected in real time, and a large language model with corresponding parameter scale is dynamically matched based on the policy text complexity score, thereby improving the utilization rate of heterogeneous computing resources. The extraction task is distributed to the optimal computing node by using a load balancing gateway, which improves the system throughput in multi-concurrency scenarios and reduces the average task response time.

[0021] Optionally, the step of generating the policy knowledge ontology structure further includes a semantic disambiguation process for policy terminology. This process includes: obtaining the administrative level metadata and business domain labels to which the multi-source unstructured policy text data belongs; identifying policy terms with polysemous names based on a large language model, and constructing terminology context vectors in conjunction with the administrative level metadata; configuring unique entity identifiers for terms with the same name in different contexts in the dynamic policy knowledge graph; wherein the entity identifiers are used to achieve cross-domain policy knowledge alignment.

[0022] By adopting the above technical solutions, the administrative level metadata and business domain labels of the policy text are obtained, and the homonymous polysemous terms are identified based on the large language model, which improves the disambiguation accuracy of cross-domain policy terms. In the knowledge graph, unique entity identifiers are configured for homonymous terms in different contexts, which improves the alignment accuracy of cross-domain policy knowledge and reduces knowledge fusion errors caused by semantic conflicts.

[0023] Optionally, the system also includes an adaptive degradation process, which includes: triggering a fault-tolerant feedback mechanism when the multi-hop path retrieval fails to obtain a valid knowledge subgraph or the large language model inference response time exceeds a preset threshold; performing keyword retrieval based on the full-text indexing engine of the policy corpus, and feeding back the retrieved policy text summary as a fallback answer to the user; wherein, the system simultaneously records the failed path of this multi-hop path retrieval for subsequent knowledge base completion.

[0024] By adopting the above technical solutions, a fault-tolerant feedback mechanism is triggered when multi-hop path retrieval fails to obtain an effective knowledge subgraph or when the reasoning response time of the large language model exceeds a preset threshold, thereby improving the service availability of the system under extreme abnormal conditions. The full-text indexing engine based on the policy corpus performs keyword retrieval and uses the retrieved policy text summary as a fallback answer, so that even if the knowledge graph retrieval fails, users can still obtain relevant reference information, thereby improving user experience satisfaction.

[0025] The second objective of this application is to provide a policy question-answering system based on multi-model collaboration and graph enhancement, comprising multiple functional modules configured to execute the aforementioned methods. These functional modules include at least: a data preprocessing module for collecting and preprocessing multi-source unstructured policy text data to construct a standard policy corpus; an ontology automatic generation module for automatically generating a policy knowledge ontology structure based on a large language model; a collaborative extraction module for configuring a multi-model collaborative extraction engine, performing entity recognition and relation extraction, and performing consistency alignment to obtain a set of structured policy knowledge triples; and a graph construction module for constructing a policy knowledge corpus based on the aforementioned data. The system comprises a knowledge ontology structure to construct a dynamic policy knowledge graph; an intent retrieval module to extract semantic features and perform multi-hop path retrieval in response to user commands to obtain a deterministic knowledge subgraph; a real-time update module to configure a policy change monitoring plugin to achieve incremental real-time updates of the policy knowledge graph; a semantic disambiguation module to configure unique entity identifiers for policy terms with the same name but multiple meanings; a fault tolerance and degradation module to trigger a full-text index fallback mechanism and record knowledge gaps when graph retrieval fails; and a fusion question-answering module to generate response text with policy source information based on the deterministic knowledge subgraph and the large language model.

[0026] The third objective of this application is to provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the aforementioned policy question-answering method based on multi-model collaboration and graph enhancement.

[0027] Compared with existing technologies, the core innovations of this application include: (1) For the first time, a multi-model collaborative integration strategy was applied to policy knowledge extraction. Through a weighted voting mechanism based on uncertainty estimation, the semantic illusion problem of a single large language model in entity recognition and relation extraction was solved. (2) An adaptive hierarchical retrieval architecture is proposed, which combines knowledge graph augmented retrieval (GraphRAG) technology with traditional multi-hop path retrieval, and significantly improves the response speed of complex queries while ensuring recall. (3) Implement an incremental knowledge graph update mechanism based on streaming computing to support low-latency synchronization of policy knowledge updates from "day-level" to "minute-level"; (4) A complete interpretability framework was constructed, and the credibility and authority of the policy Q&A system were improved through visualization of the logical deduction chain, confidence scoring and traceability labels. Attached Figure Description

[0028] Figure 1 This is a schematic diagram of the policy question answering method based on multi-model collaboration and graph enhancement in this application. The diagram shows the steps of data collection and preprocessing, automatic knowledge ontology generation, multi-model collaborative extraction and consistency alignment, knowledge graph construction and real-time updating, intent understanding and multi-hop path retrieval, and fusion question answering and source tracing output.

[0029] Figure 2 This is a block diagram of the policy question answering system based on multi-model collaboration and graph enhancement in this application. The diagram shows the functions of the data preprocessing module, the ontology automatic generation module, the collaborative extraction module, the graph construction module, the intent retrieval module, and the fusion question answering module, as well as their mutual data flow relationships. Detailed Implementation

[0030] like Figure 1 As shown in the figure, this application discloses a policy question answering method based on multi-model collaboration and graph enhancement, which includes the following steps.

[0031] The method in this embodiment is executed on a server configured with a GPU cluster. The server includes at least eight A100 GPUs (each with at least 80GB of video memory), a CPU cluster (each node with at least 64 cores), at least 512GB of memory, and a distributed file system for storage. Large language models are called via an inference API, including a general-purpose large language model with at least 30 billion parameters, a policy text fine-tuning model with approximately 10 billion parameters, and a lightweight summarization model with approximately 7 billion parameters.

[0032] We acquire multi-source unstructured policy text data through data acquisition terminals and preprocess the multi-source unstructured policy text data to construct a standard policy corpus.

[0033] On a server configured with a GPU cluster, a large language model with no less than 30 billion parameters is invoked through the inference API. Based on the semantic understanding capability of the large language model, a global scan of the standard policy corpus is performed. By identifying the core business terms, administrative level dimensions, and policy relationship logic in the policy text, a policy knowledge ontology structure including policy entity types, attribute dimensions, and relational constraints is automatically generated.

[0034] A multi-model collaborative extraction engine is configured, which utilizes multiple large language models with different parameter scales or fine-tuning directions to perform entity recognition and relation extraction tasks on a standard policy corpus. The extraction results of multiple large language models are cross-validated and aligned for consistency through a preset conflict resolution strategy. The conflict resolution strategy introduces an uncertainty estimation mechanism to quantify the confidence of the model output. Adaptive integration is used to dynamically adjust the number of participating models according to the query difficulty, resulting in a set of structured policy knowledge triples.

[0035] Based on the policy knowledge ontology structure, the set of structured policy knowledge triples is mapped to a graph database to construct a dynamic policy knowledge graph.

[0036] Responding to user-inputted policy consultation commands, the system extracts semantic features and query intent from these commands using a large language model. Based on these semantic features, it employs knowledge graph-enhanced retrieval technology to perform multi-hop path retrieval within a dynamic policy knowledge graph to obtain a deterministic knowledge subgraph related to the policy consultation command. The multi-hop path retrieval process includes: before performing vector retrieval, using a large language model to perform graph structure-aware query decomposition of the user query, identifying the entity types and relational path templates required for the query; using relational path templates to impose type constraints on the candidate node set, reducing invalid candidates; and then re-ranking the knowledge graph path score and vector similarity score according to learnable weights to return a more accurate knowledge subgraph.

[0037] Based on the structured facts provided by the deterministic knowledge subgraph, and combined with the natural language generation capabilities of the large language model, a response text with policy source information is constructed and output to the user terminal.

[0038] Understandably, the data collection terminals are deployed on government information disclosure websites, policy document publishing platforms, and internal office system data interfaces. Through configured web crawlers or application programming interface (API) listeners, an event-driven real-time acquisition method is used for platforms that provide subscription push notifications. For platforms that do not support push notifications, incremental crawling is performed at a configurable frequency (default no less than once every 30 minutes) to detect and capture newly released policy documents. The acquired multi-source unstructured policy text data includes, but is not limited to, Hypertext Markup Language (HTML) web page content, Portable Document Format (PDE) files, Word Processing (BMP) file files, and plain text files. Among them, PDE files can be extracted by an Optical Character Recognition (OCR) module, which uses a multimodal OCR visual language model based on a visual language model.

[0039] Among them, the multimodal OCR visual language model consists of two models: DeepSeek-OCR-2 and MinerU. It can directly process tables, charts, and seals in PDFs, realize the structured parsing of policy documents while maintaining the original hierarchy and format, support OCR and semantic understanding of policy documents in image form, and establish multimodal knowledge representation to support joint retrieval of images and text. The multimodal knowledge representation includes: structured parsing of tables in policy documents, converting table content into entities and relationships in a graph; recognizing seal images in policy documents and extracting the institution name and issuing unit information from the seal; semantic understanding of flowcharts, organizational charts, etc. in policy documents, converting them into path relationships in a graph, and is robust to common interference such as tables and seals in policy documents.

[0040] The preprocessing process first performs a unified character encoding conversion on all texts to ensure the use of Unicode UTF-8 encoding. Then, it performs cleaning operations to remove non-Chinese characters, redundant spaces, and line breaks. Next, it performs Chinese word segmentation based on a custom dictionary and a Hidden Markov Model. This custom dictionary can include over 100,000 policy-related professional terms, such as "high-tech enterprise certification," "R&D expense deduction," and "specialized, refined, and innovative SMEs." After cleaning and word segmentation, the system generates a unique file identifier for each policy document and attaches metadata, including the full name of the issuing unit, the date of issuance, the document number, the validity status, and the relevant business area tag. The business area tag can be selected from a predefined tag set, which can include no fewer than 15 categories such as technological innovation, industrial support, talent introduction, tax incentives, and financial support. This constructs a standard policy corpus, which is stored in a key-value database. The key is the file identifier, and the value is a structured document containing the original text, word segmentation results, and metadata.

[0041] Understandably, the process of automatically generating a policy knowledge ontology structure by globally scanning a standard policy corpus based on the semantic understanding capabilities of a large language model involves, specifically, the system calling a large language model with no fewer than 30 billion parameters, which has undergone supervised fine-tuning on massive amounts of Chinese corpus and policy texts. The process of generating the policy knowledge ontology structure can be designed as a multi-round iterative prompting engineering process: the first round of prompting requires the model to act as a "data sorting expert," scanning 20% ​​of the randomly sampled documents in the corpus to initially identify frequently occurring core business terms and their possible types. For example, identifying "applying enterprise" as "entity type: organization" and "funding amount" as "attribute type: number". The first round of prompts requires the model to act as a "relationship analysis expert," analyzing the potential relationships between terms in the text based on the term list output in the first round. For example, it might identify a "application" relationship between "applying company" and "funding amount," and a "jurisdiction" or "review" relationship between "provincial science and technology department" and "applying company." The third round of prompts requires the model to synthesize the information from the first two rounds and act as a "graph structuring expert," outputting a draft ontology structure conforming to a JSON template. This draft ontology structure must strictly adhere to the predefined JSON template, which defines ontologyName, entities, relations, nodeConfig, etc. Required fields include edgeConfig; each object in the entities array must include name, type, description, and attributes fields. For example, an entity is defined as {"name": "High-tech Enterprise", "type": "Organization", "description": "A legal person recognized as meeting the national high-tech enterprise standards", "attributes": ["Recognition Batch", "Valid Until", "Industry"]}; each object in the relations array must include name, sourceType, targetType, description, and cardinality fields. For example, a relation is defined as {"name": "Eligible", "sourceType": "High-tech Enterprise", "targetType": "Tax Incentives", "description": "Indicates that high-tech enterprises are eligible for specific tax reduction policies", "cardinality": "One-to-Many"}; after receiving the draft ontology structure output by the model, the system uses a JSON template validator to check it. If it does not conform to the format or key fields are missing, a repair loop is triggered, and error information is fed back to the model and it is required to regenerate until a JSON ontology description file with valid syntax and structure is obtained;The ontology structure generated through this process includes policy entity types such as "policy document," "issuing agency," "applicable entities," "support measures," and "application conditions," attribute dimensions such as "maximum amount," "validity period," "applicable region," and "application deadline," as well as relational constraints such as "reference," "revision," "include," "applicable to," and "issued by," thereby achieving automated structured modeling of the complex knowledge system in the policy domain.

[0042] It is understandable that the preprocessing steps for multi-source unstructured policy text data also include a processing procedure for extremely long policy documents. This processing procedure for extremely long policy documents includes: using an overlapping sliding window mechanism to semantically segment the extremely long policy documents, ensuring that adjacent segments retain a preset proportion of overlapping content to maintain contextual coherence; extracting summaries for each semantic segment based on a large language model, and associating and storing the summary information with the original segment text; wherein, the summary information is used to provide global semantic guidance in the subsequent knowledge extraction process to avoid the loss of policy elements due to text length limitations.

[0043] To address the common characteristics of policy documents being lengthy and structurally complex, this application's embodiments design an ultra-long document processing mechanism. Specifically, the system sets a fixed-length semantic window, the size of which can be dynamically adjusted based on the maximum context length of the large language model used, typically set to 70% of the model's maximum token count. For example, for a model supporting 8000 tokens, assuming an average of 1.5 tokens per Chinese character, it can process approximately 5300 Chinese characters, and the window size is set to 5300 Chinese characters. When segmenting a single policy document, a sliding window approach is used, with each slide step being 80% of the window size. This means that there is a 20% overlap between adjacent segments. This overlap ratio has been experimentally determined and can effectively prevent key information from being fragmented by falling precisely on segment boundaries while maintaining segment independence. For each segmented text obtained through the sliding window, the system calls a lightweight large language model for summary extraction. With approximately seven billion parameters, the system has been fine-tuned for policy text summarization tasks. The summary extraction prompts require the model to summarize the core policy intent or key clauses of a segment in one or two sentences. For example, for a text describing application conditions, the generated summary might be "This section stipulates the registration time and R&D investment ratio conditions that enterprises applying for innovation funds must meet." The generated summary text is stored together with the corresponding original segment text and the segment's start and end indexes in the original text, forming a "segment-summary" pair. In the subsequent knowledge extraction stage, when processing a segment, the system first loads its own summary and simultaneously attempts to load the summaries of its preceding and following segments as contextual references. These summary information collectively constitute a lightweight global semantic view, guiding the extraction model to better understand the roles and connections of entities and relationships in the current segment within the full text. This effectively avoids the problem of only seeing partial information due to model input length limitations, leading to one-sided or detached policy elements from the core policy intent.

[0044] Understandably, the steps of cross-validating and aligning the extraction results of multiple large language models through a pre-defined conflict resolution strategy include: assigning weight scores based on historical accuracy to each large language model participating in collaborative extraction; statistically analyzing the frequency of occurrence and corresponding weighted scores of extraction results from different large language models for the same policy entity or relation; triggering a voting mechanism based on confidence thresholds in response to discrepancies in extraction results among different large language models, selecting the extraction result with the highest weighted score as the final structured output, where the weighted score combines historical accuracy weights and uncertainty estimation weights, with models having lower uncertainty being assigned higher decision weights; the uncertainty estimation weights are calculated by using semantic entropy or token probability variance as a quantitative indicator of confidence for a single extraction; distinguishing between sub-tasks such as entity recognition, relation extraction, and attribute classification, and maintaining weights for each model in each sub-task category; and prioritizing the use of the model with higher accuracy as the judge when the output discrepancies among the models exceed a threshold.

[0045] Specifically, the system can configure at least three different large language models to participate in collaborative extraction. These large language models differ in parameter size, architecture, or fine-tuning data. For example, Model A is a general-purpose model with hundreds of billions of parameters, Model B is a model with tens of billions of parameters that has been fine-tuned on legal and policy texts, and Model C is a model based on an encoder-decoder architecture and specifically trained on entity relation joint extraction tasks. The system maintains a dynamic weight score for each model. The initial weight can be set based on its extraction accuracy F1 score on a predefined validation set. For example, a model with an F1 accuracy of 92% has an initial weight of 0.92. After each production environment extraction task is completed, the system randomly selects 5% of the results for manual or high-confidence automatic verification and dynamically updates the weights of each model based on the verification results. The weight update formula is: ; Where α is the smoothing coefficient (default is 0.9), and A_sample is the accuracy of this random sampling verification (A_sample = number of correct verifications / total number of samples); after each update, all model weights are normalized to ensure ΣW_i = 1; weight updates are performed asynchronously after each batch of sampling tasks is completed, without blocking the main process.

[0046] This ensures that the weights reflect the model's recent performance. When extracting a policy text, the three models run independently, each outputting its identified set of structured triples. For the same potential entity mentioned in the text, such as "innovative SMEs in XX City," different models may provide different normalized representations or type judgments. The conflict resolution strategy first standardizes all outputs, including the standardization of entity names (e.g., uniformly removing the "XX City" prefix or supplementing the full name) and the mapping of relation names (e.g., uniformly mapping "applicable" and "eligible" to "enjoyable"). Subsequently, statistics are performed on each standardized candidate triple, for example, <innovative SMEs, enjoyable, R&D subsidies>: calculating whether each model outputs this triple. If an output is generated, the current weights of that model are included in the total score of the triple. Simultaneously, the frequency of how many models simultaneously output the triple is recorded. A confidence threshold is set; for example, the triple must have a total score exceeding 0.85 and be output by at least two models to be accepted. If multiple competing relation judgments exist for the same group of entities, for example, model A considers the relation "enjoyable" while model B considers it "should conform," the total score of each relation candidate is calculated, and the relation with the highest score is ultimately selected as the output. This voting mechanism based on weights and frequency effectively reduces the illusionary extraction caused by specific biases, insufficient training data, or random errors in a single model, and improves the accuracy and reliability of the final structured knowledge triple set through collective decision-making.

[0047] It is understandable that the process of constructing a dynamic policy knowledge graph also includes the process of achieving real-time updates of policy knowledge. This real-time update process includes: configuring a policy change monitoring plugin to monitor update callback signals from the policy release platform in real time; automatically extracting incremental policy text and triggering a multi-model collaborative extraction engine for real-time processing in response to detected real-time update events such as policy repeal, revision, or new issuance; and logically deleting or correcting attributes of invalid nodes and relationships in the dynamic policy knowledge graph based on the incremental extraction results. The real-time update process ensures low-latency synchronization between the policy knowledge graph and the physical corpus. It employs a streaming computing framework to achieve near-real-time knowledge updates, uses incremental indexing technology to avoid full reconstruction, and implements an automatic knowledge conflict detection and resolution mechanism, supporting historical version tracing. Historical version tracing is achieved by adding timestamp attributes of the start and end dates of validity to each edge in the knowledge graph, constructing a temporal knowledge graph that supports historical policy tracing queries. The automatic knowledge conflict detection and resolution mechanism automatically determines whether a full text replacement, partial revision, or parallel validity is achieved by semantically comparing old and new clauses based on a large language model, and updates the relationship annotations in the graph accordingly.

[0048] To ensure the timeliness of the policy knowledge graph, this application implements a real-time update mechanism. The policy change monitoring plugin is deployed as a persistent background service, monitoring changes through multiple methods: for government information disclosure application interfaces that provide subscription functionality, the plugin directly subscribes to update notifications; for mainstream policy publishing websites, the plugin periodically (e.g., every 30 minutes) performs incremental crawling, detecting new documents by comparing webpage hash values ​​or publication time lists; in addition, the plugin can also access the electronic data interface of officially published policy repeal and revision lists; once a real-time update event is detected, the plugin parses the event type, including new policy releases, full-text revisions of old policies, partial clause revisions, policy repeals, etc., and extracts the unique identifier or Uniform Resource Locator (URL) of the relevant policy document; for newly released or revised documents, the plugin automatically downloads the full text and sends it to the preprocessing and long document processing flow to form a new standard corpus; subsequently, a multi-model collaborative extraction engine is immediately triggered to process the incremental text, generating a new set of structured triples; after receiving the incremental triples, the graph construction module starts a graph database transaction and executes the knowledge graph... Knowledge fusion operation: For newly added entities and relationships, they are directly inserted into the graph; for those identified as revisions to existing policies, the system needs to handle knowledge conflicts; the specific strategy of the automatic knowledge conflict detection and resolution mechanism is to set "effective start time" and "effective end time" attributes for each policy entity node in the graph, and each relationship edge also has "based document" and "clause index" attributes; when incremental processing identifies that a policy document has been replaced or revised by a new document, the "effective end time" of the entity node corresponding to the old policy document is updated to the effective date of the new policy, and its status is marked as historical version; at the same time, the relationship edges between the old policy and other entities are also marked as invalid; the new policy and its derived entities and relationships are inserted into the graph in an effective state and connected to the old policy entities through revision self-relationships; for revisions of some clauses, the system can also locate the knowledge subgraph corresponding to the specific clause and correct the attributes or relationships of the affected nodes; this ensures that the knowledge base on which the question-and-answer system is based remains synchronized with the real-world policy status, solving the problem that traditional static knowledge graphs cannot adapt to frequent policy changes.

[0049] It is understandable that the process of constructing a dynamic policy knowledge graph also includes a cross-document policy conflict detection process. This process includes: performing cross-document entity constraint detection on new policies with the same applicable objects and the same policy fields; identifying exclusive constraint clauses and establishing explicit mutually exclusive edges in the knowledge graph; and proactively prompting users of existing policy conflicts or declaration mutually exclusive restrictions in question-and-answer responses.

[0050] It is understandable that the steps of constructing a response text with policy source information and outputting it to the user terminal include: recording the policy source index tags associated with each retrieval path during multi-hop path retrieval; transforming the structured facts in the deterministic knowledge subgraph into logical deduction chains, and using these logical deduction chains as limiting constraints for the generation of the large language model; and in the generated response text, mapping the deduction conclusions to the policy source index tags through citation annotations. The steps of constructing a response text with policy source information and outputting it to the user terminal also include introducing an interpretability mechanism; wherein, the interpretability mechanism includes: introducing an attention mechanism... The visualization model focuses on key information, gradually revealing the reasoning process to enhance user trust. A confidence scoring system quantifies the reliability of answers, and interactive explanations allow users to inquire about reasoning details. The process of constructing and outputting response texts with policy source information to user terminals also includes a fact-checking process: extracting key facts such as numbers, dates, and organization names from the generated response; precisely comparing these with the values ​​of corresponding nodes or attributes in the knowledge graph, including string matching and numerical range verification; automatically triggering regeneration or targeted correction if inconsistencies are found; and marking the verification status of each assertion in the response to enhance user judgment of the answer's credibility.

[0051] In this embodiment, after the intent retrieval module performs multi-hop path retrieval and obtains a deterministic knowledge subgraph, the question-answering fusion module first parses the deterministic knowledge subgraph. This deterministic knowledge subgraph is essentially a small network composed of entity nodes and relational edges. The intent retrieval module traverses this network and converts it into a structured text description, i.e., a logical deduction chain. For example, the deterministic knowledge subgraph includes nodes "Enterprise A," "High-tech Enterprise," and "Enterprise Income Tax Reduction / Exemption," and relations "belongs to" and "eligible for." The converted logical chain text is: "Given: Enterprise A belongs to a high-tech enterprise (according to Article X of document 'Guoke Fa Huo

[2021] XX'); Rule: High-tech enterprises can enjoy enterprise income tax reduction / exemption (according to Article XX of document 'Implementation Regulations of the Enterprise Income Tax Law'); Conclusion: Enterprise A can enjoy enterprise income tax reduction / exemption." This logical chain text, along with the graph structure data of the original knowledge subgraph and the policy original text index tags (including document tags) associated with each fact node recorded during the retrieval process, is used to parse the subgraph. The identifier, document name, specific clause number, or paragraph index, along with other information, are input as part of the system prompts into the large language model responsible for generating the final response. The large language model is explicitly instructed to base its answers on logical chains of facts, use natural language to organize them, and must include superscript or endnotes in parentheses at specific conclusions in the answer to indicate the original policy text index. For example, the generated response might be: "According to relevant policies, Company A, as a recognized high-tech enterprise (based on the 'Guoke Fa Huo

[2021] No. XX'), can enjoy a preferential policy of reducing the corporate income tax rate to 15% (based on Article XX of the 'Implementation Regulations of the Corporate Income Tax Law')." In this way, the generation process of the large language model is confined to the scope of deterministic facts provided by the deterministic knowledge subgraph, avoiding fictitious or outdated information that might arise from the model's free interpretation. Furthermore, each key argument in the answer has a clear policy source, forming a complete interpretable path, greatly enhancing the credibility and authority of the question-and-answer results.

[0052] It is understandable that the steps of constructing a response text with policy source information and outputting it to the user terminal also include an iterative retrieval and self-verification process. This iterative retrieval and self-verification process includes: decomposing complex policy consultation instructions into multiple subqueries using a large language model; independently performing knowledge graph retrieval on each subquery and aggregating the results of multiple rounds of retrieval into a structured context; generating a preliminary answer based on a deterministic knowledge subgraph, and then re-verifying the factual consistency of key assertions. If inconsistencies are found, regeneration or targeted correction is triggered; and the verification status of each assertion is marked in the response, including whether it has been verified or not.

[0053] Understandably, the steps for multi-hop path retrieval in a dynamic policy knowledge graph based on semantic features include: calculating the cosine similarity between the semantic vector of the policy consultation instruction and the attributes of each node in the dynamic policy knowledge graph; dynamically delineating the candidate node set based on semantic relevance scores and determining the retrieval depth based on intent recognition results; the retrieval process employs an adaptive retrieval strategy, including query rewriting to convert the user's natural language query into a standard retrieval expression, a hybrid retrieval strategy combining keyword retrieval and semantic retrieval, dynamically selecting the optimal retrieval strategy based on query intent recognition, and continuously optimizing the retrieval results based on user feedback; and executing a graph traversal algorithm starting from the candidate node set to extract a highly relevant knowledge subgraph that covers the user's potential intent.

[0054] Specifically, first, the user inputs a policy consultation command, such as "What talent subsidies can an AI startup registered in a certain city and district apply for?", which is then processed by a large language model to extract semantic features and identify intent. The large language model outputs a dense semantic vector of the command, while parsing out the core intent as "inquiring about subsidy eligibility" and identifying the key entities "a certain city and district", "artificial intelligence", "startup", and "talent subsidy". Next, the retrieval system loads pre-computed semantic vectors of the attribute text (such as name, description, and type) of all entity nodes in the dynamic policy knowledge graph. The cosine similarity between the command semantic vector and the attribute vector of each node is calculated to obtain a score between 0 and 1. The system sets a dynamic threshold, which is adjusted according to the clarity of the entities identified in this round of query.

[0055] The formula for calculating the dynamic threshold τ is: ; Where C_entity is the mean confidence score of entity recognition (value range [0, 1], output by the intent recognition module). Take 0.55, The threshold is set to 0.80. When C_entity > 0.7, τ approaches 0.75; when C_entity < 0.4, τ approaches 0.60, achieving smooth dynamic adjustment. When the threshold is between these two values, it is calculated using linear interpolation. Finally, all nodes with similarity higher than the threshold are selected to form an initial candidate node set. In this example, this might include nodes such as "a certain district in a certain city," "artificial intelligence enterprise," "start-up enterprise," and "talent subsidy in a certain province." Simultaneously, the intent recognition result is used to determine the retrieval depth, i.e., the maximum number of hops. "Query qualification" intents typically require 2-3 hops of associated paths, so the maximum depth is set to 3. The retrieval starts simultaneously from the candidate node set and executes a breadth-first graph traversal algorithm. During traversal, it moves along relation edges and evaluates the comprehensive semantic relevance of the nodes and relations along the path. This comprehensive semantic relevance is determined by the initial similarity between each node on the path and the query instruction, as well as the type weight of the relation edges. The algorithm explores from the starting point... The system starts by traversing all possible paths within the maximum depth range and collects the nodes and edges along those paths. Finally, from all traversed paths, the system selects all nodes and relationships involved in the paths that connect multiple query keywords (e.g., simultaneously connecting region, industry type, enterprise stage, and subsidy type) and have the highest overall relevance, merging them into a connected, highly relevant knowledge subgraph. This highly relevant knowledge subgraph includes directly related entities as well as intermediate entities that act as bridges (e.g., it may include the node "5050 Talent Recruitment Plan of a Certain City," which connects "a certain district of a certain city" and "talent subsidy"), thereby covering users' potential, unexpressed related intentions and achieving targeted deep knowledge retrieval.

[0056] It is understandable that configuring a multi-model collaborative extraction engine also includes a process for scheduling heterogeneous model computing resources. This process includes: real-time collection of the memory usage and computing load status of each computing node in the server cluster; dynamic matching of large language models with corresponding parameter sizes based on the complexity score of the policy text to be processed; and distribution of extraction tasks to the optimal computing node using a load balancing gateway. The scheduling process achieves efficient utilization of model computing resources in multi-concurrency scenarios.

[0057] To address the demands of processing large-scale policy corpora and handling high-concurrency user inquiries, the system can be designed with a scheduling mechanism for heterogeneous model computing resources. The collaborative extraction engine is deployed on a heterogeneous computing cluster, which includes various types of graphics processing unit (GPU) nodes. For example, nodes equipped with high-speed video graphics array memory are used to run large models with hundreds of billions of parameters, nodes equipped with Titan RTX or RTX 4090 processors are used to run models with tens of billions of parameters, and nodes equipped with consumer-grade GPUs are used to run lightweight or summarization models. Each node runs a resource monitoring daemon that collects key metrics for that node once per second, including GPU memory usage percentage, GPU core utilization, system memory usage, and the length of the currently executing task queue, and reports these metrics to the central scheduler in real time via a message queue. When new policy text needs to be extracted, the preprocessing module performs a rapid analysis of the text, generating a complexity score. The scoring factors include text length, terminology density, average sentence length, and the number of logical connectors, with a score range from 1 to 10. Specific scoring algorithm: Complexity score ; Where L_norm is the number of text characters normalized to the maximum value, T_density is the proportion of technical terms to the total number of words, S_len_norm is the average sentence length normalized to the maximum value, and C_density is the proportion of logical connectors (such as "therefore", "but", "and") to the total number of sentences; the final score S is rounded to the nearest integer and limited to the range [1, 10].

[0058] When the scheduler receives an extraction task request with a complexity score of S, it matches it according to a preset strategy: short or simple texts with a complexity score of S below 3 are assigned to lightweight model nodes; medium-complexity texts with a score of S between 4 and 7 are assigned to model nodes with tens of billions of parameters; and long or complex policy texts with a score of S above 8 are assigned to large model nodes with hundreds of billions of parameters.

[0059] Meanwhile, the scheduler comprehensively considers the current load of the target node type, selects the node with the lowest memory usage and shortest task queue among nodes of the same type, and routes the task requests to it through the load balancing gateway. The gateway is responsible for maintaining the connection pool with each node and handling the serialization and deserialization of requests. In addition, the system also sets up a priority queue, and incremental update extraction tasks with high real-time requirements have higher priority than background batch processing tasks. Through this dynamic scheduling strategy based on text complexity and real-time load, the system can maximize the throughput of the entire cluster, avoid the resource waste problem of large model nodes being blocked by simple tasks while lightweight nodes are idle, and ensure the stability and efficiency of the system in high-concurrency knowledge extraction and question answering scenarios.

[0060] It is understandable that the steps in generating the policy knowledge ontology structure also include a semantic disambiguation process for policy terminology. This process includes: acquiring the administrative level metadata and business domain labels of the multi-source unstructured policy text data; identifying policy terms with polysemous names based on a large language model and constructing terminology context vectors in conjunction with the administrative level metadata; configuring unique entity identifiers for terms with the same name in different contexts in a dynamic policy knowledge graph; wherein, the entity identifiers are used to achieve cross-domain policy knowledge alignment.

[0061] Specifically, when implementing automatic ontology generation and knowledge extraction, the system integrates a dedicated semantic disambiguation process. First, in the preprocessing stage, the system adds administrative level metadata (e.g., national, provincial, municipal, district / county level) and business domain labels (e.g., science and technology, finance, human resources and social security, housing and construction) to each policy text, which serve as the context for disambiguation. When the large language model identifies a latent term, such as "special funds," it simultaneously analyzes the context sentences and document metadata. The disambiguation algorithm constructs an enhanced semantic vector representation based on the latent term and its context: concatenating the latent term itself, its corresponding sentence, and the document's administrative level and domain labels into a single text, which is then input into a sentence encoder model to obtain a fused vector. The system maintains a "term-meaning" candidate library, which records typical context vectors for "special funds" under different meanings. For example, meaning one is "science and technology research and development special funds" (often appearing in science and technology fields and national-level documents), containing... The second meaning is "Special Fund for the Development of Small and Medium-sized Enterprises" (commonly found in the industrial and information technology sector and provincial documents). By calculating the similarity between the enhanced vector of the current term and the meaning vectors in the candidate library, and setting a threshold, such as 0.8, the system determines which meaning the term specifically refers to in the current context. If the similarity is below the threshold, it is considered a new meaning variant and stored as a new candidate in the library. When constructing the knowledge graph, for identified terms with the same name but different meanings, the system can create different entity nodes and assign globally unique entity identifiers, such as "Special Fund #Science and Technology" and "Special Fund #Industrial and Information Technology". Simultaneously, the administrative level and business domain to which the entity node belongs are explicitly recorded in its attributes. When performing cross-policy or cross-domain knowledge retrieval or association analysis, the system can accurately distinguish and match terms by comparing entity identifiers and hierarchical domain attributes, avoiding incorrect association of "Special Fund" in different contexts, thus ensuring the consistency and accuracy of semantics within the knowledge graph.

[0062] It is understood that the method in this application embodiment also includes a system adaptive degradation process, which includes: triggering a fault-tolerant feedback mechanism when multi-hop path retrieval fails to obtain a valid knowledge subgraph or when the reasoning response time of the large language model exceeds a preset threshold; performing keyword retrieval based on the full-text indexing engine of the policy corpus and feeding back the retrieved policy text summary as a fallback answer to the user; wherein, the system simultaneously records the failed path of this multi-hop path retrieval for subsequent knowledge base completion.

[0063] To ensure system robustness and user experience, this application designs an adaptive degradation handling mechanism. The system sets health checkpoints and timeout monitoring on critical paths. In the question-answering process, two main stages may fail: first, after multi-hop path retrieval, the obtained deterministic knowledge subgraph is empty or has very few nodes (less than two), indicating a lack of directly related structured knowledge in the graph; second, when the question-answering module calls the large language model to generate the final response, the model response time exceeds a preset threshold, such as 15 seconds. When either of these situations occurs, the system immediately triggers a fault-tolerant feedback mechanism. This fault-tolerant feedback mechanism first attempts a fallback retrieval based on the full-text index of the policy corpus. The full-text indexing engine is built on the open-source search engine Elasticsearch and establishes an inverted index for all preprocessed policy text content. The system extracts core keywords from the user's original query (usually provided by the initial intent recognition module), such as "talent subsidy for artificial intelligence startups in a certain city and district," and uses these keywords to search in the full-text index. The engine returns several policy text fragments or pre-stored document summaries with the highest relevance. The system performs simple deduplication and sorting on these fragments or summaries. The system sorts and assembles the information into a prompt message, such as "Based on a search of relevant policy databases, the following information may be relevant to your query: 1. The 'Several Opinions on Accelerating the Development of the Artificial Intelligence Industry in a Certain City and District' mentions that rental subsidies will be provided to newly introduced talents in the field of artificial intelligence... 2. The 'Eagle Plan' Implementation Plan of a Certain Province stipulates that one-time start-up subsidies will be provided to qualified early-stage technology enterprises..." This assembled text, with the added note "This is reference information based on full-text search; please refer to official documents for details," is returned directly to the user terminal as a fallback answer. Simultaneously, the system can fully record the failed query request, search path (or timeout step), and the final fallback keywords and returned document identifiers. These records are stored in a dedicated "Knowledge Gap Log" database. Regularly (e.g., weekly), analysis tasks scan this log to statistically analyze frequently failed query patterns or missing knowledge points, providing clear data support for the completion and optimization of the knowledge graph. For example, it may prompt knowledge engineers to supplement policy knowledge of a specific region or industry, or adjust ontology relationships to cover new query intents, thereby achieving the system's self-evolution.

[0064] Parameter selection experiment: To verify the rationality of the key parameters in the technical solution of this application, a comparative experiment was conducted, and the results are shown in Table 1.

[0065] Experimental environment: 1. Policy corpus size: 500,000 documents; 2. Server configuration: 8×A100 GPUs (80GB video memory); 3. Test dataset: 1000 policy consultation instructions (manually annotated).

[0066] Table 1. Comparison of key parameters and experimental results:

[0067] Experimental conclusion: 1. A 20% overlap ratio for sliding windows can effectively maintain contextual coherence and prevent the loss of policy elements; 2. The confidence threshold adopts a dynamic adjustment mechanism. =0.55, =0.80), which can be adaptively optimized according to query complexity; 3. The collaborative extraction of three heterogeneous models achieves an optimal balance between accuracy and computational cost; 4. The maximum search depth of 3 hops can cover the vast majority of policy-related queries while ensuring response speed.

[0068] The above parameter selections are all based on actual test data, ensuring the feasibility and optimality of the technical solution.

[0069] like Figure 2 As shown, this application also discloses a policy question-answering system based on multi-model collaboration and graph enhancement, comprising: a data preprocessing module for collecting and preprocessing multi-source unstructured policy text data to construct a standard policy corpus; an ontology automatic generation module for automatically generating a policy knowledge ontology structure based on a large language model; a collaborative extraction module for configuring a multi-model collaborative extraction engine, performing entity recognition and relation extraction, and performing consistency alignment to obtain a set of structured policy knowledge triples; a graph construction module for constructing a dynamic policy knowledge graph based on the policy knowledge ontology structure; an intent retrieval module for extracting semantic features in response to user commands and performing multi-hop path retrieval to obtain a deterministic knowledge subgraph; a real-time update module for configuring a policy change monitoring plugin to achieve incremental real-time updates of the policy knowledge graph; a semantic disambiguation module for configuring unique entity identifiers for policy terms with the same name but multiple meanings; a fault tolerance and degradation module for triggering a full-text index fallback mechanism and recording knowledge gaps when graph retrieval fails; and a fusion question-answering module for generating response text with policy source information based on the deterministic knowledge subgraph and the large language model.

[0070] This application also discloses a computer-readable storage medium storing a computer program that, when executed by a processor, implements the aforementioned policy question-answering method based on multi-model collaboration and graph enhancement.

[0071] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional modules is used as an example. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. The specific working process of the system, device, and unit described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0072] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, or indirect coupling or communication connection between apparatuses or units, and may be electrical, mechanical, or other forms.

[0073] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0074] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0075] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory, random access memory, magnetic disks, or optical disks.

[0076] The above are all preferred embodiments of this application and are not intended to limit the scope of protection of this application. Any feature disclosed in this specification (including the abstract and drawings) may be replaced by other equivalent or similar features unless specifically stated otherwise. That is, unless specifically stated otherwise, each feature is only one example of a series of equivalent or similar features.

Claims

1. A policy question answering method based on multi-model collaboration and graph enhancement, characterized in that, include: Multi-source unstructured policy text data is acquired through a data acquisition terminal, and the multi-source unstructured policy text data is preprocessed to construct a standard policy corpus; On a server configured with a GPU cluster, a large language model with no less than 30 billion parameters is called through the inference API. Based on the semantic understanding capability of the large language model, a global scan of the standard policy corpus is performed. By identifying the core business terms, administrative level dimensions, and policy association logic in the policy text, a policy knowledge ontology structure including policy entity types, attribute dimensions, and relational constraints is automatically generated. A multi-model collaborative extraction engine is configured, which utilizes multiple large language models with different parameter scales or fine-tuning directions to perform entity recognition and relation extraction tasks on the standard policy corpus. The extraction results of multiple large language models are cross-validated and aligned for consistency through a preset conflict resolution strategy. The conflict resolution strategy introduces an uncertainty estimation mechanism to quantify the confidence of the model output and adopts adaptive integration to dynamically adjust the number of participating models according to the query difficulty, thereby obtaining a set of structured policy knowledge triples. Based on the policy knowledge ontology structure, the set of structured policy knowledge triples is mapped to a graph database to construct a dynamic policy knowledge graph; In response to a policy consultation instruction input by a user, the semantic features and query intent in the policy consultation instruction are extracted through a large language model. Based on the semantic features, knowledge graph-enhanced retrieval technology is used to perform multi-hop path retrieval in the dynamic policy knowledge graph to obtain a deterministic knowledge subgraph related to the policy consultation instruction. The multi-hop path retrieval process includes: before performing vector retrieval, first using a large language model to perform graph structure-aware query decomposition on the user query, identifying the entity types and relational path templates required for the query; using relational path templates to impose type constraints on the candidate node set to reduce invalid candidates; and then mixing and reordering the knowledge graph path score and vector similarity score according to learnable weights to return a more accurate knowledge subgraph. Based on the structured facts provided by the deterministic knowledge subgraph, and combined with the natural language generation capabilities of the large language model, a response text with policy source information is constructed and output to the user terminal.

2. The policy question answering method based on multi-model collaboration and graph enhancement according to claim 1, characterized in that, The preprocessing step for the multi-source unstructured policy text data also includes a processing procedure for extremely long policy documents, which includes: An overlapping sliding window mechanism is used to semantically segment ultra-long policy documents to ensure that a preset proportion of overlapping content is retained between adjacent segments to maintain contextual coherence. Summarize each semantic segment based on a large language model, and store the summary information in association with the original segment text; The summary information is used to provide global semantic guidance in the subsequent knowledge extraction process to avoid the loss of policy elements due to text length limitations; The step of cross-validating and aligning the extraction results of multiple large language models using a preset conflict resolution strategy includes: Assign a weight score based on its historical accuracy to each large language model participating in collaborative extraction; For the same policy entity or relationship, the frequency of occurrence and corresponding weighted scores of the extraction results from different large language models are statistically analyzed. In response to discrepancies in extraction results among different large language models, a voting mechanism based on a confidence threshold is triggered, selecting the extraction result with the highest weighted score as the final structured output. The weighted score combines historical accuracy weights and uncertainty estimation weights, assigning higher decision weights to models with lower uncertainty. The uncertainty estimation weights are calculated by using semantic entropy or token probability variance as a quantitative indicator of single extraction confidence; distinguishing between subtasks including entity recognition, relation extraction, and attribute classification, and maintaining weights for each model on each subtask category; when the discrepancy in the outputs of different models exceeds a threshold, the model with higher accuracy is preferentially called as the judge.

3. The policy question answering method based on multi-model collaboration and graph enhancement according to claim 2, characterized in that, The step of constructing a dynamic policy knowledge graph also includes a process for real-time updating of policy knowledge, which includes: Configure a policy change monitoring plugin to monitor update callback signals from the policy release platform in real time; In response to real-time updates of policy repeal, revision, or new issuance, the incremental policy text is automatically extracted and the multi-model collaborative extraction engine is triggered for real-time processing. Based on the incremental extraction results, the invalid nodes and relationships in the dynamic policy knowledge graph are logically deleted or their attributes are corrected. The real-time update process ensures low-latency synchronization between the policy knowledge graph and the physical corpus. This process employs a streaming computing framework for near real-time knowledge updates, uses incremental indexing technology to avoid full reconstruction, and implements an automatic knowledge conflict detection and resolution mechanism, supporting historical version tracing. Historical version tracing is achieved by adding timestamp attributes of the start and end dates of the validity period to each edge in the knowledge graph, constructing a temporal knowledge graph that supports historical policy queries. The automatic knowledge conflict detection and resolution mechanism uses a large language model to semantically compare new and old clauses, automatically determining whether it is a full text replacement, partial revision, or parallel validity, and updates the relationship annotations in the graph accordingly. The step of constructing a dynamic policy knowledge graph also includes a cross-document policy conflict detection process, which includes: Perform cross-document entity constraint detection on newly added policies with the same applicable objects and the same policy areas; Identify exclusive constraint clauses and establish explicit mutually exclusive edges in the knowledge graph; In the Q&A response, proactively remind users of any policy conflicts or mutually exclusive application restrictions.

4. The policy question answering method based on multi-model collaboration and graph enhancement according to claim 3, characterized in that, The step of constructing a response text with policy source information and outputting it to the user terminal includes: When performing the multi-hop path retrieval, record the policy text index label associated with each retrieval path; The structured facts in the deterministic knowledge subgraph are transformed into logical deduction chains, and these logical deduction chains are used as limiting constraints for the generation of the large language model. In the generated response text, the derivation conclusions are correlated with the index tags of the original policy text through reference annotations; The step of constructing a response text with policy source information and outputting it to the user terminal also includes introducing an interpretability mechanism; The explainability mechanism includes: introducing an attention mechanism to visualize key information that the model focuses on, gradually revealing the reasoning process to enhance user trust, establishing a confidence scoring system to quantify the reliability of the answer, and supporting interactive explanations that allow users to ask follow-up questions about the reasoning details; the step of constructing a response text with policy source information also includes a fact-checking process: extracting key facts containing numbers, dates, and organization names from the generated response; performing precise comparisons with the values ​​of corresponding nodes or attributes in the knowledge graph, including string matching and numerical range verification; automatically triggering regeneration or targeted correction if inconsistencies are found; and marking the verification status of each assertion in the response to enhance users' judgment of the answer's credibility; The step of constructing a response text with policy source information and outputting it to the user terminal also includes an iterative retrieval and self-verification process, which includes: Complex policy consultation instructions are broken down into multiple subqueries using a large language model; Knowledge graph retrieval is performed independently for each subquery, and the results from multiple rounds of retrieval are aggregated into a structured context; After generating a preliminary answer based on the aforementioned deterministic knowledge subgraph, the consistency of facts in key assertions is verified again. If inconsistencies are found, regeneration or targeted correction is triggered. In the reply, indicate the verification status of each statement, including whether it has been verified or not.

5. A policy question answering method based on multi-model collaboration and graph enhancement according to claim 4, characterized in that, The step of performing multi-hop path retrieval in the dynamic policy knowledge graph based on the semantic features includes: Calculate the cosine similarity between the semantic vector of the policy consultation instruction and the attributes of each node in the dynamic policy knowledge graph; The candidate node set is dynamically delineated based on semantic relevance scores, and the retrieval depth is determined based on intent recognition results. The retrieval process adopts an adaptive retrieval strategy, including query rewriting to convert the user's natural language query into a standard retrieval expression, a hybrid retrieval strategy combining keyword retrieval and semantic retrieval, dynamic selection of the optimal retrieval strategy based on query intent recognition, and continuous optimization of retrieval result reordering based on user feedback. Starting from the candidate node set, a graph traversal algorithm is executed to extract a highly relevant knowledge subgraph that can cover the user's potential intent.

6. The policy question answering method based on multi-model collaboration and graph enhancement according to claim 5, characterized in that, The step of configuring the multi-model collaborative extraction engine also includes a scheduling process for heterogeneous model computing resources, which includes: Real-time collection of memory usage and computing load status of each computing node in the server cluster; Based on the complexity score of the policy text to be processed, a large language model with corresponding parameter size is dynamically matched. Use a load balancing gateway to distribute extraction tasks to the nodes with the best computing power. The scheduling process enables efficient utilization of model computing resources in multi-concurrency scenarios.

7. The policy question answering method based on multi-model collaboration and graph enhancement according to claim 1, characterized in that, The step of generating the policy knowledge ontology structure also includes a semantic disambiguation process for implementing policy terminology, which includes: Obtain the administrative level metadata and business domain tags to which the multi-source unstructured policy text data belongs; Based on the large language model, policy terms with polysemous names are identified, and term context vectors are constructed by combining the administrative level metadata. In the dynamic policy knowledge graph, unique entity identifiers are configured for terms with the same name in different contexts; The entity identifier is used to align policy knowledge across domains.

8. The policy question answering method based on multi-model collaboration and graph enhancement according to claim 1, characterized in that, It also includes a system adaptive degradation process, which includes: When the multi-hop path retrieval fails to obtain a valid knowledge subgraph, or when the large language model inference response time exceeds a preset threshold, a fault-tolerant feedback mechanism is triggered. The full-text indexing engine based on the policy corpus performs keyword retrieval and provides the retrieved policy text abstracts as a fallback answer to the user. The system also records the failed paths of this multi-hop path retrieval for subsequent knowledge base completion.

9. A policy question-answering system based on multi-model collaboration and graph enhancement, characterized in that, This includes a plurality of functional modules configured to perform the method as described in any one of claims 1 to 8, said functional modules comprising at least: The data preprocessing module is used to collect and preprocess multi-source unstructured policy text data to build a standard policy corpus; The ontology automatic generation module is used to automatically generate policy knowledge ontology structures based on large language models; The collaborative extraction module is used to configure a multi-model collaborative extraction engine, perform entity recognition and relation extraction and perform consistency alignment to obtain a set of structured policy knowledge triples. The graph construction module is used to construct a dynamic policy knowledge graph based on the policy knowledge ontology structure. The intent retrieval module is used to extract semantic features and perform multi-hop path retrieval in response to user instructions in order to obtain a deterministic knowledge subgraph. The real-time update module is used to configure the policy change monitoring plugin to achieve incremental real-time updates of the policy knowledge graph; The semantic disambiguation module is used to configure unique entity identifiers for policy terms with the same name but multiple meanings; The fault tolerance and degradation module is used to trigger the full-text index fallback mechanism and record the knowledge gap when the graph retrieval fails; The integrated question-and-answer module is used to generate response text with policy source information based on the deterministic knowledge subgraph and the large language model.

10. A computer-readable storage medium storing a computer program that, when executed by a processor, implements a policy question-answering method based on multi-model collaboration and graph enhancement as described in any one of claims 1 to 8.

Images