Method and apparatus for monitoring and updating document semantic consistency

Abstract

Embodiments of the present application provide a document semantic consistency monitoring and updating processing method and device. The method comprises: determining a baseline event package corresponding to a release source of a document by using a preset large language model; based on the baseline event package, extracting a clause-level entity node in the file by using the large language model, constructing a dependency edge with metadata, and forming a knowledge graph of clause-level risk dependencies; obtaining a change event of the document, mapping the change event to the knowledge graph, calculating a risk score of an affected clause in the knowledge graph, and determining a structured risk report based on the risk score; executing a hierarchical response based on the structured risk report to generate revision suggestions, obtaining confirmed revision content as a response result, and performing versioned updating of the knowledge graph based on the response result, and performing adaptive learning and optimization of the large language model by using the response result. The scheme of the present application solves the problems of poor timeliness, high semantic risk and low updating efficiency of the document.

Description

Technical Field

[0001] This application relates to the field of document processing technology, specifically to a method and apparatus for monitoring and updating document semantic consistency. Background Technology

[0002] With the deep integration of knowledge management, natural language processing, and artificial intelligence technologies, document semantic consistency monitoring and intelligent update technologies have become crucial support for enterprise digital governance and business collaboration. Various technical documents and business specifications serve as the core basis for internal circulation, multi-departmental collaboration, and research activities within enterprises, facing challenges such as high update frequency and complex revisions, resulting in massive dynamic data flows. Traditional document management primarily relies on manual intervention; however, with the surge in data volume and significantly shortened update cycles, traditional methods are no longer sufficient to meet modern business needs in terms of real-time performance, accuracy, and automation.

[0003] Currently, the industry's technical system for document management and change detection mainly covers four core components: automated data collection technology based on web crawlers, structured parsing technology based on knowledge graphs, change detection and semantic risk assessment technology, and intelligent update response technology. However, existing technical solutions still have many shortcomings in practical applications, as follows:

[0004] 1) Existing web crawler technologies mostly rely on predefined rules or script simulations, which have limitations when dealing with complex web page scenarios such as dynamic loading and multi-level redirects. Furthermore, the lack of identification of document lifecycle attributes and real-time data source access capabilities leads to lag in change detection, making it difficult to achieve incremental responses at the minute or even second level.

[0005] 2) Existing solutions mostly remain at the coarse-grained document-level association level, failing to delve into the fine-grained clause-level parsing. Furthermore, the lack of optimization for parsing non-textual elements (such as complex tables and flowcharts) leads to semantic recognition biases, making it difficult to fully reconstruct the document's logical structure and deep semantics.

[0006] 3) Existing change detection solutions typically only identify surface-level textual differences or perform simple weight propagation, lacking effective assessment of deep semantic impact. Furthermore, due to the absence of a timeliness engine and cross-modal risk propagation mechanism, the system struggles to accurately quantify the consistency risks caused by document changes, resulting in high false positive and false negative rates.

[0007] 4) Most existing systems adopt a passive alarm-based design, lacking fully automated revision capabilities driven by artificial intelligence, and have not formed a complete human-machine collaborative closed loop. In addition, the system architecture has poor adaptability and is difficult to meet the deployment requirements of large-scale, high-concurrency, and dynamically evolving systems.

[0008] In summary, existing technologies face technical bottlenecks in multiple aspects, including data acquisition, deep semantic analysis, quantification of change impact, and automated response, making it difficult to achieve closed-loop management of the entire document lifecycle. There is an urgent need for a technical solution that can address these issues. Summary of the Invention

[0009] At least one embodiment of this application provides a method and apparatus for monitoring and updating document semantic consistency, which solves the problems of poor timeliness, high semantic risk, and low update efficiency caused by document dynamics in the prior art.

[0010] To solve the above-mentioned technical problems, this application is implemented as follows:

[0011] In a first aspect, embodiments of this application provide a method for monitoring and updating document semantic consistency, including:

[0012] Using a pre-defined large-scale language model, a baseline event package corresponding to the publication source of the document is determined, wherein the baseline event package includes the file corresponding to the document and the metadata associated with the file;

[0013] Based on the baseline event package, the large language model is used to extract the clause-level entity nodes in the file, construct dependency edges with metadata, and form a knowledge graph of clause-level risk dependencies.

[0014] Perform periodic monitoring to obtain change events of the document, map the change events to the knowledge graph, calculate the risk score of the affected clauses in the knowledge graph, and determine a structured risk report based on the risk score;

[0015] Based on the structured risk report, a tiered response is executed to generate revision suggestions. Confirmed revisions are obtained as response results. The knowledge graph is then updated in a versioned manner based on the response results. Finally, the large language model is adaptively optimized using the response results.

[0016] Optionally, using a pre-defined large language model, the baseline event package corresponding to the document's publication source is determined, including:

[0017] Receive a crawling task request; the crawling task request is used to crawl the file corresponding to the document published by the source.

[0018] Based on the crawling task request, a set of candidate URLs is obtained by using a pre-set large language model to infer the starting point Uniform Resource Locator URL or search keywords.

[0019] The page structure of the pages in the candidate URL set is parsed using the large language model.

[0020] Based on the parsed page structure, the file is located and downloaded, and the file and its associated metadata are encapsulated into a baseline event package; the metadata includes the file's target topic, webpage address, crawling path, and timestamp;

[0021] After determining the baseline event package, a pre-defined large language model is used, and periodic monitoring is performed to obtain change events.

[0022] Optionally, based on the baseline event package, the large language model is used to extract the clause-level entity nodes in the file, construct dependency edges with metadata, and form a knowledge graph of clause-level risk dependencies, including:

[0023] The files in the baseline event package are subjected to layout analysis and text extraction to convert unstructured documents into structured text.

[0024] The large language model is used to parse structured text, identify and extract clause-level entity nodes in the file, and generate a corresponding semantic vector representation for each node;

[0025] Scan the structured text, identify the reference relationships between clauses, and combine the semantic vector representation to perform entity linking and determine the dependency edges between clause nodes;

[0026] Based on the identified reference relationship type, metadata containing reference features and confidence is generated for the dependency edge, thus generating a dependency edge with metadata;

[0027] The clause-level entity nodes are integrated with the dependency edges with metadata to construct a knowledge graph of clause-level risk dependencies.

[0028] Optionally, periodic monitoring is performed to obtain change events of the document, the change events are mapped to the knowledge graph, a risk score is calculated for the affected clauses in the knowledge graph, and a structured risk report is determined based on the risk score, including:

[0029] Continuous monitoring is performed by accessing data subscription sources or change detection interfaces. The large language model is used to parse the announcement list to identify new announcements and then download the target file from the details page.

[0030] The target file is hashed and compared with the file in the baseline event package. The validity is verified by semantic similarity analysis. The verification result is obtained. The valid changed or added file is encapsulated into an initial change event, and the unchanged file is discarded.

[0031] The timeliness of the initial change event is verified based on the document lifecycle rules, and a timeliness risk label is marked. The timeliness risk label includes at least one of the following: invalid document reference, ineffective document reference, and document transition period status.

[0032] The verification result, the timeliness risk label, and the initial change event are integrated to generate the change event;

[0033] The change event is mapped to the knowledge graph, the risk score of the affected clause in the knowledge graph is calculated, and a structured risk report is determined based on the risk score.

[0034] Optionally, the change event is mapped to the knowledge graph, a risk score is calculated for the affected clauses in the knowledge graph, and a structured risk report is determined based on the risk score, including:

[0035] The change event is analyzed to determine the change node, and the corresponding old clause node is located in the knowledge graph and a mapping relationship is established.

[0036] Starting from the changed node, perform a reverse traversal along the dependency edges in the knowledge graph to collect the upstream clause nodes affected by the change;

[0037] Read the metadata of the dependent edges between the upstream clause node and the change node, identify the reference relationship type, and determine the basic weight;

[0038] The semantic sensitivity between the changed content corresponding to the changed node and the upstream clause text corresponding to the upstream clause node is calculated, and a comprehensive risk score is obtained by combining the basic weights. The comprehensive risk score is then checked for consistency by using a natural language reasoning model to detect whether there is a logical conflict between the upstream clause text and the changed content. If a conflict exists, the risk score is determined to be the preset highest risk level.

[0039] Summarize the risk scores, impact paths, and reasons for the determination of all affected clauses to generate a structured risk report.

[0040] Optionally, a tiered response is performed based on the structured risk report to generate revision suggestions, the response results are used to perform version updates on the knowledge graph, and the response results are used to adaptively learn and optimize the large language model, including:

[0041] Analyze the structured risk report to determine the risk value;

[0042] If the risk value is greater than or equal to the first risk threshold, a first response result is determined, which is used to indicate that the corresponding clause is marked as expired and a revision task is generated.

[0043] If the risk value is greater than the second risk threshold and less than the first risk threshold, a second response result is determined, and the second response result is used to indicate the generation of an early warning notification; the first risk threshold is greater than the second risk threshold.

[0044] If the risk value is less than or equal to the second risk threshold, a third response result is determined, which is used to indicate the execution of automatic updates;

[0045] Based on the determined response result, the revision engine is invoked to inject context containing the original reference clause, the state before the change, and the state after the change into the large language model, and a revision suggestion containing evidence anchors is generated.

[0046] Based on the response results, create new version nodes for the knowledge graph, record change logs, and establish a version rollback mechanism.

[0047] After the knowledge graph is updated to a version, the response results are used to adaptively learn and optimize the large language model.

[0048] Optionally, the large language model can be adaptively learned and optimized using the response results, including:

[0049] Receive feedback instructions regarding the proposed revisions, including acceptance, partial acceptance, or rejection;

[0050] The revision suggestions and manual revisions corresponding to the feedback instructions are structured into supervised fine-tuning training sample pairs. Based on the supervised fine-tuning training sample pairs, the parameters of the large language model are fine-tuned.

[0051] Based on the feedback instructions and system operation data, multi-dimensional performance indicators are calculated, and a performance effect report including trend analysis and key indicator evaluation is generated.

[0052] Based on the parameter fine-tuning results and the performance report, the adaptive learning optimization of the large language model is performed.

[0053] Secondly, embodiments of this application provide a processing apparatus for monitoring and updating document semantic consistency, including:

[0054] The first processing module is used to determine the baseline event package corresponding to the publication source of the document using a preset large language model, wherein the baseline event package includes the file corresponding to the document and the metadata associated with the file;

[0055] The second processing module is used to extract the clause-level entity nodes in the file based on the baseline event package and the large language model, construct dependency edges with metadata, and form a knowledge graph of clause-level risk dependencies.

[0056] The third processing module is used to perform periodic monitoring to obtain change events of the document, map the change events to the knowledge graph, calculate the risk score of the affected clauses in the knowledge graph, and determine a structured risk report based on the risk score.

[0057] The fourth processing module is used to perform a graded response based on the structured risk report to generate revision suggestions, obtain the confirmed revision content as the response result, perform version updates on the knowledge graph based on the response result, and use the response result to perform adaptive learning optimization on the large language model.

[0058] Thirdly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the method as described in any one of the first aspects.

[0059] Fourthly, embodiments of this application provide a computer program product, including computer instructions that, when executed by a processor, implement the steps of the method as described in any one of the first aspects.

[0060] Compared with existing technologies, the document semantic consistency monitoring and updating method and apparatus provided in this application utilizes a large model to construct a baseline event package of the document publishing source, extracts clause-level entity nodes and metadata dependency edges, and constructs a clause-level risk dependency knowledge graph. Secondly, document change events are mapped to the graph, risk scores of affected clauses are calculated, and a structured risk report is generated. Based on the report, a tiered response is executed to generate revision suggestions, and confirmed revision content is obtained, which is used both for version updates of the knowledge graph and for adaptive learning optimization of the large language model. This application's solution achieves dynamic closed-loop management of documents, effectively solving the problems of poor timeliness, high semantic risk, and low update efficiency in highly dynamic environments. Attached Figure Description

[0061] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of this application. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0062] Figure 1 A flowchart illustrating a document semantic consistency monitoring and updating method provided in this application embodiment;

[0063] Figure 2 This is a flowchart illustrating the processing of the crawler layer in an embodiment of this application.

[0064] Figure 3 This is a flowchart illustrating the processing of the clause-level risk dependency graph construction layer in an embodiment of this application.

[0065] Figure 4 A flowchart illustrating the semantic change impact propagation and risk assessment layer provided in this application embodiment;

[0066] Figure 5 A flowchart illustrating the intelligent hierarchical response and update layer provided in this application embodiment;

[0067] Figure 6 This is a structural diagram of a document semantic consistency monitoring and updating processing device provided in an embodiment of this application. Detailed Implementation

[0068] The terms "first," "second," etc., used in this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such terms can be used interchangeably where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first" and "second" are generally of the same class, without limiting the number of objects; for example, the first object can be one or more. Furthermore, "or" in this application indicates at least one of the connected objects. For example, "A or B" covers three scenarios: Scenario 1: including A but not B; Scenario 2: including B but not A; Scenario 3: including both A and B. The character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0069] The term "instruction" in this application can be either a direct instruction (or explicit instruction) or an indirect instruction (or implicit instruction). A direct instruction can be understood as one in which the sender explicitly informs the receiver of specific information, the operation to be performed, or the requested result, etc.; an indirect instruction can be understood as one in which the receiver determines the corresponding information based on the instruction sent by the sender, or makes a judgment and determines the operation to be performed or the requested result, etc., based on the judgment result.

[0070] To enable those skilled in the art to better understand the embodiments of the present invention, the limitations of the prior art are first explained in conjunction with specific implementation methods as follows:

[0071] 1. Limited crawling capabilities, passive and inefficient path planning: Existing general-purpose crawlers (such as Scrapy and Selenium) mostly rely on static rules or script simulations, resulting in passive path planning and a lack of proactive reasoning capabilities. This makes it difficult to effectively penetrate multi-level redirects, dynamically loaded scripts, and hidden attachment resources on announcement pages. Furthermore, existing solutions generally lack real-time access mechanisms to multi-source data interfaces or business subscription services, failing to achieve efficient responses within 5 minutes after document content changes. This often leads to omissions in baseline data collection, missing evidence chains, or high false alarm rates in update monitoring.

[0072] 2. Coarse-grained knowledge graph construction, lack of semantic relationship capabilities, and weak cross-modal capabilities: Existing knowledge graph construction technologies mostly remain at the document or chapter level, failing to achieve clause-level entity parsing and disambiguation, and are unable to accurately identify potential risks such as implicit referencing relationships (e.g., "its latest version applies to this document"). Furthermore, existing methods lack a cross-modal knowledge modeling module, making it difficult to structure non-textual elements such as data tables and detection flowcharts; and they generally employ generalized large language models, lacking specific fine-tuning based on general industry-standard texts, leading to biases in semantic recognition within specific industry domains.

[0073] 3. Crude change detection, lacking semantic-level risk propagation and timeliness assessment: Traditional change detection schemes (such as those based on Diff algorithms or PageRank) can only capture surface text differences or topological weights, failing to identify indirect semantic-level effects, and lacking logical conflict detection mechanisms based on Natural Language Inference (NLI). Furthermore, existing systems lack timeliness assessment mechanisms to support the propagation of changes to tables and cross-modal data, making it difficult to quantify latent problems such as obsolete specifications still being cited, definition migrations, or broken citation chains.

[0074] 4. Lagging response mechanisms, lacking intelligent revision, quantitative assessment, and engineering implementation capabilities: Existing intelligent update response mechanisms mostly adopt a passive alarm design, only indicating risks without generating executable revision drafts, and lacking a closed loop of expert feedback and continuous model learning. Furthermore, existing solutions generally lack lightweight expert feedback modules and quantitative performance evaluation dashboards, and are mostly limited to single-machine or small enterprise architectures, making it difficult to meet the needs of large-scale industry applications and multi-system integration.

[0075] As described in the background section, existing technologies suffer from several drawbacks: web crawlers rely on static rules, resulting in passive paths and delayed timeliness; knowledge graphs are coarse-grained, lacking clause-level parsing and cross-modal processing; change detection focuses solely on textual differences, lacking semantic risk propagation and conflict detection; and response mechanisms are passive, lacking intelligent revision and model self-evolution loops. These issues severely restrict the timeliness, accuracy, and compliance of digital governance of multi-source document knowledge bases, leading to high core document maintenance costs and difficulty in identifying potential implicit semantic risks. To address at least one of these problems, embodiments of this application provide a method and apparatus for monitoring and updating document semantic consistency. This can reduce or avoid the occurrence of the above situations, thereby achieving closed-loop management across the entire chain from data collection, clause-level graph construction, risk propagation assessment to intelligent response updates. This ensures the semantic consistency and timeliness of technical specification documents, effectively reducing compliance risks and improving decision-making efficiency.

[0076] This application provides a method and apparatus for monitoring and updating document semantic consistency. The method and apparatus are based on the same concept, and since they address similar problems, their implementations can be mutually referenced; repeated details will not be elaborated further.

[0077] Please refer to Figure 1 This application provides a method for monitoring and updating document semantic consistency, comprising:

[0078] Step 11: Using a pre-defined large-scale language model, determine the baseline event package corresponding to the document's publication source, wherein the baseline event package includes the file corresponding to the document and the metadata associated with the file.

[0079] In this embodiment, the pre-set large-scale language model is a domain-specific optimized model. This model uses a massive dataset of related domain documents and associated files for continuous fine-tuning, and incorporates domain-specific semantic features, dedicated prompt templates, and domain knowledge instruction sets. It can accurately identify core content such as document citation relationships and terminology definitions. The document publication source is an authoritative official website (e.g., URL 1), and the document type is preferably PDF to preserve the original layout and technical parameter information.

[0080] The large-scale language model completes the full crawling of documents in the published source through a self-planned tool call process. The crawled document files and associated metadata (including document target topic, web page address, crawling link, timestamp, etc.) are encapsulated to form a baseline event package, which serves as the basic data for subsequent semantic analysis and change monitoring.

[0081] Step 12: Based on the baseline event package, use the large language model to extract the clause-level entity nodes in the file, construct dependency edges with metadata, and form a knowledge graph of clause-level risk dependencies.

[0082] In this embodiment, the document files in the baseline event package are preprocessed. Using OCR tools and a deep layout analysis algorithm, unstructured PDF documents are converted into structured text. Simultaneously, non-text elements such as tables and flowcharts are extracted and converted into structured nodes. A large-scale language model parses the structured text, identifies and extracts clause-level entity nodes from the file, and generates a corresponding semantic vector representation for each node, used for subsequent semantic similarity analysis and entity linking.

[0083] The system scans structured text, identifies citation relationships between clauses, performs entity linking and disambiguation processing using semantic vector representation, determines dependencies between clause nodes, and generates metadata for dependency edges based on citation relationship types (such as normative citations, term definitions, repeals, revisions, etc.), including citation features (date-added / dateless) and confidence levels. Finally, it integrates clause-level entity nodes with dependency edges containing metadata to construct a knowledge graph of clause-level risk dependencies. The graph is stored using graph databases such as Neo4j, supporting efficient querying and traversal.

[0084] Step 13: Perform periodic monitoring to obtain change events of the document, map the change events to the knowledge graph, calculate the risk score of the affected clauses in the knowledge graph, and determine a structured risk report based on the risk score.

[0085] Here, a large-scale language model-driven monitoring module performs periodic monitoring, connecting to the official RSS subscription, structured data interface, and webhook for detecting webpage changes from the publishing source to achieve rapid capture of change events (triggered within 5 minutes of a change occurring). The captured new announcements or documents are parsed, the target files are downloaded, and their validity is verified through hash comparison and semantic similarity analysis. Valid changed or newly added files are selected and encapsulated as initial change events.

[0086] The initial change event is validated based on document lifecycle rules, and timeliness risk tags are added (such as expired document references, ineffective document references, and document transitional status). The validation results, timeliness risk tags, and initial change event are integrated to generate a complete change event. The change event is mapped to a knowledge graph to locate the corresponding change node and old clause node. Affected upstream clause nodes are collected through reverse dependency traversal, and the risk score for each affected clause is calculated. Consistency verification is performed using a natural language inference model, and finally, a structured risk report is generated.

[0087] Step 14: Execute a tiered response based on the structured risk report to generate revision suggestions, obtain the confirmed revision content as the response result, perform a version update on the knowledge graph based on the response result, and use the response result to perform adaptive learning optimization on the large language model.

[0088] In this application, a tiered response strategy is implemented based on the risk score in the structured risk report: high-risk clauses are marked as expired and a revision task is generated; medium-risk clauses receive an early warning notification; and low-risk clauses are automatically updated. The revision engine is invoked to inject the context of the original cited clauses, their pre-change state, and their post-change state into a large language model, generating revision suggestions containing evidentiary anchors.

[0089] The system obtains feedback instructions (adoption, partial adoption, or rejection) from domain experts regarding revision suggestions. Confirmed revisions are used as the response, and version updates are performed on the knowledge graph, creating new node versions, recording change logs, and establishing a version rollback mechanism. Simultaneously, the revision suggestions corresponding to the feedback instructions and the manually revised content are structured into supervised fine-tuning training sample pairs. Parameters are fine-tuned on large-scale language models, and multi-dimensional performance indicators are calculated based on system runtime data to achieve adaptive learning optimization of the model, improving the accuracy of subsequent processing.

[0090] It should be noted that the solution proposed in this application can be implemented based on an integrated system architecture, which aims to achieve semantic consistency monitoring and intelligent updates. The system includes four interconnected layers: (1) a crawler layer for data collection and change monitoring; (2) a clause-level risk dependency graph construction layer for knowledge structuring and dependency modeling; (3) a semantic change impact propagation and risk assessment layer for change impact analysis and risk quantification; and (4) an intelligent hierarchical response and update layer for automated response and system self-evolution. The entire system is driven by the Agentic Large Language Model (LLM) as its core, combined with knowledge graph technology and semantic analysis algorithms to form a closed-loop process. The system can be deployed in a cloud environment or on a local server, supporting distributed computing to process large-scale data.

[0091] Optionally, step 11 above includes:

[0092] Receive a crawling task request; the crawling task request is used to crawl the file corresponding to the document published by the source.

[0093] Based on the crawling task request, a set of candidate URLs is obtained by using a pre-set large language model to infer the starting point Uniform Resource Locator URL or search keywords.

[0094] The page structure of the pages in the candidate URL set is parsed using the large language model.

[0095] Based on the parsed page structure, the file is located and downloaded, and the file and its associated metadata are encapsulated into a baseline event package; the metadata includes the file's target topic, webpage address, crawling path, and timestamp;

[0096] After determining the baseline event package, a pre-defined large language model is used, and periodic monitoring is performed to obtain change events.

[0097] In this embodiment, the crawling task request is initiated by the user or the system, explicitly specifying the document publishing source (e.g., URL 1) and the target topic (e.g., current documents related to air pollution). The crawling task request can be submitted through system interface input or API call to ensure that the crawling scope and target are clear. Based on the publishing source and target topic in the crawling task request, the large-scale language model autonomously infers the starting URL for crawling (e.g., the homepage of the publishing source or the homepage of a topic-related section), calls the SiteSearch tool to generate search keywords (e.g., "list of current documents related to air pollution"), performs the search operation, and returns a candidate URL set. The candidate URL set contains URLs of search results pages, list pages, detail pages, etc., related to the target topic. The large-scale language model calls the HttpFetch tool to obtain the HTML text of each page in the candidate URL set, performs structural parsing on the HTML text, identifies the page type (search page, list page, detail page, or attachment page), and extracts key structural elements such as pagination information, list items, detail page links, and attachment links from the page, providing a basis for subsequent page traversal and file location.

[0098] Based on the page structure parsing results, generalized iteration and pagination traversal are performed. All pages are traversed, and the HttpFetch tool is called to access the details page for each document entry. If the page contains announcements, multi-level links, or non-direct attachment pages, the large language model dynamically generates subtasks for recursive exploration until the target file (PDF / HTML format) is located.

[0099] The PDFExtractor tool is invoked to identify and download target files, and metadata associated with the files is collected, including the target subject of the file, the source webpage address, the complete crawl link (URL path from the search page to the attachment page), the crawl timestamp, etc. All files and their corresponding metadata are encapsulated to form a baseline event package.

[0100] Once the baseline event package is completed, the system automatically switches to periodic monitoring mode. The monitoring module is driven by a large language model and accesses the announcement or notification section of the publishing source according to a preset period (such as daily or weekly) to capture and verify change events, ensuring that document changes can be detected in a timely manner.

[0101] It should be noted that this application can be implemented based on a crawler layer, which is the system's entry layer. Its principle lies in using Agentic LLM to simulate the planning and decision-making abilities of human experts, overcoming the limitations of static rules in traditional crawlers. Traditional crawlers (such as Scrapy-based rule matching) cannot handle dynamic web pages, JavaScript rendering, or multi-level navigation, resulting in incomplete or uncollected files. This layer introduces a two-stage task mechanism: Stage A is a one-time full baseline construction, ensuring the complete collection of all currently valid official source documents under a specific topic (such as "air pollution"); Stage B is periodic change monitoring, enabling real-time verification and false alarm filtering. Through the LLM's inference chain and tool calls, the Agent can autonomously generate action plans, such as from portal search to pagination iteration, and then to PDF extraction. This design, based on reinforcement learning and cue engineering, makes the system adaptive, capable of handling website structure changes without requiring manual rule adjustments.

[0102] Optionally, after step 11 above, this application further includes: performing periodic monitoring to obtain change events of the document.

[0103] In one specific embodiment, step 11 of this application can be based on Figure 2 The implementation involves the LLM-Agent receiving a dual task prompt upon system startup. Task A (one-time): "Crawl all current valid standards on [specific topic] from [specific website]". Task B: After Task A is completed, it automatically switches to "Periodicly monitor changes on [specific topic] from [specific website]". The Agent uses LLM to generate an initialization plan based on the topic content, including tool call order, retry strategy, and security boundaries (such as the maximum number of jump levels).

[0104] In Phase A, the initial baseline is constructed. Task reasoning and starting point determination: The agent infers the starting point URL (e.g., the homepage of mee.gov.cn) based on topic hints and calls the SiteSearch tool to search for keywords (e.g., "list of current air pollution standards"). The tool returns a set of candidate search result page URLs. Structure identification and content analysis: The agent obtains the page's HTML text via HttpFetch, and the LLM parses the structure (including pagination, list items, detail page links, etc.) to identify the page type (search page, list page, detail page, or attachment page). Generalized iteration and pagination traversal: The agent plans and executes an iterative loop: traversing all pages (a loop counter prevents infinite loops); calling HttpFetch to access the detail page for each standard item. When a page contains announcement jumps, multi-level links, or non-direct attachment pages, the agent dynamically generates subtasks to continue recursively exploring until the termination condition is met (e.g., reaching the maximum number of jump levels or finding the target attachment). Attachment extraction and baseline encapsulation: On the detail page, the agent calls PDFExtractor to identify and download the source file (PDF / HTML), supporting regular expressions and DOM parsing. All downloaded files and associated metadata (including title, source URL, crawl path, timestamp, topic, etc.) are packaged into a "baseline event package". Push and audit logs: The agent pushes the baseline event to the knowledge graph construction layer and records a complete operation log (including tool call sequence and crawl path) to support auditable tracing and subsequent discrepancy verification.

[0105] In Phase B, continuous change monitoring is performed. After the baseline is completed, the Agent automatically switches to periodic monitoring mode (e.g., daily or weekly scheduled execution). The scheduler triggers the Agent to access the announcement or notification section. The Agent calls HttpFetch and LLM to parse the announcement list page, extracting announcement entries and publication times. Based on timestamps, title keywords, or URL hashes, it detects "new announcements" relative to the last access. The Agent accesses the new announcement details page, LLM infers the announcement type (newly released / revised / repealed), and calls PDFExtractor to extract attachment files. If the path hierarchy changes (e.g., multi-level redirects or structural adjustments), the Agent automatically reuses the generalization iteration logic from Phase A to autonomously locate the final PDF.

[0106] For each new file, the Agent calculates a hash value and compares it with the baseline version. If the hashes match, semantic vector similarity is further calculated. Only when a valid change or addition is detected, the Agent encapsulates a "verified change event" and records the complete crawling chain (announcement URL → details URL → PDF URL).

[0107] The engine can perform multi-dimensional date logic verification and automatically generate risk labels in real time during the event verification stage at the crawler layer, combining the release date, implementation date, repeal date, and transition period provisions of each document. The engine first extracts structured metadata from the announcement page, then uses a rule engine and LLM for joint reasoning to automatically identify and label specific time-sensitive risk tags such as "repealed clauses are still cited downstream," "not yet implemented but already cited," and "semantic dual-track coexistence during the transition period." For example, when it detects that an air pollutant emission standard has been repealed, the engine immediately marks all clauses still referencing its old version as "high risk of time-sensitive breakage" and passes this label to the risk propagation layer as a weighting factor.

[0108] By integrating with the government's official RSS subscription, structured data interface, and webhook for detecting website changes, the mechanism automatically triggers local incremental updates and risk recalculation of the risk dependency graph within 5 minutes of a document change. This mechanism avoids the enormous computational overhead of traditional solutions that require full document reconstruction: when a new announcement is detected by RSS feed or webhook, the agent only performs incremental crawling, clause parsing, and edge updates on the affected documents, and then performs differential propagation calculations along the local subgraph of the risk dependency graph. For example, for a newly revised standard published on the Ministry of Environmental Protection's website, only the corresponding clause nodes and associated edges need to be updated, without having to reprocess thousands of historical documents in the entire subject database.

[0109] Network and structural anomaly handling: The system integrates an exponential backoff and retry mechanism to handle network failures; if the website structure changes abruptly, LLM automatically reconstructs the structure parsing prompt and adjusts the iteration path; Loop control and security boundaries: LLM prompts include security constraint instructions to prevent infinite recursion; Each task includes the maximum number of pages, the maximum jump depth, and the maximum runtime limit; Parallelism and throughput optimization: The tool call chain supports parallel execution strategies to improve the efficiency of large-scale crawling.

[0110] In an optional embodiment, step 12 described above includes:

[0111] The files in the baseline event package are subjected to layout analysis and text extraction to convert unstructured documents into structured text.

[0112] The large language model is used to parse structured text, identify and extract clause-level entity nodes in the file, and generate a corresponding semantic vector representation for each node;

[0113] Scan the structured text, identify the reference relationships between clauses, and combine the semantic vector representation to perform entity linking and determine the dependency edges between clause nodes;

[0114] Based on the identified reference relationship type, metadata containing reference features and confidence is generated for the dependency edge, thus generating a dependency edge with metadata;

[0115] The clause-level entity nodes are integrated with the dependency edges with metadata to construct a knowledge graph of clause-level risk dependencies.

[0116] In this embodiment, MinerU enhanced OCR recognition technology and deep layout analysis algorithm are used to segment the PDF file in the baseline event package, handle noise such as watermarks and multi-column layouts, and extract blocks such as titles, paragraphs, tables, and flowcharts from the page. The unstructured PDF document is converted into structured text in block-level Markdown format. At the same time, non-text elements such as tables and flowcharts are initially structured to lay the foundation for subsequent clause extraction and node construction. A large language model parses the structured text, identifies clause-level entities in the file, and each clause-level entity node corresponds to an independent clause (such as clause 5.1 in "GBXX"). Structured data of the nodes is generated, with the specific format as: {"doc_id": "document identifier", "clause_id": "clause identifier", "text": "clause content", "semantic_vector":[semantic vector array], "version": "document version", "timestamp": "timestamp"}.

[0117] Using a pre-defined embedding model, the text content of each clause-level entity node is vector-encoded to generate a corresponding semantic vector representation, which is used to capture the semantic features of the clause and support subsequent entity linking and semantic similarity analysis.

[0118] The system scans structured text and uses a combination of regular expression matching and large-scale language model reasoning to identify citation relationships between clauses, including normative citations, terminological citations, repeals, and revisions.

[0119] By combining the semantic vector representation of clause-level entity nodes, context-aware algorithms such as SimBERT similarity matching are used to perform entity linking and disambiguation. Fuzzy references (such as "see Clause 5") are mapped to the corresponding clause nodes in combination with the context, the dependency relationships between clause nodes are determined, and the starting node (upstream clause) and ending node (downstream clause) of the dependency edge are clarified.

[0120] Based on the identified reference relationship type, corresponding metadata is generated for each dependency edge. The metadata includes reference features (reference_type: dated or undated), relationship type (relation_type), reference text (citation_text), confidence (output by a large language model, usually not less than 0.95), etc., forming dependency edges with metadata, which are used for subsequent risk score calculation.

[0121] By leveraging graph database APIs, clause-level entity nodes and dependency edges with metadata are batch-inserted into the graph database to construct a clause-level risk dependency knowledge graph. The graph employs a directed graph structure, clearly presenting the dependencies and metadata information between clauses, supporting subsequent mapping of change events, risk propagation, and path tracing. After construction, the accuracy of dependency edges is checked through random sampling, and parallel processing techniques are used to optimize the graph construction efficiency for large documents.

[0122] It should be noted that the clause-level risk dependency graph construction layer is based on a fine-grained extension of knowledge graphs. Traditional graphs are only at the document level; this invention introduces clause-level nodes and risk dependency edges to capture indirect semantic risks, particularly "undated citations," which are common in policy standards. The main principle of this layer is to combine NLP entity recognition and link disambiguation to ensure that inconsistent citation expressions are uniformly mapped to the specification ID. The graph adopts a directed graph structure, with nodes as clause entities and edges carrying rich metadata to support downstream risk propagation. Preprocessing uses OCR and existing layout analysis programs to preserve document structure, parsing uses LLM, and edge construction emphasizes the diversity of relationship types, making the graph more semantically deep. This layer solves the disambiguation failure problem of existing technologies and improves accuracy through context-aware algorithms. Graph storage can use Neo4j or graph databases, supporting query optimization.

[0123] It's worth mentioning that this LLM is a dedicated large-scale language model for the policy domain, developed and deployed for further development. This model uses a massive dataset of authoritative policy texts, including national standards, industry standards, and regulations from the Ministry of Environmental Protection, for continuous fine-tuning. Building upon general pre-training, it incorporates specially designed and integrated prompt templates and domain-specific instruction sets for policy-specific semantic features such as "normative citations," "term definition migration," "undated citations," and "implementation date constraints." This dedicated model fundamentally solves the semantic bias problems commonly found in general large-scale language models in non-policy scenarios, such as misinterpreting "emission limits" as ordinary numerical values ​​or ignoring the mandatory logic of "latest version applicable." This improves semantic recognition accuracy to over 95% in clause entity linking and risk dependency edge construction, ensuring the system's professionalism and reliability in complex environmental protection policy environments, and forming a significant technical distinction from existing solutions that rely on general models.

[0124] Simultaneously, in the preprocessing stage of clause-level node parsing, MinerU enhanced OCR recognition technology and deep layout analysis algorithms are integrated to completely transform non-text elements in policy documents, such as emission limit tables, testing method flowcharts, parameter matrices, and appendix charts, into structured graph nodes and establish related edges. This module first segments the PDF layout into blocks, parsing each row of the emission limit table into an independent "LimitEntity" node (containing structured attributes such as pollutant name, limit value, unit, applicable industry, and monitoring conditions). Each step in the testing flowchart is parsed into a sequence of "ProcedureStep" nodes and connected via temporal edges such as "Follows" and "DependsOn." Then, "TableReference" or "FigureReference" links are established between these nodes and the corresponding clause nodes. When table or chart data changes, the system can directly trigger risk propagation and semantic consistency reassessment of downstream clauses along these links. Compared with existing knowledge graph construction schemes that can only process plain text, this invention fundamentally solves the technical problem that a large number of key technical parameters in policy standards are presented in the form of tables and diagrams and cannot be used in a structured way, thus greatly improving the completeness of the graph and the accuracy of risk assessment.

[0125] Reference Figure 3 The process shown involves inputting event packets from the crawler layer, scanning the PDF using an OCR tool, and extracting block-level Markdown (e.g., headings, paragraphs, tables) through layout analysis. Noise such as watermarks or multi-column layouts is handled to ensure clean text.

[0126] The LLM inputs Markdown and structures it into clause nodes. Each node's JSON: {"doc_id": "GB12345-2025", "clause_id": "5.1", "text": "Clause content", "semantic_vector": [array of vectors], "version": "2025", "timestamp": "2025-07-11"}. Vector generation uses an embedding model (such as bge-large-zh-v1.5) to capture semantics.

[0127] Scan the node text to extract citation statements (e.g., regular expression matching "GB / T \d+-\d+ Article \d+.\d+"). Disambiguation algorithm: Rule-based (regex) for handling normative citations; model-based (LLM) for handling ambiguity, such as mapping "See Article 5" to the clause_id of the upstream document in context. Unified into Canonical Clause ID (e.g., gb_t_12345_2020_clause_5_1) and stored in the graph to avoid ambiguity.

[0128] Extract the "normative reference" part and in-text references, and classify the relationship types.

[0129] [[ID=�]]Create edges (u -> v) where u depends on v. Metadata: {"reference_type": "dated" or "undated", "relation_type": "Reference" | "Defines" | "UsesTerm" | "Repeals" | "Amends", "citation_text": "Original text", "confidence": 0.95 (LLM output)}. It should be noted that specific relationship types and semantic examples can be referred to in Table 1 below.

[0130] Table 1:

[0131]

[0132] Graph construction: Use the graph library API to add nodes and edges, supporting batch insertion. Further, verification and optimization: Random sampling to check the edge accuracy; Optimization using parallel processing for large documents.

[0133] This application can output a knowledge graph database by inputting a baseline PDF package into the clause-level risk dependency graph construction layer.

[0134] In an optional embodiment, step 13 above includes:

[0135] Continuous monitoring is performed by accessing data subscription sources or change detection interfaces. The large language model is used to parse the announcement list to identify new announcements and then download the target file from the details page.

[0136] The target file is hashed and compared with the file in the baseline event package. The validity is verified by semantic similarity analysis. The verification result is obtained. The valid changed or added file is encapsulated into an initial change event, and the unchanged file is discarded.

[0137] The timeliness of the initial change event is verified based on the document lifecycle rules, and a timeliness risk label is marked. The timeliness risk label includes at least one of the following: invalid document reference, ineffective document reference, and document transition period status.

[0138] The verification result, the timeliness risk label, and the initial change event are integrated to generate the change event;

[0139] The change event is mapped to the knowledge graph, the risk score of the affected clause in the knowledge graph is calculated, and a structured risk report is determined based on the risk score.

[0140] In this embodiment, the system accesses the official RSS subscription, structured data interface, and webpage change detection webhook of the document publishing source to achieve continuous monitoring. When the publishing source publishes new announcements, revised announcements, or repealed announcements, relevant information is captured in a timely manner. A large language model calls the HttpFetch tool to obtain the announcement list page, parses the announcement list, extracts announcement entries and publication times, and detects new announcements relative to the last visit based on timestamps, title keywords, or URL hashes.

[0141] The large language model accesses the details page of the new announcement, infers the announcement type (newly released, revised, repealed), and calls the PDF Extractor tool to extract the target file (such as the revised document file, the repeal notice file, etc.) from the details page. If there are multiple jumps or page structure adjustments on the details page, the generalization iteration logic in the baseline construction stage is automatically reused to autonomously locate and download the target file.

[0142] The downloaded target file is hashed to obtain its hash value, which is then compared with the hash value of the corresponding file in the baseline event package. If the hash values ​​are different, the semantic vector similarity between the target file and the baseline file is further calculated (cosine similarity is calculated using Sentence-BERT embedding vectors). If the cosine similarity is ≤0.95, it is determined to be a valid change. If the hash values ​​are the same or the cosine similarity is >0.95, it is determined to be unchanged, and the unchanged file is discarded.

[0143] Encapsulate valid changed files or newly added files (files without a corresponding baseline file) into an initial change event. The initial change event includes basic information such as file information, announcement URL, details URL, crawl link, and change type.

[0144] Based on document lifecycle rules, and using large-scale language models and rule engines for joint reasoning, structured metadata (such as release date, implementation date, repeal date, transition period provisions, etc.) is extracted from announcement pages or target files to perform timeliness verification on initial change events.

[0145] Based on the verification results, timeliness risk labels will be automatically marked, including but not limited to: invalid document references (repeated clauses are still referenced by downstream users), ineffective document references (clauses that have not yet been implemented are referenced in advance), and document transition status (new and old clauses coexist semantically during the transition period). Timeliness risk labels will be used as weighting factors for subsequent risk scoring.

[0146] By integrating the validity verification results (valid changes / additions), timeliness risk labels, and basic information of the initial change event, a complete change event is generated. The change event clearly records information such as change type, target file, crawling link, verification results, and timeliness risk labels, providing complete data support for subsequent mapping to the knowledge graph and calculation of risk scores.

[0147] The target files in the change event are preprocessed, parsed, and embedded with vectors to generate new clause-level entity nodes. The corresponding old clause nodes are located in the knowledge graph, and a mapping relationship is established, marking the status of the changed nodes. Affected upstream clause nodes are collected through reverse dependency traversal. A comprehensive risk score is calculated by combining dependency edge metadata and semantic sensitivity. After consistency verification, all relevant information of affected clauses is summarized to generate a structured risk report. The structured risk report includes the affected clause identifier, risk score, impact path, reason for judgment, and recommended actions.

[0148] Furthermore, the change event is mapped to the knowledge graph, a risk score is calculated for the affected clauses in the knowledge graph, and a structured risk report is determined based on the risk score, including:

[0149] The change event is analyzed to determine the change node, and the corresponding old clause node is located in the knowledge graph and a mapping relationship is established.

[0150] Starting from the changed node, perform a reverse traversal along the dependency edges in the knowledge graph to collect the upstream clause nodes affected by the change;

[0151] Read the metadata of the dependent edges between the upstream clause node and the change node, identify the reference relationship type, and determine the basic weight;

[0152] The semantic sensitivity between the changed content corresponding to the changed node and the upstream clause text corresponding to the upstream clause node is calculated, and a comprehensive risk score is obtained by combining the basic weights. The comprehensive risk score is then checked for consistency by using a natural language reasoning model to detect whether there is a logical conflict between the upstream clause text and the changed content. If a conflict exists, the risk score is determined to be the preset highest risk level.

[0153] Summarize the risk scores, impact paths, and reasons for the determination of all affected clauses to generate a structured risk report.

[0154] Reference Figure 4 As shown in this embodiment, the change event is parsed, and the clause-level entity nodes in the target file are extracted as change nodes (v_new). Each change node contains information such as document identifier, clause identifier, changed text, and semantic vector. Based on the document identifier and version number, the corresponding old clause node (v_old) is located in the knowledge graph, establishing a one-to-one mapping relationship between the change node and the old clause node. At the same time, the status of the change node is set to "changed", and metadata such as change timestamp, version difference, and hash value are recorded. A temporary "change source" label is created in the knowledge graph to facilitate subsequent risk propagation tracking.

[0155] Starting from the changed node (v_new), a reverse traversal is performed along the dependency edges (u→v, where u is the upstream clause node and v is the changed node) in the knowledge graph, using either depth-first or breadth-first search, to collect all upstream clause nodes (u) that depend on the changed node. To avoid computational explosion, a maximum propagation depth (e.g., 3 hops) and a lower bound on the impact threshold (e.g., 0.1) are set, and low-relevance paths are pruned in advance, retaining only upstream clause nodes that are highly relevant to the changed node.

[0156] Read the metadata of each dependency edge (u→v), extract the reference relationship type (reference_type, relation_type), and determine the base weight (W_base) according to the preset weight rules. The specific weight rules are as follows:

[0157] 1. If reference_type ==="undated" (date-unspecified reference): W_base = 1.0 (forced update, highest weight);

[0158] 2. If reference_type == "dated" (note: date reference): referenced version == change node version: W_base = 0.0 (not affected by this change); referenced version < change node version: W_base = 0.0 (obsolete reference, to be handled separately);

[0159] 3. If relation_type ==="Defines" (term definition relationship) and the change node has changed: W_base = 0.9;

[0160] 4. If relation_type ==="UsesTerm": W_base = 0.8;

[0161] 5. If relation_type == "Reference" (ordinary reference relationship): W_base = 0.6;

[0162] 6. If relation_type=="Repeals" (repeated relationship): trigger special repeal logic and do not directly assign the base weight.

[0163] Two methods are used to calculate semantic sensitivity S(u,Δv), which can be selected according to the actual application scenario. In the production environment, the vector difference method is used. In high-precision scenarios, the LLM judgment method is used. For example, if a prompt is input to a large language model, the prompt content is: "Term A content: [u.text]; Term B has changed from '[v_old.text]' to '[v_new.text]'; Please determine whether A needs to be adjusted due to the change of B? Output: High / Medium / Low"; the output of the large language model is mapped to semantic sensitivity S: High → 0.9, Medium → 0.6, Low → 0.3.

[0164] For each affected upstream clause node u, risks are aggregated along all propagation paths to obtain a final comprehensive risk score. For high-risk nodes with a comprehensive risk score R(u) > 0.5, a Natural Language Inference (NLI) triple is constructed, including the premise (upstream clause text u.text) and the hypothesis (change node text v_new.text). A pre-trained NLI model is used to determine the relationship between the two: if it is a Contradiction (semantic conflict), R(u) is set to 1.0 (the preset highest risk level), triggering an emergency alarm; if it is an Entailment (logical compatibility), the original value of R(u) is maintained; if it is Neutral (uncertain), R(u) is increased by 0.1.

[0165] If the change node corresponds to a repealed relationship (relation_type=="Repeals"), then all upstream clause nodes that point to the old clause node corresponding to the change node will be automatically marked as "reference invalid", the overall risk score will be set to 1.0, and the "breakpoint" will be marked in the impact path.

[0166] The system aggregates relevant information from all affected clause nodes, including affected clause identifiers, comprehensive risk scores, impact paths, reasons for judgment, and recommended actions, to generate a structured risk report. This structured risk report supports sorting, filtering, and pagination, and can be directly consumed by downstream intelligent tiered response and update layers.

[0167] In one optional embodiment, a tiered response is performed based on the structured risk report to generate revision suggestions, the response results are used to perform version updates on the knowledge graph, and the response results are used to adaptively learn and optimize the large language model, including:

[0168] Analyze the structured risk report to determine the risk value;

[0169] If the risk value is greater than or equal to the first risk threshold, a first response result is determined, which is used to indicate that the corresponding clause is marked as expired and a revision task is generated.

[0170] If the risk value is greater than the second risk threshold and less than the first risk threshold, a second response result is determined, and the second response result is used to indicate the generation of an early warning notification; the first risk threshold is greater than the second risk threshold.

[0171] If the risk value is less than or equal to the second risk threshold, a third response result is determined, which is used to indicate the execution of automatic updates;

[0172] Based on the determined response result, the revision engine is invoked to inject context containing the original reference clause, the state before the change, and the state after the change into the large language model, and a revision suggestion containing evidence anchors is generated.

[0173] Based on the response results, create new version nodes for the knowledge graph, record change logs, and establish a version rollback mechanism.

[0174] After the knowledge graph is updated to a version, the response results are used to adaptively learn and optimize the large language model.

[0175] In this embodiment, the structured risk report is parsed to extract the comprehensive risk score of each affected clause, which serves as the risk value for that clause. The risk value ranges from [0,1], with a higher risk value indicating a greater risk of semantic inconsistency in the clause. Two risk thresholds are preset: a first risk threshold (e.g., 0.9) and a second risk threshold (e.g., 0.5), where the first risk threshold is greater than the second risk threshold. Based on the comparison between the risk value and the threshold, the corresponding response result is determined: If the risk value ≥ the first risk threshold (high risk): a first response result is determined, which instructs the corresponding clause to be marked as expired and generates a revision task, which is then pushed to domain experts for review; If the second risk threshold < the risk value < the first risk threshold (medium risk): a second response result is determined, which instructs the generation of an early warning notification, pushed to relevant personnel to remind them to pay attention and conduct manual verification; If the risk value ≤ the second risk threshold (low risk): a third response result is determined, which instructs the execution of automatic updates, directly updating the clause content and knowledge graph node information without manual intervention.

[0176] Based on the determined response, the revision engine is invoked. The revision engine injects contextual information into the large language model. The contextual information includes the original cited clause text, the pre-change state (old clause text) and post-change state (new clause text) of the change node. Based on the contextual information, the large language model generates revision suggestions containing evidence anchors (the basis for the cited change and semantic relationships). The revision suggestions must conform to the semantics of the clause and ensure consistency with the changed content.

[0177] Based on the response results (confirmed revisions), the knowledge graph is updated in a versioned manner. A Git-like version management mechanism is adopted to create new versions for the affected clause nodes without overwriting the original node versions, ensuring that historical versions are traceable.

[0178] Record detailed change logs, including information such as the person who made the change, the time of the change, the reason for the change, and the content of the change (in the format {who, when, why, what}). At the same time, establish a version rollback mechanism, which supports rolling back to any historical version via API calls to avoid data loss due to accidental operations.

[0179] After the knowledge graph version update is completed, the response results (the revised content and feedback confirmed by experts) will be used as training data to adaptively learn and optimize the large language model, thereby improving the accuracy of subsequent clause parsing, citation recognition, and revision suggestion generation, forming a closed-loop optimization mechanism of "AI suggestions → expert feedback → data accumulation → model fine-tuning".

[0180] Furthermore, the large-scale language model is adaptively optimized using the response results, including:

[0181] Receive feedback instructions regarding the proposed revisions, including acceptance, partial acceptance, or rejection;

[0182] The revision suggestions and manual revisions corresponding to the feedback instructions are structured into supervised fine-tuning training sample pairs. Based on the supervised fine-tuning training sample pairs, the parameters of the large language model are fine-tuned.

[0183] Based on the feedback instructions and system operation data, multi-dimensional performance indicators are calculated, and a performance effect report including trend analysis and key indicator evaluation is generated.

[0184] Based on the parameter fine-tuning results and the performance report, the adaptive learning optimization of the large language model is performed.

[0185] In this embodiment, a lightweight expert feedback module provides domain experts with an intuitive one-click annotation interface. Experts can quickly select one of three feedback instructions for each revision suggestion generated by the system: "adopt", "partially adopt", or "reject". They can fill in the specific reasons for modification and the content of manual revision through the drop-down menu or short text box. The feedback instructions are uploaded to the system in real time.

[0186] The system backend automatically structures the revision suggestions (AI-generated content) corresponding to the feedback instructions and the manually revised content (expert-confirmed content) to form a supervised fine-tuning (SFT) training sample pair (original Prompt + expert revised text). The training sample pair contains key information such as contextual information, AI-generated revision suggestions, expert feedback results, and final revised content.

[0187] Based on the constructed supervised fine-tuning training sample pairs, the parameters of a large language model are fine-tuned, adjusting parameters related to semantic recognition, citation resolution, and revision suggestion generation, so that the model gradually conforms to the judgment logic of domain experts and improves processing accuracy.

[0188] Based on expert feedback (adoption rate, rejection rate, etc.) and system operation data (risk false negative rate, response latency, map update throughput, etc.), multi-dimensional performance indicators are calculated. Core indicators include:

[0189] 1. Draft Amendment Adoption Rate: The percentage of AI-generated revised clauses that were actually adopted by experts or adopted after modification;

[0190] 2. Reduction in manual processing time: The percentage reduction in average processing time compared to a purely manual revision model;

[0191] 3. Risk underreporting rate: The percentage of high-risk clauses that were not identified, as verified by third-party manual sampling.

[0192] 4. System response latency: The average number of minutes from the occurrence of a change to the generation of a complete risk report;

[0193] 5. Graph update throughput: The number of clause nodes successfully processed per minute.

[0194] Based on the above multidimensional performance indicators, a performance effect report is generated. The report includes indicator trend line charts, bar charts comparing with benchmark values, and key KPI analysis, which intuitively present the model optimization effect.

[0195] By combining the model parameter fine-tuning results with the performance report, we analyze the advantages and disadvantages of the model optimization process. If a certain performance indicator fails to meet the preset standard (such as the adoption rate of the revised draft being lower than 80%), we adjust the construction method of the training sample pairs or fine-tune the parameters to continue fine-tuning the model. If all performance indicators meet the preset standard, we complete this adaptive learning optimization and use the fine-tuned model as a new preset large-scale language model for subsequent document semantic consistency monitoring and update processing, forming a closed loop of continuous optimization.

[0196] Specifically, refer to Figure 5 As shown, input a risk report and classify it as R(u). If it is determined to be high-risk, the revision engine is invoked, it is marked as "expired," and a task is pushed out. If it is determined to be medium-risk, a notification is sent; for low-risk: the text is automatically updated, and log records are kept.

[0197] To further optimize the human-machine collaborative closed loop, this application specifically designed a lightweight expert feedback module, providing an intuitive one-click annotation interface. This allows domain experts to quickly select from three options for each draft revision generated by the system: "adopt," "partially modify," or "completely reject," and fill in specific reasons for modification via drop-down menus or short text boxes. The module automatically structures the annotation results into high-quality supervised fine-tuning (SFT) training sample pairs in the background, while simultaneously recording expert modification traces as auditable evidence. Compared to existing solutions with vague feedback mechanisms, low data conversion efficiency, and heavy expert workload, this lightweight module reduces the average feedback time from 5 minutes to less than 30 seconds, significantly increasing expert participation and training data accumulation speed, providing stable and high-quality closed-loop power for continuous model evolution.

[0198] Furthermore, this application constructs a complete quantitative evaluation system, defining core quantifiable indicators such as "draft adoption rate" (the percentage of AI-generated revised clauses actually adopted by experts or adopted after modification), "manual time reduction rate" (the percentage reduction in average processing time compared to the purely manual revision mode), "risk underreporting rate" (the percentage of unidentified high-risk clauses verified by third-party manual sampling), "system response latency" (the average number of minutes from the occurrence of a change to the generation of a complete risk report), and "map update throughput" (the number of clause nodes successfully processed per minute). After each round of adaptive fine-tuning, the system automatically generates a performance report including a trend line chart, a bar chart comparing with the benchmark value, and key KPI analysis, and presents it visually on the management dashboard. This quantitative evaluation system not only ensures that every model optimization can be verified and traced through objective data, but also provides auditable and quantifiable technical support for performance commitments, effect evaluations, and continuous improvement in the subsequent patent implementation stage, which stands in stark contrast to existing feedback mechanisms that lack systematic quantitative indicators.

[0199] In terms of engineering implementation, a highly scalable distributed architecture is adopted, using a Neo4j cluster combined with the Elasticsearch vector storage engine as the core storage layer to achieve sub-second query, traversal, and risk propagation calculation for hundreds of thousands of clause nodes. This architecture effectively solves the problems of single point of failure, query bottlenecks, and memory overflow faced by existing solutions in large-scale data scenarios through sharding and replication mechanisms. It supports horizontal scaling to dozens of nodes to adapt to the scale requirements of a national-level policy database. For example, when processing over 100,000 historical standards in the environmental protection field, the system can still maintain a single risk propagation query response time of less than 1 second. Simultaneously, a complete multi-terminal adaptation interface system has been developed, including a web-based fine-grained review platform, a mobile real-time alert APP, and standardized government API interfaces, which can be directly and seamlessly integrated into existing government information systems and enterprise compliance management systems. This system supports various use cases, such as "reviewing complex draft revisions and receiving expert feedback on desktop", "receiving high-risk instant push notifications and one-click confirmation on mobile devices", and "achieving data interoperability with higher-level regulatory platforms via government APIs". Compared with existing solutions that are limited to enterprise-level stand-alone or local area network deployments and lack government adaptation capabilities, this application has stronger cross-platform compatibility, user-friendliness, and promotion and implementation value, and can truly meet the actual needs of government departments and large enterprises for intelligent management of policies and standards.

[0200] This application has at least the following advantages over the prior art:

[0201] (1) Improved completeness and timeliness: Through Agent autonomous planning and deep retrieval, the penetration and update defects of traditional crawlers are overcome, ensuring the integrity of baseline data and the accuracy of monitoring results, and significantly reducing information omissions and resource waste.

[0202] (2) Precise semantic modeling: The clause-level knowledge graph is used to encode complex semantic logic such as "no date" and combined with the propagation algorithm to realize the quantitative analysis of semantic impact, effectively identifying and avoiding potential implicit semantic risks.

[0203] (3) Intelligent closed-loop update: The system realizes full-process automation from alarm discovery to draft revision generation, human-machine collaboration accelerates the dynamic update of policies and standards, and continuously optimizes AI decision-making capabilities through adaptive learning.

[0204] (4) Auditability and security assurance: Construct a traceable chain of evidence and a versioned storage mechanism to support historical backtracking and difference comparison, and meet the strict requirements of high-compliance industries for security and auditability.

[0205] (5) Policy domain specificity and multimodal processing capability: By adopting a dedicated fine-tuning model and cross-modal modeling technology, it accurately captures policy semantic features and non-textual key parameters, achieving an essential difference from existing general technologies, greatly reducing semantic bias and information loss risks, and providing unprecedented professional protection for high compliance fields such as environmental protection.

[0206] (6) Real-time performance, quantitative assessability and engineering implementation capability: Through real-time dynamic updates, policy timeliness engine, quantitative assessment dashboard and multi-terminal government affairs adaptation, a minute-level intelligent closed loop is formed, and objective and auditable performance indicators are provided to ensure that the system can operate efficiently, stably and scalably in actual national-level application scenarios.

[0207] The various methods of the embodiments of this application have been described above. Apparatus for implementing the above methods will now be provided.

[0208] Please refer to Figure 6 This application also provides a processing apparatus for monitoring and updating document semantic consistency, comprising:

[0209] The first processing module 61 is used to determine the baseline event package corresponding to the publication source of the document using a preset large language model, wherein the baseline event package includes the file corresponding to the document and the metadata associated with the file;

[0210] The second processing module 62 is used to extract the clause-level entity nodes in the file based on the baseline event package and the large language model, construct dependency edges with metadata, and form a knowledge graph of clause-level risk dependencies.

[0211] The third processing module 63 is used to perform periodic monitoring to obtain change events of the document, map the change events to the knowledge graph, calculate the risk score of the affected clauses in the knowledge graph, and determine a structured risk report based on the risk score.

[0212] The fourth processing module 64 is used to perform a graded response based on the structured risk report to generate revision suggestions, obtain the confirmed revision content as the response result, perform versioning updates on the knowledge graph based on the response result, and use the response result to perform adaptive learning optimization on the large language model.

[0213] Optionally, the first processing module 61 described above is specifically used for:

[0214] Receive a crawling task request; the crawling task request is used to crawl the file corresponding to the document published by the source.

[0215] Based on the crawling task request, a set of candidate URLs is obtained by using a pre-set large language model to infer the starting point Uniform Resource Locator URL or search keywords.

[0216] The page structure of the pages in the candidate URL set is parsed using the large language model.

[0217] Based on the parsed page structure, the file is located and downloaded, and the file and its associated metadata are encapsulated into a baseline event package; the metadata includes the file's target topic, webpage address, crawling path, and timestamp;

[0218] After determining the baseline event package, a pre-defined large language model is used, and periodic monitoring is performed to obtain change events.

[0219] Optionally, the second processing module 62 described above is specifically used for:

[0220] The files in the baseline event package are subjected to layout analysis and text extraction to convert unstructured documents into structured text.

[0221] The large language model is used to parse structured text, identify and extract clause-level entity nodes in the file, and generate a corresponding semantic vector representation for each node;

[0222] Scan the structured text, identify the reference relationships between clauses, and combine the semantic vector representation to perform entity linking and determine the dependency edges between clause nodes;

[0223] Based on the identified reference relationship type, metadata containing reference features and confidence is generated for the dependency edge, thus generating a dependency edge with metadata;

[0224] The clause-level entity nodes are integrated with the dependency edges with metadata to construct a knowledge graph of clause-level risk dependencies.

[0225] Optionally, the third processing module 63 described above is specifically used for:

[0226] Continuous monitoring is performed by accessing data subscription sources or change detection interfaces. The large language model is used to parse the announcement list to identify new announcements and then download the target file from the details page.

[0227] The target file is hashed and compared with the file in the baseline event package. The validity is verified by semantic similarity analysis. The verification result is obtained. The valid changed or added file is encapsulated into an initial change event, and the unchanged file is discarded.

[0228] The timeliness of the initial change event is verified based on the document lifecycle rules, and a timeliness risk label is marked. The timeliness risk label includes at least one of the following: invalid document reference, ineffective document reference, and document transition period status.

[0229] The verification result, the timeliness risk label, and the initial change event are integrated to generate the change event;

[0230] The change event is mapped to the knowledge graph, the risk score of the affected clause in the knowledge graph is calculated, and a structured risk report is determined based on the risk score.

[0231] Optionally, the fourth processing module 64 described above is specifically used for:

[0232] The change event is analyzed to determine the change node, and the corresponding old clause node is located in the knowledge graph and a mapping relationship is established.

[0233] Starting from the changed node, perform a reverse traversal along the dependency edges in the knowledge graph to collect the upstream clause nodes affected by the change;

[0234] Read the metadata of the dependent edges between the upstream clause node and the change node, identify the reference relationship type, and determine the basic weight;

[0235] The semantic sensitivity between the changed content corresponding to the changed node and the upstream clause text corresponding to the upstream clause node is calculated, and a comprehensive risk score is obtained by combining the basic weights. The comprehensive risk score is then checked for consistency by using a natural language reasoning model to detect whether there is a logical conflict between the upstream clause text and the changed content. If a conflict exists, the risk score is determined to be the preset highest risk level.

[0236] Summarize the risk scores, impact paths, and reasons for the determination of all affected clauses to generate a structured risk report.

[0237] Optionally, the fourth processing module 64 described above is also specifically used for:

[0238] Analyze the structured risk report to determine the risk value;

[0239] If the risk value is greater than or equal to the first risk threshold, a first response result is determined, which is used to indicate that the corresponding clause is marked as expired and a revision task is generated.

[0240] If the risk value is greater than the second risk threshold and less than the first risk threshold, a second response result is determined, and the second response result is used to indicate the generation of an early warning notification; the first risk threshold is greater than the second risk threshold.

[0241] If the risk value is less than or equal to the second risk threshold, a third response result is determined, which is used to indicate the execution of automatic updates;

[0242] Based on the determined response result, the revision engine is invoked to inject context containing the original reference clause, the state before the change, and the state after the change into the large language model, and a revision suggestion containing evidence anchors is generated.

[0243] Based on the response results, create new version nodes for the knowledge graph, record change logs, and establish a version rollback mechanism.

[0244] After the knowledge graph is updated to a version, the response results are used to adaptively learn and optimize the large language model.

[0245] Optionally, the fourth processing module 64 described above is also specifically used for:

[0246] Receive feedback instructions regarding the proposed revisions, including acceptance, partial acceptance, or rejection;

[0247] The revision suggestions and manual revisions corresponding to the feedback instructions are structured into supervised fine-tuning training sample pairs. Based on the supervised fine-tuning training sample pairs, the parameters of the large language model are fine-tuned.

[0248] Based on the feedback instructions and system operation data, multi-dimensional performance indicators are calculated, and a performance effect report including trend analysis and key indicator evaluation is generated.

[0249] Based on the parameter fine-tuning results and the performance report, the adaptive learning optimization of the large language model is performed.

[0250] It should be noted that the device in this embodiment corresponds to the method described above, and the implementation methods in each of the above embodiments are applicable to the embodiments of this device, achieving the same technical effect. The device provided in this application embodiment can implement all the method steps implemented in the above method embodiments and achieve the same technical effect. Therefore, the parts and beneficial effects that are the same as those in the method embodiments will not be described in detail here.

[0251] This application also provides a computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements the various processes of the document semantic consistency monitoring and updating method embodiments described above, and achieves the same technical effect. To avoid repetition, it will not be described again here. The computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc.

[0252] This application also provides a computer program product, including computer instructions. When the computer instructions are executed by a processor, they implement the various processes of the above-described document semantic consistency monitoring and updating processing method embodiment, and can achieve the same technical effect. To avoid repetition, they will not be described again here.

[0253] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0254] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0255] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A method for monitoring and updating document semantic consistency, characterized in that, include: Using a pre-defined large-scale language model, a baseline event package corresponding to the publication source of the document is determined, wherein the baseline event package includes the file corresponding to the document and the metadata associated with the file; Based on the baseline event package, the large language model is used to extract the clause-level entity nodes in the file, construct dependency edges with metadata, and form a knowledge graph of clause-level risk dependencies. Perform periodic monitoring to obtain change events of the document, map the change events to the knowledge graph, calculate the risk score of the affected clauses in the knowledge graph, and determine a structured risk report based on the risk score; Based on the structured risk report, a tiered response is executed to generate revision suggestions. Confirmed revisions are obtained as response results. The knowledge graph is then updated in a versioned manner based on the response results. Finally, the large language model is adaptively optimized using the response results.

2. The method according to claim 1, characterized in that, Using a pre-defined large-scale language model, determine the baseline event package corresponding to the document's publication source, including: Receive a crawling task request; the crawling task request is used to crawl the file corresponding to the document published by the source. Based on the crawling task request, a set of candidate URLs is obtained by using a pre-set large language model to infer the starting point Uniform Resource Locator URL or search keywords. The page structure of the pages in the candidate URL set is parsed using the large language model. Based on the parsed page structure, the file is located and downloaded, and the file and its associated metadata are encapsulated into a baseline event package; the metadata includes the file's target topic, webpage address, crawling path, and timestamp; After determining the baseline event package, a pre-defined large language model is used, and periodic monitoring is performed to obtain change events.

3. The method according to claim 1, characterized in that, Based on the baseline event package, the large language model is used to extract the clause-level entity nodes in the file, construct dependency edges with metadata, and form a knowledge graph of clause-level risk dependencies, including: The files in the baseline event package are subjected to layout analysis and text extraction to convert unstructured documents into structured text. The large language model is used to parse structured text, identify and extract clause-level entity nodes in the file, and generate a corresponding semantic vector representation for each node; Scan the structured text, identify the reference relationships between clauses, and combine the semantic vector representation to perform entity linking and determine the dependency edges between clause nodes; Based on the identified reference relationship type, metadata containing reference features and confidence is generated for the dependency edge, thus generating a dependency edge with metadata; The clause-level entity nodes are integrated with the dependency edges with metadata to construct a knowledge graph of clause-level risk dependencies.

4. The method according to claim 1, characterized in that, Perform periodic monitoring to obtain change events of the document, map the change events to the knowledge graph, calculate the risk score of the affected clauses in the knowledge graph, and determine a structured risk report based on the risk score, including: Continuous monitoring is performed by accessing data subscription sources or change detection interfaces. The large language model is used to parse the announcement list to identify new announcements and then download the target file from the details page. The target file is hashed and compared with the file in the baseline event package. The validity is verified by semantic similarity analysis. The verification result is obtained. The valid changed or added file is encapsulated into an initial change event, and the unchanged file is discarded. The timeliness of the initial change event is verified based on the document lifecycle rules, and a timeliness risk label is marked. The timeliness risk label includes at least one of the following: invalid document reference, ineffective document reference, and document transition period status. The verification result, the timeliness risk label, and the initial change event are integrated to generate the change event; The change event is mapped to the knowledge graph, the risk score of the affected clause in the knowledge graph is calculated, and a structured risk report is determined based on the risk score.

5. The method according to claim 1 or 4, characterized in that, Mapping the change events to the knowledge graph, calculating the risk score of the affected clauses in the knowledge graph, and determining a structured risk report based on the risk score, including: The change event is analyzed to determine the change node, and the corresponding old clause node is located in the knowledge graph and a mapping relationship is established. Starting from the changed node, perform a reverse traversal along the dependency edges in the knowledge graph to collect the upstream clause nodes affected by the change; Read the metadata of the dependent edges between the upstream clause node and the change node, identify the reference relationship type, and determine the basic weight; The semantic sensitivity between the changed content corresponding to the changed node and the upstream clause text corresponding to the upstream clause node is calculated, and a comprehensive risk score is obtained by combining the basic weights. The comprehensive risk score is then checked for consistency by using a natural language reasoning model to detect whether there is a logical conflict between the upstream clause text and the changed content. If a conflict exists, the risk score is determined to be the preset highest risk level. Summarize the risk scores, impact paths, and reasons for the determination of all affected clauses to generate a structured risk report.

6. The method according to claim 1, characterized in that, Based on the structured risk report, a tiered response is executed to generate revision suggestions. The response results are used to perform version updates on the knowledge graph, and the response results are used to adaptively learn and optimize the large language model, including: Analyze the structured risk report to determine the risk value; If the risk value is greater than or equal to the first risk threshold, a first response result is determined, which is used to indicate that the corresponding clause is marked as expired and a revision task is generated. If the risk value is greater than the second risk threshold and less than the first risk threshold, a second response result is determined, and the second response result is used to indicate the generation of an early warning notification; the first risk threshold is greater than the second risk threshold. If the risk value is less than or equal to the second risk threshold, a third response result is determined, which is used to indicate the execution of automatic updates; Based on the determined response result, the revision engine is invoked to inject context containing the original reference clause, the state before the change, and the state after the change into the large language model, and a revision suggestion containing evidence anchors is generated. Based on the response results, create new version nodes for the knowledge graph, record change logs, and establish a version rollback mechanism. After the knowledge graph is updated to a version, the response results are used to adaptively learn and optimize the large language model.

7. The method according to claim 1 or 6, characterized in that, Adaptive learning optimization of the large language model using the response results includes: Receive feedback instructions regarding the proposed revisions, including acceptance, partial acceptance, or rejection; The revision suggestions and manual revisions corresponding to the feedback instructions are structured into supervised fine-tuning training sample pairs. Based on the supervised fine-tuning training sample pairs, the parameters of the large language model are fine-tuned. Based on the feedback instructions and system operation data, multi-dimensional performance indicators are calculated, and a performance effect report including trend analysis and key indicator evaluation is generated. Based on the parameter fine-tuning results and the performance report, the adaptive learning optimization of the large language model is performed.

8. A processing apparatus for monitoring and updating document semantic consistency, characterized in that, include: The first processing module is used to determine the baseline event package corresponding to the publication source of the document using a preset large language model, wherein the baseline event package includes the file corresponding to the document and the metadata associated with the file; The second processing module is used to extract the clause-level entity nodes in the file based on the baseline event package and the large language model, construct dependency edges with metadata, and form a knowledge graph of clause-level risk dependencies. The third processing module is used to perform periodic monitoring to obtain change events of the document, map the change events to the knowledge graph, calculate the risk score of the affected clauses in the knowledge graph, and determine a structured risk report based on the risk score. The fourth processing module is used to perform a graded response based on the structured risk report to generate revision suggestions, obtain the confirmed revision content as the response result, perform version updates on the knowledge graph based on the response result, and use the response result to perform adaptive learning optimization on the large language model.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the method as described in any one of claims 1 to 7.

10. A computer program product, characterized in that, Includes computer instructions that, when executed by a processor, implement the steps of the method as described in any one of claims 1 to 7.

Images