Multi-engine cooperative official document review method and system, and related device

Abstract

The application provides a kind of multi-engine cooperation's official document review method, system and related equipment, wherein, method includes first to official document structured analysis, mapping physical style attribute as structured semantic node;Then cooperatively executes three levels of check: first level deterministic rule check, for quickly screening format and punctuation error;Second level fact knowledge check, verify the accuracy of entity information with knowledge base;Third level semantic logic check, utilize large language model to process complex semantic and logic problem, when executing semantic logic check, the application constructs prompt word dynamically and injects the reasoning process of large language model by taking the results of the first two levels of check as hard constraint, can effectively solve the problem of rigid rules and model illusion in prior art, on the basis of guaranteeing format and fact accuracy, realize the identification of deep logic error, improve the quality and efficiency of official document review.

Description

Technical Field

[0001] This invention relates to the field of electronic government technology, and in particular to a multi-engine collaborative document review method, system, and related equipment. Background Technology

[0002] In the field of electronic government processing, the standardization, accuracy, and compliance of official documents are core requirements. Existing document review and verification technologies mainly fall into two categories, but both have significant shortcomings: 1. Traditional rule-based matching techniques: These techniques mainly rely on keyword matching, regular expressions, or simple syntax tree analysis (such as early Word plugins). Although these techniques are fast, they have a very high false alarm rate, cannot understand the contextual semantics, and are completely powerless against complex government statements and logical consistency (such as time inversion).

[0003] 2. Technology based on pure generative large model: Directly using the general large language model (LLM) for end-to-end proofreading, although it has strong semantic understanding capabilities, has serious "illusion" problems. It is easy to tamper with factual data in the original text (such as personal names, place names, and document numbers), and it is unexplainable and uncontrollable. It lacks deterministic constraints on the strict format of official documents (such as document standard GB / T 9704) and sensitive words in government affairs, and cannot meet the rigor requirements of government affairs scenarios.

[0004] In summary, existing document verification schemes cannot simultaneously address deterministic format constraints, factual data accuracy, and deep semantic logic rationality. Therefore, there is an urgent need to develop a hybrid document verification method that integrates rule determinism and model intelligence. Summary of the Invention

[0005] This invention provides a multi-engine collaborative document review method, system, and related equipment. The main purpose of this invention is to solve the technical problems mentioned in the background art of the prior art.

[0006] The first aspect of this invention provides a multi-engine collaborative document review method, comprising: The document to be reviewed is structured and parsed to identify and map the physical style attributes in the document to structured semantic nodes. The first level of deterministic rule verification is performed, and the structured semantic nodes are screened for format and punctuation errors based on preset format specifications and regular expressions. Perform a second-level factual knowledge verification, based on an external knowledge base, to verify the entities in the official document; Based on at least one result of the first-level deterministic rule verification and the second-level factual knowledge verification, prompt words containing constraint information are dynamically constructed, and a large language model is invoked to perform a third-level semantic logic verification to identify complex semantic, logical, or stylistic errors in the official document. The results of the first-level deterministic rule verification, the second-level factual knowledge verification, and the third-level semantic logic verification are aggregated and output to the user interface.

[0007] In an optional embodiment of the first aspect of the present invention, the step of performing structured parsing on the official document to be reviewed, and identifying and mapping the physical style attributes in the official document to structured semantic nodes, includes: The document's physical style attributes, including the font, font size, indentation, and alignment of each paragraph, are analyzed. Based on the physical style attributes, the unstructured text paragraphs of the official document are mapped into structured semantic nodes including titles, body text, and signatures, so as to provide contextual basis for the first-level deterministic rule verification.

[0008] In an optional embodiment of the first aspect of the present invention, the multi-engine collaborative document review method further includes a short-circuit mechanism: After the first-level deterministic rule verification is completed, it is determined whether a preset fundamental format error has been detected; If the fundamental format error is detected, the subsequent second-level factual knowledge verification and the third-level semantic logic verification are terminated, and only a format error report is output.

[0009] In an optional embodiment of the first aspect of the present invention, the aggregation of the results of the first-level deterministic rule verification, the second-level factual knowledge verification, and the third-level semantic logic verification, and the output to the user interface, includes: Receive multiple verification result fragments asynchronously returned by the first-level deterministic rule verification, the second-level factual knowledge verification, and the third-level semantic logic verification in a streaming data format; Multiple verification result fragments are reassembled in real time into a structured verification object containing error type, in-situ coordinates, and modification suggestions; Based on the document index mapping table established by the structured parsing of the official document, the original coordinates of the structured verification object are reversed to the character-level position of the official document, so as to realize the real-time highlighting of the review results.

[0010] In an optional embodiment of the first aspect of the present invention, the step of performing second-level factual knowledge verification, based on an external knowledge base, to perform factual verification of entities in the official document includes: Receive verification requests carrying a verification timestamp; From one or more multi-version external knowledge bases that are bound to effective time and expiration time, match and call the valid knowledge set corresponding to the verification timestamp; Based on the aforementioned effective knowledge set, the spatiotemporal accuracy of the organization names, laws and regulations, or personal entities in the official documents is verified.

[0011] In an optional embodiment of the first aspect of the present invention, the third-level semantic logic verification further includes tiered processing of the verification results: By embedding metacognitive instructions into the prompt words, the large language model can output a confidence score for the modification suggestions while simultaneously outputting modification suggestions. Based on the preset error risk level and the confidence score, a differentiated handling strategy is automatically matched. The handling strategy includes automatic correction, suggestive prompts, strong blocking prompts, or prohibition of automatic correction.

[0012] In an optional embodiment of the first aspect of the present invention, when the second-level factual knowledge verification involves multiple person entities arranged in parallel, the second-level factual knowledge verification further includes: Using dependency parsing, a sequence of parallel person entities is extracted from the official document. From the government knowledge graph, retrieve and obtain the level or ranking weight corresponding to each entity in the sequence of person entities; The sequence of the individuals is traversed, and the weights of adjacent individuals are compared to detect whether there are inversion pairs, so as to determine whether the ranking of the individuals is compliant.

[0013] A second aspect of the present invention provides a multi-engine collaborative document review system, the multi-engine collaborative document review system comprising: The document parsing module is used to perform structured parsing of official documents to be reviewed, and to identify and map the physical style attributes in the official documents to structured semantic nodes; The first-level verification module is used to perform first-level deterministic rule verification, which screens the structured semantic nodes for format and punctuation errors based on preset format specifications and regular expressions. The secondary verification module is used to perform the second-level factual knowledge verification, which verifies the entities in the official document based on an external knowledge base. The three-level verification module is used to dynamically construct prompt words containing constraint information based on at least one result of the first-level deterministic rule verification and the second-level factual knowledge verification, and call the large language model to perform the third-level semantic logic verification to identify complex semantic, logical or stylistic errors in the official document. The result aggregation module is used to aggregate the results of the first-level deterministic rule verification, the second-level factual knowledge verification, and the third-level semantic logic verification, and output them to the user interface.

[0014] A third aspect of the present invention provides a multi-engine collaborative document review device, the multi-engine collaborative document review device comprising: a memory and at least one processor, wherein the memory stores instructions, and the memory and the at least one processor are interconnected via a line; The at least one processor invokes the instructions in the memory to cause the multi-engine collaborative document review device to execute the multi-engine collaborative document review method as described in any one of the first aspects of the present invention.

[0015] A fourth aspect of the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the multi-engine collaborative document review method as described in any one of the first aspects of the present invention.

[0016] Beneficial Effects: This invention provides a multi-engine collaborative document review method, system, and related equipment. The method includes first performing structured parsing on the document, mapping physical style attributes to structured semantic nodes; then collaboratively performing three levels of verification: the first level is deterministic rule verification, used to quickly screen for format and punctuation errors; the second level is factual knowledge verification, using a knowledge base to verify the accuracy of entity information; and the third level is semantic logic verification, using a large language model to handle complex semantic and logical problems. When performing semantic logic verification, this invention uses the results of the first two levels of verification as hard constraints, dynamically constructs prompt words, and injects them into the reasoning process of the large language model. This effectively solves the problems of rigid rules and model illusion in existing technologies, and while ensuring the accuracy of format and facts, it achieves the identification of deep logical errors, improving the quality and efficiency of document review. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of an embodiment of the multi-engine collaborative document review method of the present invention; Figure 2 This is a schematic diagram illustrating an embodiment of the overall architecture and end-to-end data flow of an intelligent document review system according to the present invention; Figure 3 This is a schematic diagram of an embodiment of the logic for entity consistency detection and government ranking verification based on government knowledge graph of the present invention; Figure 4 This is a schematic diagram of an embodiment of the principle of spatiotemporal logic graph construction and temporal logic conflict detection of the present invention; Figure 5This is a schematic diagram of an embodiment of the structure of dynamic assembly of large model prompt words and defensive semantic review according to the present invention; Figure 6 This is a schematic diagram of an embodiment of the multi-engine collaborative document review system of the present invention; Figure 7 This is a schematic diagram of an embodiment of a multi-engine collaborative document review device of the present invention. Detailed Implementation

[0018] The terms "first," "second," "third," "fourth," etc. (if present) in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" or "having" and any variations thereof are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0019] See Figure 1 The first aspect of this invention provides a multi-engine collaborative document review method, comprising: S100. The document to be reviewed is parsed in a structured manner, and the physical style attributes in the document are identified and mapped to structured semantic nodes. In this invention, before the review begins, the system first parses the unstructured .docx document into structured data, thus providing the basis for all subsequent rule validations.

[0020] In an optional embodiment of the first aspect of the present invention, the step of performing structured parsing on the official document to be reviewed, and identifying and mapping the physical style attributes in the official document to structured semantic nodes, includes: Parse the physical style attributes including the font, font size, indentation, and alignment of each paragraph in the official document. Taking a DOC document named "Notice on Holding the City-wide Digital Transformation Training Conference" as an example, in this step, the system calls a document parsing library (such as a parser based on the DOM model), traverses each paragraph of the official document, and for each paragraph, extracts detailed physical style attributes. For example, for the first paragraph: "Notice on Holding the City-wide Digital Transformation Training Conference", the parsed attributes are: {font: Fangzheng Xiaobiao Song_GBK, font size:二号, alignment: centered,...}, for the second paragraph: "All XXX units:", the parsed attributes are: {font: Fangsong_GB2312, font size:三号, alignment: left-aligned, left indent: 2 characters,...}, for the third paragraph: "1. Training content", the parsed attributes are: {font:黑体, font size:三号, first line indent: 2 characters, text: 1...}.

[0021] Based on the physical style attributes, map the unstructured text paragraphs of the official document to structured semantic nodes including titles, main texts, and signatures, to provide a context basis for the first-level deterministic rule verification. In this step, the system maps the physical style attributes obtained in the previous step to logically semantic nodes based on preset heuristic rules. For example, the first paragraph is mapped to {type: main title, content:...} due to its typical features of "centered" and "二号" font size, the second paragraph is mapped to {type: addressee unit, content:...}, and the third paragraph is mapped to {type: secondary title, content:...} because its text starts with "1." and is in bold.

[0022] The process of step S100 of the present invention provides a basis for all subsequent context-based verifications. At the same time, the system will also establish an index mapping table for each character, recording its absolute position in the original document.

[0023] S200. Perform the first-level deterministic rule verification, and screen for format and punctuation errors in the structured semantic nodes based on preset format specifications and regular expressions. In the present invention, after the parsing in step S100 is completed, the first-level verification engine is started. The first-level verification engine will apply hard-coded rules and regular expressions to the semantic nodes mapped in step S100 to quickly screen for basic and deterministic errors such as format, punctuation, and measurement units.

[0024] Format check: For example, it is found that the secondary title "1. Training content" in the third paragraph uses a half-width full stop ".", while the national official document standard GB / T 9704 requires the use of a full-width full stop ".". The system will immediately generate a low-risk format error record.

[0025] Punctuation checking: For example, if two consecutive periods "." are detected in the body of a document, the system will generate a punctuation misuse error.

[0026] In the above example, the document structure is clear and can be successfully mapped. However, in another scenario, if the uploaded document has a messy format, causing step S100 to fail to identify any valid title or body structure, the document parsing module will return a "structure parsing failed" flag. In this case, the system will trigger a short-circuit mechanism, directly terminating the subsequent steps and returning a high-priority error to the front end, such as "the document format is seriously incorrect and cannot be intelligently reviewed," thereby avoiding expensive model inference for documents that cannot be processed. That is, in an optional embodiment of the first aspect of the present invention, the multi-engine collaborative document review method further includes a short-circuit mechanism: after the first-level deterministic rule verification is completed, it is determined whether a preset fundamental format error is detected; if the fundamental format error is detected, the subsequent second-level factual knowledge verification and the third-level semantic logic verification are terminated, and only a format error report is output.

[0027] S300. Perform the second-level factual knowledge verification. Based on an external knowledge base, perform factual verification on the entities in the official document. In this invention, after passing the first-level verification, the system will initiate the second-level verification. In the second-level verification, the system will use a NER model pre-trained in the government affairs domain to extract entities from the text, such as institutional entities (e.g., Municipal Bureau XXX), personal entities (e.g., Zhang San, Li Si), and legal entities (e.g., "XX City Big Data Development Regulations"). Then, factual verification will be performed on these entities.

[0028] In an optional embodiment of the first aspect of the present invention, the step of performing second-level factual knowledge verification, based on an external knowledge base, to perform factual verification of entities in the official document includes: Receive a verification request with a verification timestamp; in this step, assuming the verification request is from October 26, 2023, the request will carry the timestamp.

[0029] From one or more external knowledge bases with multiple versions bound to their effective and expiration dates (supporting hot updates and version management), the system matches and retrieves the valid knowledge set corresponding to the verification timestamp. In this step, when verifying the legal entity "XX City Big Data Development Regulations", the system queries the internal legal knowledge base, which records multiple versions of the regulations (the system can not only verify the current document with the latest rules, but also call the historical rule base to retrospectively verify the compliance of old documents, possessing the ability to travel through time). Each version is bound to its effective and expiration dates. If the system finds that a new version of the regulations has been in effect since September 1, 2023, the system will generate a high-risk error: "The cited regulations are not the latest version. Please confirm whether you need to update to the latest version." After the user confirms the use of the latest version, the valid "XX City Big Data Development Regulations" is obtained as the valid knowledge set.

[0030] Based on the aforementioned effective knowledge set, the spatiotemporal accuracy of the organization names, laws and regulations, or personal entities in the official documents is verified. In this invention, the verification content of factual knowledge mainly includes the standardization of organization names, the validity of laws and regulations, and the correctness of the order of personal names and positions. In this step, a "whitelist / blacklist" mechanism can be adopted to prevent the "illusion" of subsequent large models and provide hard constraints on factual content.

[0031] In an optional embodiment of the first aspect of the present invention, for the verification of person entities, when the second-level factual knowledge verification involves multiple person entities arranged in parallel, the second-level factual knowledge verification further includes: Using dependency parsing, a sequence of parallel person entities is extracted from the official document. In this step, assuming that the document contains the sentence "After research and decision by XX, Comrade Zhang San and Comrade Li Si will be jointly responsible for this work", the system identifies that "Zhang San" and "Li Si" constitute a parallel sequence of person entities S=[Zhang San, Li Si].

[0032] From the government knowledge graph, retrieve and obtain the level or ranking weight corresponding to each entity in the sequence of person entities; in this step, the system will query the government knowledge graph to obtain the weight vectors of the two people, for example W (Zhang San) = {Rank: 2}, W (Li Si) = {Rank: 1}.

[0033] The system iterates through the sequence of person entities and checks for inversion pairs by comparing the weights of adjacent entities to determine if the ranking is compliant. In this step, the system compares adjacent person entities in the sequence and finds that W (Zhang San) > W (Li Si), indicating an inversion pair. The system generates a very high-risk "person ranking error" warning and provides a suggestion: According to the XX principle, it is recommended to adjust it to "Comrade Li Si, Comrade Zhang San".

[0034] S400: Based on at least one result of the first-level deterministic rule verification and the second-level factual knowledge verification, dynamically construct prompt words containing constraint information, and call the large language model to perform the third-level semantic logic verification to identify complex semantic, logical, or stylistic errors in the official document.

[0035] In this invention, a hierarchical collaborative verification method is designed. Essentially, a deterministic engine (rules, knowledge) is used to set boundaries and provide constraints for an uncertain engine (large model). The deterministic facts and errors discovered by the first and second levels are injected as mandatory instructions into the prompts of the third-level large model, which restricts the model's free play from the source and prevents it from overturning known facts. The main purpose of the third-level verification is to use the deep contextual understanding capabilities of the Large Language Model (LLM) to handle complex semantics, logical consistency, risks in government expression, and writing style issues that cannot be covered by the first two levels. The verification content includes syntactic errors, time logic inversion (such as meeting time being earlier than notification time), inconsistencies between contexts, inappropriate word choice, and whether the tone is appropriate.

[0036] For example, after the first and second level verifications of this invention are completed, the system will dynamically construct prompt words (through role setting, negative constraints, few-shot learning, and thought chain guidance) and call the large language model for semantic logic verification. For example, the prompt words may include the following: System instructions: such as "You are a rigorous document review expert..."; Precedence constraint injection: inject the facts determined in the first two levels as hard constraints, such as the constraint: "City XX Bureau" is the correct organization name and cannot be modified, and the second-level heading "1. Training Content" has a half-width punctuation error; Text to be reviewed: The main text of the document, such as "This training meeting is scheduled to be held on November 15. Please complete the registration before November 20."

[0037] In an optional embodiment of the first aspect of the present invention, the third-level semantic logic verification further includes tiered processing of the verification results: By embedding metacognitive instructions into the prompt words, the large language model can output a confidence score for the modification suggestions while outputting modification suggestions. In this embodiment, the prompt words also include metacognitive instructions. An exemplary metacognitive instruction can be as follows: "Please classify each problem you find and give your confidence score (0-100) for the modification suggestion."

[0038] Based on the preset error risk level and the confidence score, a differentiated handling strategy is automatically matched. This strategy includes automatic correction, suggestive prompts, strong blocking prompts, or prohibition of automatic correction. This invention designs a four-level differentiated handling strategy for errors of different risk levels, ranging from automatic correction (low risk), suggestive prompts (medium risk), strong blocking prompts (high risk), to strictly prohibiting automatic correction (extremely high risk, such as government statements). This ensures efficiency while safeguarding the safety red lines of government scenarios.

[0039] For example, in this invention, after receiving the prompt word, the large language model performs deep semantic analysis as follows: For instance, the large language model discovers a logical contradiction in the text to be reviewed: "The registration deadline (November 20) is later than the meeting date (November 15)". The model outputs a result in NDJSON format: {type: LogicError, suggestion: The registration deadline is later than the meeting date, confidence: 99}. When the system receives this result, it matches the preset "strong blocking prompt" handling strategy based on the error type "LogicError" and its confidence score of 99.

[0040] S500: Aggregate the results of the first-level deterministic rule verification, the second-level factual knowledge verification, and the third-level semantic logic verification, and output them to the user interface. Specifically, this step in the present invention may include: The system receives multiple verification result fragments asynchronously returned by the first-level deterministic rule verification, the second-level factual knowledge verification, and the third-level semantic logic verification in a streaming data format. In this invention, to optimize user experience, the system does not wait for the entire proofreading to be completed, but instead returns the results in streaming format via the NDJSON protocol. That is, during the execution of steps S200, S300, and S400, the results are not returned only after all steps are completed. The punctuation error result in step S200 is generated first, and the system will immediately package it into an NDJSON object and send it. The large model inference result in step S400 is returned word by word in streaming format, and the aggregator listens to the token stream in real time.

[0041] Multiple verification result fragments are reassembled in real time into a structured verification object containing error type, in-situ coordinates, and modification suggestions. In this step, when the aggregator receives a complete NDJSON object (such as an object about time logic errors) from the stream in step S400, it will immediately begin processing, extracting the quoted text: "Complete registration before November 20th". Using the index mapping table established in step S100, the system quickly finds the character-level position of this text in the original document, for example, {start: 520, end: 531}. The system pushes this structured verification object {type: LogicError, start: 520, end: 531, ...} to the front end through the SSE (Server-Sent Events) channel.

[0042] Based on the document index mapping table established by the structured parsing of the official document, the in-situ coordinates of the structured verification object are reversed to the character-level position of the official document to achieve real-time highlighting of the verification results. In this step, taking the example from the previous step, after the front end receives the structured object, it will immediately highlight characters 520 to 531 in red in the editor, achieving a smooth user experience of "scanning and highlighting simultaneously".

[0043] In summary, the multi-engine collaborative document review method of this invention achieves the best balance between efficiency, intelligence and security through a funnel-shaped architecture of three-level collaboration of rules, knowledge and models, supplemented by various innovative mechanisms such as streaming interaction, dynamic knowledge and hierarchical processing. It can simultaneously meet the requirements of document review in three dimensions: format standardization, factual accuracy and semantic logic.

[0044] To better understand the step details of the multi-engine collaborative document review method of the present invention, the present invention also provides the following embodiments: Example 1: End-to-end intelligent document review and verification method. This example discloses an intelligent document review and verification method that integrates multimodal parsing and streaming collaborative computing (see [link to relevant documentation]). Figure 2 This method breaks through the limitations of traditional single engines and constructs an automated closed loop from unstructured document parsing to high-precision structured proofreading output.

[0045] Step 1: Document Parsing and Metadata Reconstruction: The system receives a request from the client through the official document review page. The request includes the document storage address and the verification strategy configuration vector. The document parsing unit reads the document's binary stream and, based on Document Object Model (DOM) technology, deconstructs the unstructured binary data into a sequence of paragraph objects P={p1, p2, ..., pn} containing content and style metadata. Each paragraph object not only contains a text string but also carries style attributes such as font, font size, indentation, and alignment.

[0046] Step 2: Semantic Label Mapping and Data Compression: The template analysis engine is invoked to perform secondary semantic enhancement on the sequence. The system incorporates a hybrid classifier based on heuristic rules and a lightweight classification model. 1) Feature extraction: Extract paragraph format features (such as "centered", "font size > 2") and text features (such as "notice about...").

[0047] 2) Tag mapping: Map paragraphs to semantic tags (such as type="first-level heading", type="body text", type="signature") based on feature vectors.

[0048] 3) Minimalist serialization: In order to optimize the utilization of the context window of subsequent large models, the system compresses rich text objects into lightweight JSON sequences paragraphsTypeJson, retaining only the index, text content and semantic type fields, which greatly reduces token consumption.

[0049] Step 3: Dynamically validate context construction: The system dynamically constructs the input vector based on the user's policy configuration using a context assembler. 1) Rule retrieval: Match the corresponding Prompt template fragment from the rule base (e.g., template T1 for "government review" and template T2 for "format check").

[0050] 2) Constraint Injection: Obtain the real-time "negative list" and "sensitive word list" from the dynamic dictionary service and inject them into the context as hard constraints.

[0051] 3) Prompt Fusion: The system prompt, structured document data, validation rules and constraints are vectorized and concatenated to generate the final inference input.

[0052] Step 4: Hierarchical Multi-Engine Collaborative Verification: The system adopts a "funnel-shaped" three-level collaborative verification mechanism to ensure the comprehensiveness and accuracy of the verification. 1) Level 1 (Deterministic Rule Layer): In the parsing stage of step 2, the hard rule scan of format and punctuation is performed in parallel using the regular expression matching algorithm, and basic errors are intercepted in milliseconds.

[0053] 2) Second level (factual knowledge layer): Combined with the domain knowledge injected in step 3, the model is guided to focus on factual features such as entity names and legal references during reasoning, thus suppressing model illusions.

[0054] 3) Third level (generative semantic layer): Invoke the streaming large language model (LLM), send the assembled context to the model inference end, perform deep semantic analysis, and identify logical paradoxes and risks in government statements.

[0055] The third-level semantic verification is triggered only when the results of the first and second-level verifications meet any of the following conditions: It involves cross-sentence logic or temporal causal relationships; It involves government-sensitive entities but was not found in the knowledge base; There is a conflict between the conclusions of the rule layer and the knowledge layer.

[0056] Step 5: Streaming Aggregation and Spatiotemporal Coordinate Mapping: The system processes the token stream returned by the model in real time through a streaming aggregator. 1) Protocol cleaning: Using a state machine parsing algorithm, unstructured thought chain data is stripped away and valid NDJSON objects are extracted.

[0057] 2) Confidence Gating: This confidence score is not calculated by rules, but is based on the "self-evaluation protocol" constraint of Prompt in step 3. The large language model scores itself based on the probability distribution of its generated tokens or the consistency of its internal logic. The system parses the confidence field in the JSON. If the score is lower than the preset threshold (e.g., 83 points), it is considered that the model has doubts about the modification suggestion and it is discarded directly.

[0058] 3) Reverse coordinate mapping: Based on the index map established in step 1, the quoted text returned by the model is reversed to the character-level coordinate range [Start, End] of the original document to ensure zero deviation in the highlight position.

[0059] 4) Real-time push: Through the SSE (Server-Sent Events) one-way channel, structured validation items are pushed to the front end in real time to achieve a "what you see is what you get" interactive experience.

[0060] Example 2: Knowledge Graph-Based Entity Consistency and Government Ranking Verification Method: This example details how to utilize a knowledge graph (KG) in the government domain to solve the problems of document entity accuracy and government ranking compliance (see [link]). Figure 3 ).

[0061] Step 201: Domain Named Entity Recognition: Perform sequence labeling on the document text, and use a pre-trained NER model for government affairs or the double-array trie algorithm to accurately extract three types of key entities: 1) Institutional entities: such as "XX Municipal Committee" and "XX Provincial Department".

[0062] 2) Person entities: such as the names and positions of specific leaders.

[0063] 3) Regulation entities: such as the full names or abbreviations of laws and regulations.

[0064] Step 202: Knowledge Graph Linking and Reasoning: Map the extracted entity set to the local government affairs knowledge graph and perform subgraph queries: 1) Alias Normalization Detection: Calculate the similarity between the extracted entity and the graph node. If the attribute annotation is "the full name is required for official documents", then trigger the "the full name is required for official documents" warning and recommend the standard full name.

[0065] 2) Regulation Lifecycle Verification: Query the State attribute of the regulation entity node. If State contains revocation or amendment, then generate an "invalid reference" error message through the verification engine.

[0066] Step 203: Sequence Sorting Verification Based on Multi-Dimensional Weights: For the scenario of multiple entities in parallel, execute the sorting algorithm based on the graph weights: 1) Sequence Extraction: Use dependency syntactic analysis to extract the parallel entity sequence S = [e1, e2,..., ek].

[0067] 2) Weight Vector Obtaining: Retrieve the weight vector W(ei) = <Level, Rank, PartyOrder> (administrative level, ranking, specific sequence) of each entity ei from the knowledge graph.

[0068] 3) Inversion Pair Detection: Traverse the sequence S and check adjacent node pairs (ei, ei+1). If there exists W(ei) < W(ei+1) (i.e., the weight of the previous entity is lower than that of the subsequent entity), then it is determined as "government affairs ranking error".

[0069] 4) Revised Sequence Generation: Reorder the sequence based on the weight vector to generate a recommended sequence that conforms to government affairs rules.

[0070] Example 3: Contradiction Detection Method Based on Spatiotemporal Logic Diagram: This example describes a logical conflict detection algorithm based on a spatiotemporal constraint network (see Figure 4 )

[0071] Step 301: Temporal Entity Normalization: Use a temporal parser to extract all temporal expressions in the text and uniformly map absolute times (such as "2026-1-15") and relative times (such as "next Thursday") to standard timestamps or time intervals.

[0072] Step 302: Event-Time Binding and Graph Construction: Through semantic role labeling (SRL), identify the predicate verbs (event trigger words) to which the temporal entities belong, and construct a set of "event-time" binary tuples E={(Eventi, Timei)}.

[0073] Construct a temporal directed acyclic graph (DAG): Nodes: Represent specific business events (such as "issuance", "holding a meeting", "deadline"), Edges: Represent the precedence constraint relationships defined by business logic.

[0074] Step 303: Injection of Constraint Rules: The system pre-sets a knowledge base of document business temporal logic rules, including but not limited to: Rule R1 (Deadline Logic): Time(Feedback / Registration Deadline) < Time(Meeting / Activity Start) Rule R2 (Issuance Logic): Time(Drafting Date) < Time(Issuance Date) Rule R3 (Causal Logic): Time(Published Date of Cited Document) < Time(Writing Date of Current Document) Step 304: Topological Sorting and Conflict Inference: Perform topological sorting or constraint satisfaction problem (CSP) solving on the constructed DAG: Traverse each constraint edge in the graph.

[0075] If the actually parsed time value satisfies Time(A) > Time(B), then a "time reversal" conflict is detected.

[0076] The system outputs the conflict type and specific logical contradiction points (such as "Registration deadline is later than the meeting start time").

[0077] Example 4: Deep Semantics and Government Affairs Risk Control Review Based on Large Models: This example describes the implementation path of using a large language model Agent for high-order semantic analysis and government affairs security review (see Figure 5 )

[0078] Step 401: Secure Alignment Prompt Engineering: Construct a structured Prompt template containing the three elements of "role-constraint-sample": 1) Role Anchoring: Set the model as a "government affairs review expert with extremely high government affairs sensitivity".

[0079] 2) Negative Constraints: Clearly state the instructions "Prohibit fabricating facts", "Prohibit modifying irrelevant original text", "Stay silent if there are no errors".

[0080] 3) Few-shot learning: Inject positive and negative examples and align the model's verification criteria through in-context learning.

[0081] Step 402: Dynamic Blacklist Injection Mechanism: During inference runtime, the latest "blacklist of government statements" and "sensitive word variant library" are injected into the Prompt's System Context via a dynamic dictionary service. The instruction model performs high-weight scanning of specific sensitive words during inference to ensure real-time compliance of current political hot words.

[0082] Step 403: Guided CoT (CoT) Reasoning: Embed CoT instructions in the System Prompt to guide the model in step-by-step reasoning: Step 1: Contextual Analysis: Determine the document type (e.g., "request for instructions") and verify that the tone of the entire document conforms to the document type specifications (e.g., "request for instructions" should not be in a commanding tone).

[0083] Step 2 Stance Detection: Scan the paragraphs involving politically sensitive issues and assess whether their political stance meets the core standards.

[0084] Step 3: Logical consistency: Check whether there are numerical contradictions in the descriptions of the same subject (such as project amount or deadline) in the preceding and following text.

[0085] Step 404: Defensive Output Parsing: Force the model to output structured NDJSON data, including error type, reason, and suggestions, and require each validation result to include a confidence field. This field is generated through a Chain-of-Thought mechanism. Before outputting modification suggestions, the model must first evaluate the necessity and certainty of the modification in the implicit thought space and quantify it into a score of 0-100.

[0086] The system implements output guardrails: the suggested text generated by the model is scanned again for sensitive words. If the suggested text itself contains sensitive words or poses a risk of illusion, the system will automatically block the suggested text, ensuring the absolute security and compliance of the review results.

[0087] See Figure 6 The second aspect of the present invention provides a multi-engine collaborative document review system, the multi-engine collaborative document review system comprising: Document parsing module 10 is used to perform structured parsing of official documents to be reviewed, identify and map the physical style attributes in the official documents to structured semantic nodes; The first-level verification module 20 is used to perform first-level deterministic rule verification, which screens the structured semantic nodes for format and punctuation errors based on preset format specifications and regular expressions. The secondary verification module 30 is used to perform the second-level factual knowledge verification, which verifies the entities in the official document based on an external knowledge base. The three-level verification module 40 is used to dynamically construct prompt words containing constraint information based on at least one result of the first-level deterministic rule verification and the second-level factual knowledge verification, and call the large language model to perform the third-level semantic logic verification to identify complex semantic, logical or stylistic errors in the official document. The result aggregation module 50 is used to aggregate the results of the first-level deterministic rule verification, the second-level factual knowledge verification, and the third-level semantic logic verification, and output them to the user interface.

[0088] In an optional embodiment of the second aspect of the present invention, the document parsing module includes: The attribute parsing unit is used to parse the physical style attributes of the document, including the font, font size, indentation, and alignment of each paragraph. The paragraph mapping unit is used to map the unstructured text paragraphs of the official document into structured semantic nodes including titles, body text, and signatures based on the physical style attributes, so as to provide contextual basis for the first-level deterministic rule verification.

[0089] In an optional embodiment of the second aspect of the present invention, the multi-engine collaborative document review system further includes a short-circuit mechanism module, wherein the short-circuit mechanism module: The format determination unit is used to determine whether a preset fundamental format error has been detected after the first-level deterministic rule verification is completed; The verification termination unit is used to terminate the subsequent second-level factual knowledge verification and the third-level semantic logic verification if the fundamental format error is detected, and only outputs a format error report.

[0090] In an optional embodiment of the second aspect of the present invention, the result aggregation module includes: The streaming receiving unit is used to receive multiple verification result fragments asynchronously returned by the first-level deterministic rule verification, the second-level factual knowledge verification, and the third-level semantic logic verification in streaming data format; The fragment recombination unit is used to recombine multiple verification result fragments in real time into a structured verification object containing error type, in-situ coordinates and modification suggestions; The proofreading and display unit is used to reverse the original coordinates of the structured verification object to the character-level position of the document based on the document index mapping table established by the structured parsing of the official document, so as to realize the real-time highlighting display of the proofreading results.

[0091] In an optional embodiment of the second aspect of the present invention, the secondary verification module includes: The request receiving unit is used to receive verification requests carrying a verification timestamp; The knowledge matching unit is used to match and call the valid knowledge set corresponding to the verification timestamp from one or more multi-version external knowledge bases that are bound to the effective time and the expiration time. The spatiotemporal verification unit is used to verify the spatiotemporal accuracy of the organization names, laws and regulations, or personal entities in the official document based on the effective knowledge set.

[0092] In an optional embodiment of the second aspect of the present invention, the third-level semantic logic verification further includes tiered processing of the verification results: By embedding metacognitive instructions into the prompt words, the large language model can output a confidence score for the modification suggestions while simultaneously outputting modification suggestions. Based on the preset error risk level and the confidence score, a differentiated handling strategy is automatically matched. The handling strategy includes automatic correction, suggestive prompts, strong blocking prompts, or prohibition of automatic correction.

[0093] In an optional embodiment of the second aspect of the present invention, when the second-level factual knowledge verification involves multiple person entities arranged in parallel, the second-level factual knowledge verification further includes: Using dependency parsing, a sequence of parallel person entities is extracted from the official document. From the government knowledge graph, retrieve and obtain the level or ranking weight corresponding to each entity in the sequence of person entities; The sequence of the individuals is traversed, and the weights of adjacent individuals are compared to detect whether there are inversion pairs, so as to determine whether the ranking of the individuals is compliant.

[0094] Figure 7This is a schematic diagram of a multi-engine collaborative document review device provided in an embodiment of the present invention. This multi-engine collaborative document review device can vary significantly due to differences in configuration or performance. It may include one or more processors 60 (central processing units, CPUs) (e.g., one or more processors) and memory 70, and one or more storage media 80 (e.g., one or more mass storage devices) for storing applications or data. The memory and storage media can be temporary or persistent storage. The program stored in the storage media may include one or more modules (not shown in the diagram), each module including a series of instruction operations on the multi-engine collaborative document review device. Furthermore, the processor may be configured to communicate with the storage media to execute the series of instruction operations in the storage media on the multi-engine collaborative document review device.

[0095] The multi-engine collaborative document review device of this invention may also include one or more power supplies 90, one or more wired or wireless network interfaces 100, one or more input / output interfaces 110, and / or one or more operating systems, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, etc. Those skilled in the art will understand that... Figure 7 The structure of the multi-engine collaborative document review equipment shown does not constitute a limitation on the multi-engine collaborative document review equipment. It may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0096] The present invention also provides a computer-readable storage medium, which can be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium, wherein the computer-readable storage medium stores instructions that, when the instructions are executed on a computer, cause the computer to perform the steps of the multi-engine collaborative document review method.

[0097] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the system or system / unit described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0098] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0099] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A multi-engine collaborative document review method, characterized in that, include: The document to be reviewed is structured and parsed to identify and map the physical style attributes in the document to structured semantic nodes. The first level of deterministic rule verification is performed, and the structured semantic nodes are screened for format and punctuation errors based on preset format specifications and regular expressions. Perform a second-level factual knowledge verification, based on an external knowledge base, to verify the entities in the official document; Based on at least one result of the first-level deterministic rule verification and the second-level factual knowledge verification, prompt words containing constraint information are dynamically constructed, and a large language model is invoked to perform a third-level semantic logic verification to identify complex semantic, logical, or stylistic errors in the official document. The results of the first-level deterministic rule verification, the second-level factual knowledge verification, and the third-level semantic logic verification are aggregated and output to the user interface.

2. The multi-engine collaborative document review method according to claim 1, characterized in that, The process of performing structured parsing on the official documents to be reviewed, and identifying and mapping the physical style attributes in the official documents to structured semantic nodes, includes: The document's physical style attributes, including the font, font size, indentation, and alignment of each paragraph, are analyzed. Based on the physical style attributes, the unstructured text paragraphs of the official document are mapped into structured semantic nodes including titles, body text, and signatures, so as to provide contextual basis for the first-level deterministic rule verification.

3. The multi-engine collaborative document review method according to claim 1, characterized in that, The multi-engine collaborative document review method also includes a short-circuit mechanism: After the first-level deterministic rule verification is completed, it is determined whether a preset fundamental format error has been detected; If the fundamental format error is detected, the subsequent second-level factual knowledge verification and the third-level semantic logic verification are terminated, and only a format error report is output.

4. The multi-engine collaborative document review method according to claim 1, characterized in that, The aggregation of the results of the first-level deterministic rule verification, the second-level factual knowledge verification, and the third-level semantic logic verification, and the output to the user interface, includes: Receive multiple verification result fragments asynchronously returned by the first-level deterministic rule verification, the second-level factual knowledge verification, and the third-level semantic logic verification in a streaming data format; Multiple verification result fragments are reassembled in real time into a structured verification object containing error type, in-situ coordinates, and modification suggestions; Based on the document index mapping table established by the structured parsing of the official document, the original coordinates of the structured verification object are reversed to the character-level position of the official document, so as to realize the real-time highlighting of the review results.

5. The multi-engine collaborative document review method according to claim 1, characterized in that, The second-level factual knowledge verification, based on an external knowledge base, involves factually verifying the entities in the official document, including: Receive verification requests carrying a verification timestamp; From one or more multi-version external knowledge bases that are bound to effective time and expiration time, match and call the valid knowledge set corresponding to the verification timestamp; Based on the aforementioned effective knowledge set, the spatiotemporal accuracy of the organization names, laws and regulations, or personal entities in the official documents is verified.

6. The multi-engine collaborative document review method according to claim 1, characterized in that, The third-level semantic logic verification also includes tiered processing of the verification results: By embedding metacognitive instructions into the prompt words, the large language model can output a confidence score for the modification suggestions while simultaneously outputting modification suggestions. Based on the preset error risk level and the confidence score, a differentiated handling strategy is automatically matched. The handling strategy includes automatic correction, suggestive prompts, strong blocking prompts, or prohibition of automatic correction.

7. The multi-engine collaborative document review method according to claim 1, characterized in that, When the second-level factual knowledge verification involves multiple person entities listed in parallel, the second-level factual knowledge verification also includes: Using dependency parsing, a sequence of parallel person entities is extracted from the official document. From the government knowledge graph, retrieve and obtain the level or ranking weight corresponding to each entity in the sequence of person entities; The sequence of the individuals is traversed, and the weights of adjacent individuals are compared to detect whether there are inversion pairs, so as to determine whether the ranking of the individuals is compliant.

8. A multi-engine collaborative document review system, characterized in that, The multi-engine collaborative document review system includes: The document parsing module is used to perform structured parsing of official documents to be reviewed, and to identify and map the physical style attributes in the official documents to structured semantic nodes; The first-level verification module is used to perform first-level deterministic rule verification, which screens the structured semantic nodes for format and punctuation errors based on preset format specifications and regular expressions. The secondary verification module is used to perform the second-level factual knowledge verification, which verifies the entities in the official document based on an external knowledge base. The three-level verification module is used to dynamically construct prompt words containing constraint information based on at least one result of the first-level deterministic rule verification and the second-level factual knowledge verification, and call the large language model to perform the third-level semantic logic verification to identify complex semantic, logical or stylistic errors in the official document. The result aggregation module is used to aggregate the results of the first-level deterministic rule verification, the second-level factual knowledge verification, and the third-level semantic logic verification, and output them to the user interface.

9. A multi-engine collaborative document review and verification device, characterized in that, The multi-engine collaborative document review device includes: a memory and at least one processor, wherein the memory stores instructions, and the memory and the at least one processor are interconnected via a line; The at least one processor invokes the instructions in the memory to cause the multi-engine collaborative document review device to execute the multi-engine collaborative document review method as described in any one of claims 1-7.

10. A computer-readable storage medium storing a computer program thereon, characterized in that, When the computer program is executed by the processor, it implements the multi-engine collaborative document review method as described in any one of claims 1-7.

Images