A file compliance review method and system based on three-layer model collaborative verification decision

Abstract

The application relates to a file compliance review method and system based on three-layer model collaborative verification decision, and belongs to the natural language processing field. The method comprises the following steps: preprocessing and analyzing an uploaded document to construct a unified document model, and performing multi-level document structure index construction; performing deterministic compliance detection on the document based on pre-defined rule configuration and document structure index by a special large model agent, simultaneously performing deep semantic review on the document in parallel by a global large model agent by using the native semantic reasoning capability of a large language model; performing evidence layer conflict detection and automatic adjudication on the output results of the special large model agent and the global large model agent by a strategy adjudication agent; rendering and highlighting the final violation results after adjudication, and displaying the review results. The application designs a unified document model and a block-level identifier mechanism, realizes one-time analysis and dual-purpose, and fundamentally eliminates the data inconsistency problem between detection and display.

Description

Technical Field

[0001] This invention relates to the field of natural language processing, and in particular to a document compliance review method and system based on a three-layer model for collaborative verification decision-making. Background Technology

[0002] With the deepening of procurement system reforms in enterprises and institutions and the continuous growth of procurement scale, the workload of document review faced by relevant regulatory departments and procurement agencies has increased dramatically. Procurement documents (including tender documents, competitive negotiation documents, competitive consultation documents, and requests for quotation, etc.) can range from dozens to hundreds of pages in length, covering multiple chapters such as project overview, qualification requirements, procurement needs, review methods, and contract terms. The content encompasses various information types, including legal and regulatory citations, numerical calculations, dates and deadlines, qualification requirements, and scoring criteria. According to existing regulations, procurement documents must meet strict compliance requirements; any oversight may lead to illegal or irregular procurement activities, supplier objections and complaints, or even the rejection of the procurement project.

[0003] The existing procurement document review methods mainly suffer from the following technical problems: 1. Manual review is inefficient and inconsistent; Currently, the compliance review of most procurement documents still relies on manual page-by-page reading and experience-based judgment. Reviewers need to repeatedly compare the consistency of key fields such as project number, project name, budget amount, and deadline in different chapters of long documents. At the same time, they also need to calculate whether the number of working days, total amount, and bid bond ratio meet the legal requirements. This process is time-consuming and tiring. Different reviewers often have different review conclusions on the same document, making it difficult to guarantee the stability of review quality. 2. Traditional rule engines lack semantic understanding capabilities. Some organizations have attempted to use rule engines based on regular expressions or keyword matching for auxiliary review, but such methods can only handle compliance items with fixed formats and clear expressions. They are powerless to handle review dimensions that require semantic understanding of requirements. For example, determining whether qualification conditions constitute unreasonable restrictions on suppliers, whether there are unquantified subjective factors in the scoring criteria, whether technical parameters are brand-oriented, and whether there are logical contradictions in the business conditions in different chapters all require deep semantic analysis of the text, which traditional rule engines cannot handle. In addition, procurement documents contain a large number of synonyms, abbreviation mappings, cross-chapter references, and other linguistic phenomena, and simple string matching is very likely to produce false positives or false negatives. 3. The reliability of relying solely on large language models for review is insufficient. In recent years, large language models have demonstrated powerful capabilities in text understanding and generation, and some studies have attempted to apply them directly to document compliance review. However, relying solely on large language models for review faces the following challenges: First, the accuracy of large language models in deterministic tasks such as numerical calculation and date reasoning is lower than that of procedural rule engines, making them prone to calculation errors or omissions. Second, the output of large language models is random; multiple reviews of the same document may produce different conclusions, lacking reproducibility. Third, large language models struggle to accurately locate the position of violating content in the original text, typically providing only vague descriptive locations, which cannot meet the requirement for precise highlighting of review results in document previews. 4. Insufficient precision in violation location. Existing document review systems typically use runtime text matching to mark violation locations, i.e., searching for and highlighting violating text fragments in the rendered document preview. This approach suffers from severe positioning bias; the use of different data sources in the document parsing and rendering stages leads to differences in format, whitespace characters, or encoding in the text previews of the detected text fields, resulting in a high matching failure rate; for table content, cross-paragraph text, and recurring text fragments, runtime matching is more prone to mis-positioning or failure to position.5. Lack of effective conflict resolution mechanisms in multi-model collaboration: When multiple models are used simultaneously for review, different models may reach different or even contradictory conclusions on the same review dimension. For example, a specific rule model determines inconsistencies in amounts based on literal comparison, while a global semantic model identifies one as a provisional estimate or subcontracting budget through semantic understanding, leading to contradictory conclusions. Therefore, existing technologies lack a systematic multi-model conflict detection and adjudication mechanism, typically relying on simple union calculations or manual judgment, which increases both the false positive rate and the burden of manual review. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of the prior art and provide a document compliance review method and system based on a three-layer model collaborative verification decision, thus solving the deficiencies of the prior art.

[0005] The objective of this invention is achieved through the following technical solution: a document compliance review method based on a three-layer model for collaborative verification decision-making, the method comprising:

[0006] Step 1: Preprocess and parse the uploaded documents to build a unified document model and construct a multi-level document structure index;

[0007] Step 2: A specialized large-scale model agent, fine-tuned and trained from the first large-scale language model, performs deterministic compliance checks on the document based on predefined rule configuration and document structure index. At the same time, a global large-scale model agent, fine-tuned and trained from the second large-scale language model, utilizes the native semantic reasoning capabilities of the second large-scale language model to perform in-depth semantic review of the document in parallel.

[0008] Step 3: Perform evidence-level conflict detection and automatic adjudication on the output results of the special large model agent and the global large model agent through the policy adjudication agent;

[0009] Step 4: Render and highlight the final violation results after the ruling, and display the review results.

[0010] Step one specifically includes the following:

[0011] A1. Users upload procurement documents in a set format through the front-end quick review page. The parser traverses the document's XML format, converts paragraph and table elements into a sequence of document blocks, assigns a stable block-level identifier to each block, and extracts key field metadata and table data. After parsing, it outputs the number of blocks, the number of tables, the text length, and the number of structural nodes.

[0012] A2. Construct a multi-level document structure index based on the unified document model, identify chapter level, section level, subsection level and attachment title, establish a five-level structure tree, and convert the unified document model into a compatible format.

[0013] The specialized large-scale intelligent agent, fine-tuned and trained from the first large-scale language model, performs deterministic compliance checks on documents based on predefined rule configuration and document structure indexing, including:

[0014] Based on the document structure index, regular expression methods and the first language model are used to perform deterministic compliance rule checks on the documents, including cross-chapter consistency checks, summary checks, working day calculation checks, threshold comparison checks, and required field integrity checks.

[0015] By using a retrieval-enhanced generation mechanism, regulations are retrieved from the regulatory knowledge base to assist in review and judgment, and rule violation results with block-level identifier location information are output.

[0016] The simultaneous use of a global large-scale model agent, fine-tuned and trained by the second large-scale language model, to perform deep semantic review of documents in parallel, leveraging the native semantic reasoning capabilities of the second large-scale language model, includes:

[0017] The global large model agent constructs paragraph anchor indexes, assigns a unique number to each non-empty paragraph in the document, embeds the number into the document text, and generates review text with anchor indexes.

[0018] Based on the document structure index, intelligent slicing is performed according to chapter boundaries. Each slice is injected with metadata summary. At the same time, legal clauses are retrieved from the legal knowledge base according to the chapter content involved in the slice and injected into the slice context.

[0019] Each slice carries system prompts and regulatory references and is sent concurrently to the second language model via semaphore control. The second language model performs a ten-dimensional deep semantic review on each slice and returns a structured violation result.

[0020] Parse paragraph numbers into block-level identifiers and perform a fallback search of normalized text for violations that cannot be located.

[0021] Verify whether each violating matching text actually exists in the original document. For non-existent matching text, extract alternative text from the evidence field. Perform rate limiting and deduplication on violating results with the same rule name.

[0022] Step three includes:

[0023] By using a rule identifier mapping table and a rule name alias table, the output results of the special large model intelligent agent and the global large model intelligent agent are associated and matched according to the review dimensions to detect differences in conclusions, differences in evidence location, and differences in numerical extraction.

[0024] The four-level strategy is executed sequentially: numerical normalization decision, rule priority decision, semantic priority decision, and user decision retention.

[0025] When making semantic-priority rulings, the quality of legal citations in the violation results is compared, and the conclusion of the party with more sufficient legal basis is adopted first. Conflicts that are automatically ruled are handled directly, while conflicts that require user ruling are marked and presented to the user.

[0026] A1 includes:

[0027] A101. Parse the XML structure of the procurement document, traverse the paragraph and table elements in the document, and generate a stable block-level identifier for each element based on the path and sequence number of the element in the document using a hash algorithm.

[0028] A102. For each document block, record its block type, region type, text content, normalized text, starting line number, parent node path, node number, starting character position, and ending character position;

[0029] A103. Perform key field extraction on the entire document. Extract structured metadata such as project name, project number, procurement budget, maximum price, procurement method, contract performance period, bid deadline, bid security amount and announcement date through regular expressions and context analysis. Each extracted field is associated with its block-level identifier.

[0030] A104. Perform enhanced parsing on the tables in the document to identify the procurement item table and the review criteria table, extract the table headers and row data, and map each cell in the table to an independent document block.

[0031] A2 includes:

[0032] A201. Scan the document block sequence and use regular expressions to identify chapter-level headings, section-level headings, subsection-level headings, and attachment headings, and construct a document structure tree containing five levels: cover, chapter, section, subsection, and attachments.

[0033] A202. Each structure node records its hierarchy, title, key value, starting block index, ending block index, and list of child nodes;

[0034] A203 provides a path query interface, which supports obtaining a set of document blocks within a specified chapter range through expressions and performing regular expression searches within a specified range.

[0035] For cross-chapter consistency verification, target field values ​​are extracted from multiple specified chapters, and the extracted values ​​are normalized before consistency comparison is performed. An alert is triggered when any two valid extracted values ​​are inconsistent after normalization.

[0036] For summary verification, extract the total amount and the list of itemized amounts respectively, convert all amounts to the same unit, and compare the difference between the itemized summary and the total amount. When the difference exceeds the preset tolerance threshold, trigger an alarm.

[0037] For working day calculation verification, extract the date range from the document, calculate the number of working days, and compare it with the statutory minimum working day requirement;

[0038] For threshold comparison verification, relevant numerical fields are extracted and their size is compared. When the value exceeds the statutory threshold, an alert is triggered.

[0039] The method further includes: accurately locating the violation results output by the first and second language models, specifically including:

[0040] Problem type classification: Based on the verb characteristics in the violation description, violations are divided into two categories: existence violations and missing violation violations;

[0041] For existing violations, a four-level progressive search is performed: the first level searches for matching text within the chapter range specified in the rules, the second level searches within the extended range, the third level searches within the entire text, and the fourth level falls back to the chapter node level for location. After each level of search is successful, a corresponding confidence limit is set.

[0042] For missing violations, the search scope is determined based on a predefined mapping table of domain keywords to expected chapters, and the most relevant node-level position is located in the expected chapter;

[0043] The output anchoring results include block-level identifier, character offset, matched text, confidence level, anchoring type, and search summary.

[0044] A document compliance review system based on a three-layer model collaborative verification decision-making system includes a unified document parsing module, a document structure indexing module, a specialized large model intelligent agent, a global large model intelligent agent, a strategy adjudication intelligent agent, a document rendering and highlighting module, and a regulatory knowledge base.

[0045] The unified document parsing module is used to perform unified document parsing on uploaded procurement documents, build a unified document model, decompose the document content into a sequence of document blocks with stable block-level identifiers, and extract document structure information and key field metadata.

[0046] The document structure indexing module is used to construct a multi-level document structure index, identify the chapter hierarchy of a document, and support text retrieval with a range limit based on chapter paths.

[0047] The specialized large-scale model intelligent agent: based on document structure indexing, it uses regularization methods and the first large language model to perform deterministic compliance rule detection on documents, and retrieves regulatory clauses from the regulatory knowledge base through a retrieval enhancement generation mechanism to assist in review and judgment, and outputs rule violation results with block-level identifier location information;

[0048] The global large model intelligent agent: constructs paragraph anchor indexes for the unified document model, embeds paragraph numbers into the document text, injects review context by combining legal clauses retrieved from the legal knowledge base, and sends it to the second large language model. The second large language model performs in-depth semantic review of the document and outputs semantic violation results with paragraph anchor references and legal basis.

[0049] The strategy adjudication agent performs evidence-level conflict detection on the output results of the special large model agent and the global large model agent, identifies the differences in conclusions, evidence positions, and numerical extraction between the special large model agent and the global large model agent in the same review dimension, performs automatic adjudication on deterministic conflicts, and reserves the decision-making of high-risk uncertain conflicts to the user.

[0050] The document rendering and highlighting module: inputs the final violation result after adjudication and the unified document model into the document rendering engine, and accurately highlights all violation locations in the document preview based on block-level identifiers and character offsets;

[0051] The aforementioned legal knowledge base is used to manage procurement-related laws, regulations, departmental rules, and local normative documents, and supports the classification management of knowledge items, the construction of vectorized indexes, and semantic similarity retrieval.

[0052] This invention offers the following advantages: a document compliance review method and system based on a three-layer model collaborative verification decision-making approach, which improves review accuracy and recall through three-layer parallel agent (intelligent agent) collaborative review; a unified document model and block-level identifier mechanism are designed to achieve one-time parsing and dual-purpose functionality, fundamentally eliminating data inconsistency between detection and display; a four-level strategy adjudication mechanism based on evidence-level conflict detection is proposed, which automatically handles deterministic conflicts through numerical normalization adjudication, rule-priority adjudication, semantic-priority adjudication, and user adjudication retention, leaving only high-risk uncertain conflicts to user adjudication; a precise positioning scheme combining paragraph anchor indexing and four-level progressive search is developed to solve the industry problem of difficult-to-precise output positioning; a multi-level document structure index is constructed to support chapter-aware scope-limited retrieval, improving rule detection accuracy; an integrated retrieval enhancement generation mechanism dynamically expands the coverage of regulatory knowledge, enhancing the regulatory basis for review judgment; and a complete evaluation index system based on the four quadrants of the confusion matrix is ​​established, covering precision, recall, F1 score, accuracy, MCC, AUC, positioning accuracy, automatic conflict adjudication rate, and parallel acceleration efficiency, supporting continuous quantitative evaluation and optimization of system review quality. Attached Figure Description

[0053] Figure 1 This is a schematic diagram of the process of the present invention. Detailed Implementation

[0054] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the detailed description of the embodiments of this application provided below with reference to the accompanying drawings is not intended to limit the scope of protection of the claimed application, but merely represents selected embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application. The present invention will be further described below with reference to the accompanying drawings.

[0055] One embodiment of the present invention relates to an intelligent compliance review method for procurement documents based on a three-layer model collaborative verification and strategy adjudication. Its core architecture is a three-layer parallel agent workflow: the first layer, a special large model Agent A1, and the second layer, a global large model Agent A2, review the same document in parallel and independently; the third layer, a strategy adjudication Agent A3, performs evidence-level conflict detection and automatic adjudication on the outputs of the first two layers; the three agents share a unified document model as the only data source, and finally achieve precise highlighting of the violation location through block-level identifiers.

[0056] like Figure 1 As shown, it specifically includes the following:

[0057] S1. Perform unified document parsing on the uploaded procurement documents and build a unified document model. In this process, B represents the document block sequence, M represents the metadata dictionary, and T represents the table data set. The document content is decomposed into a document block sequence with stable block-level identifiers, while document structure information and key field metadata are extracted. The unified document model serves as the sole data source for subsequent rule detection and document rendering.

[0058] Furthermore, S1 specifically includes:

[0059] S101. Parse the XML format of the procurement document, traverse the paragraph and table elements in the document, generate a stable block-level identifier for each element, and generate each document block. The block-level identifier is generated using a hash algorithm based on the element's path and ordinal number in the document, as shown in the following formula:

[0060] ,

[0061] in, The path to the element in the XML document. The identifier is the node number, and || is for string concatenation. This identifier remains stable as long as the document content remains unchanged, so that the violation location output by the rule detection can be directly mapped to the rendered document preview without runtime text matching.

[0062] S102. For each document block, record its type, region type, text content, normalized text, starting line number, parent node path, node number, starting character position, and ending character position. The region type includes cover region, table of contents region, and body region.

[0063] S103. Perform key field extraction on the entire document. Extract structured metadata such as project name, project number, procurement budget, maximum price, procurement method, contract performance period, bid deadline, bid security amount, and announcement date through regular expressions and context analysis. Each extracted field is associated with its block-level identifier.

[0064] S104. Perform enhanced parsing on the tables in the document, identify table types such as procurement item tables and review criteria tables, extract table headers and row data, and map each cell in the table to an independent document block.

[0065] S2. A multi-level document structure index is built based on a unified document model to identify the hierarchical relationships between document chapters, supporting text retrieval limited to a specific chapter path. Specifically, this includes the following:

[0066] S201. Scan the document block sequence and identify chapter-level headings, section-level headings, subsection-level headings, and attachment headings using regular expressions, and construct a document structure tree containing five levels: cover, chapter, section, subsection, and attachments.

[0067] S202. Each structure node records its hierarchy, title, key value, starting block index, ending block index, and list of child nodes;

[0068] S203 provides a path-based query interface, which supports obtaining a set of document blocks within a specified chapter range and performing regular expression searches within a specified range using path expressions. Path expressions support multi-level nested queries.

[0069] S3, the first-layer specialized large model is obtained by fine-tuning and training the first large language model. Based on the document structure index, it uses regularization methods and domain-fine-tuned first large language model to perform deterministic compliance rule detection on documents, including cross-chapter consistency verification, numerical summary verification, working day calculation verification, threshold comparison verification, and required field integrity verification. It also retrieves relevant regulations from the legal knowledge base through a retrieval enhancement generation mechanism to assist in review and judgment, and outputs rule violation results with block-level identifier location information.

[0070] Furthermore, the steps in S3 specifically include the following:

[0071] S301. Load the predefined review rule configuration. Each rule includes a rule identifier, rule name, applicable scenario, warning level, hard-coded detection specification, and false alarm protection strategy. The hard-coded detection specification defines the rule type, extraction target, detection location, normalization method, and triggering conditions.

[0072] S302. Automatically filter the applicable rule subset based on the procurement method of the document;

[0073] S303. For cross-chapter consistency verification rules, target field values ​​are extracted from multiple specified chapters, and the extracted values ​​are normalized before consistency comparison is performed. The normalization process includes unifying full-width and half-width characters, folding spaces, converting units, and standardizing formats. An alert is triggered when any two valid extracted values ​​are inconsistent after normalization.

[0074] S304. For numerical summary verification rules, extract the total amount and itemized amount lists respectively, convert all amounts to the same unit, and then compare the difference between the itemized summary and the total amount; trigger an alert when the difference exceeds the preset tolerance threshold.

[0075] S305. For rules on working day calculation and verification, extract the date range from the document, combine it with China's statutory holidays and workday adjustment calendar to calculate the number of working days, and compare it with the statutory minimum working day requirement; at the same time, it supports the strict calculation mode based on the Civil Code, which starts from the next day;

[0076] S306. For threshold comparison and verification rules, extract relevant numerical fields and compare the text size relationship. When the value exceeds the statutory threshold, trigger an alert.

[0077] S307. For review dimensions that require legal basis support, relevant clauses are retrieved from the legal knowledge base through a retrieval enhancement generation mechanism: the current review context is encoded into a query vector, the most relevant clauses are retrieved from the vector index of the legal knowledge base through cosine similarity, and the retrieved legal clauses are injected as evidence into the review and judgment process to enhance the legal basis for dimensions such as deposit compliance, SME policies, and qualification conditions.

[0078] S308. The detection result of each rule includes whether it was triggered, confidence score, evidence text, reasoning process, modification suggestions, evidence block identifier list and extracted value dictionary; the confidence score ranges from 0 to 1 and is dynamically calculated based on the sufficiency of evidence and detection certainty.

[0079] S4. The second-layer global model is obtained by fine-tuning and training the second-largest language model. It constructs paragraph anchor indexes for the unified document model, embeds paragraph numbers into the document text, and injects relevant legal clauses retrieved from the legal knowledge base into the review context. It is then sent to the second-largest language model in the same domain for fine-tuning. The second-largest language model performs in-depth semantic review of the document and outputs semantic violation results with paragraph anchor references and legal basis. Among them, the specialized large model and the global large model run in parallel and independently on the same document, sharing the same legal knowledge base.

[0080] Furthermore, the steps in S4 specifically include the following:

[0081] S401, Paragraph Anchor Index Construction: Traverse the document block sequence in the unified document model, assign a unique paragraph number to each non-empty paragraph, establish a mapping relationship between paragraph number and block-level identifier, embed the paragraph number in the document text in the form of a tag, and generate review text with anchor index.

[0082] S402, Structure-Aware Intelligent Slicing: When the document length exceeds the limit for a single review, intelligent slicing is performed based on the document structure index and chapter boundaries. Metadata summaries are injected into the context of each slice to ensure that each slice contains complete chapter semantic information. At the same time, relevant legal provisions are retrieved from the legal knowledge base according to the review dimensions involved in each slice, and the retrieved legal provisions are injected into the slice context as a reference.

[0083] S403, Concurrent Review Execution: Each slice, carrying system and user prompts, is concurrently sent to the second language model. The system prompts define the review dimensions, output format, and positioning requirements. Review dimensions include compliance of review standards, fair competition clauses, SME policies, security deposit compliance, completeness of eligibility requirements, standardization of procurement needs, consistency of contract terms, completeness of content, standardization of text, and standardization of sensitive words and content. Concurrency control limits the number of slices processed simultaneously through a semaphore mechanism.

[0084] S404, Paragraph Anchor Point Location Parsing: The paragraph number reference returned by the second language model is parsed into the corresponding block-level identifier, and the violation location is accurately located through paragraph anchor point index mapping; for violations that cannot be located by paragraph number, normalized text matching is used to perform a fallback search in the document block.

[0085] S405. Quality verification and deduplication of violation results: Verify whether the matching text of each violation actually exists in the original document, and extract alternative text from the evidence field for non-existent matching text; perform flow restriction and deduplication on violation results with the same rule name.

[0086] Furthermore, the review dimensions include:

[0087] Review of evaluation criteria: whether the total score of the test is full marks, whether the sum of the scores of the sub-items is equal to the score of the major category, whether there are any unquantified subjective scoring factors, and whether the evaluation method is unique and clear;

[0088] Fair competition review: whether the test restricts or excludes suppliers under unreasonable conditions, including social security contribution requirements, registered capital restrictions, geographical restrictions, specific brand targeting, requirements for test reports to bear the manufacturer's official seal, and implicit exclusion of small and medium-sized enterprises by certification requirements;

[0089] SME policy review: Check whether the industry classification of SMEs matches the procurement target, whether price deduction discounts are repeatedly set for projects specifically for SMEs, and whether there are any missing price deduction discount clauses for SMEs;

[0090] Sensitive word and content standardization review: Simultaneously conduct screening across six dimensions, including sensitive content, classified information, inappropriate language, official document standardization, abnormal symbol format, and personal privacy information.

[0091] S5. Accurately locate violations in the outputs of the two large language models using a plain text anchoring engine; specifically including:

[0092] Problem type classification: Based on the verb characteristics in the violation description, violations are divided into two categories: existence violations and missing violations; existence violations refer to the presence of non-compliant content in the document, while missing violations refer to the absence of required content in the document;

[0093] For existing violations, a four-level progressive search is performed: the first level searches for matching text within the chapter range specified by the rule, the second level searches within the extended range, the third level searches within the entire text, and the fourth level falls back to the chapter node level for location; after each level of search is successful, a corresponding confidence limit is set;

[0094] For missing violations, the search scope is determined based on a predefined mapping table of domain keywords to expected chapters, and the most relevant node-level position is located in the expected chapter;

[0095] The output anchoring results include block-level identifier, character offset, matched text, confidence level, anchoring type, and search summary.

[0096] S6, the third-layer strategy adjudication agent performs evidence-level conflict detection on the output results of the special large model and the global large model, identifies the differences in conclusions, evidence positions, and numerical extraction between the two models on the same review dimension, performs automatic adjudication on deterministic conflicts, and leaves high-risk uncertain conflicts to be adjudicated by the user.

[0097] Furthermore, the steps in S6 specifically include the following:

[0098] S601. Conflict Identification: The output results of the first-layer specialized large-scale model agent and the second-layer global large-scale model agent are correlated and matched according to the rule dimension. The correspondence between the two layers of results is established through the rule identifier mapping table and the rule name alias table. For the same review dimension that is matched, the following conflict types are detected: conclusion difference, severity difference, evidence content difference, evidence uncertainty, numerical extraction difference, and review scope difference.

[0099] S602, Numerical Normalization Adjudication: When the conflict type is numerical extraction difference, the numerical values ​​extracted by both parties are normalized, including removing thousands separators, unifying the monetary unit, and unifying the case. If the normalized values ​​are consistent, the adjudication is automatically to merge and adopt them.

[0100] S603, Rule Priority Adjudication: When the type of rule involved in the conflict belongs to the set of deterministic calculation tasks, including cross-chapter numerical consistency verification, summary verification, working day calculation verification, announcement period verification, mandatory field verification and threshold verification, and there is no difference in the scope of review, the conclusion of the special large model will be automatically adopted.

[0101] S604, Semantic Priority Adjudication: When the reasoning process of the global large model agent contains semantic context keywords, and the specialized large model agent has marked the result as requiring manual review, the global large model conclusion is automatically adopted. Semantic context keywords include abbreviations, aliases, summaries, refinements, provisional estimates, quantifications, ambiguities, subjective opinions, and contextual information. When adjudicating, the conclusion of the party containing references to the legal knowledge base is adopted first.

[0102] S605, User Reservation of Decision: For conflicts with Level 1 warnings and opposing conclusions, conflicts with insufficient reliability of evidence location, and conflicts with different evidence locations, the final decision shall be reserved to the user.

[0103] S7. Input the final violation results after adjudication and the unified document model into the document rendering engine, and accurately highlight all violation locations in the document preview based on block-level identifiers and character offsets.

[0104] Furthermore, the steps in S7 specifically include the following:

[0105] S701. Render the document preview page using a sequence of document blocks in the unified document model, with each rendered block carrying its block-level identifier as a positioning attribute.

[0106] S702. Group all violation results by block-level identifiers and precisely mark the violation positions in each document block by character offset;

[0107] S703. Differentiated visual markings are used based on the source and type of violation: violations by specific large model agents are marked with the first color, violations by global large model agents are marked with the second color, violations by sensitive words are marked with the third color, and violations with conflicts are marked with the fourth color dashed line.

[0108] S704. Generate a list of violations for quick location, supporting clicks to jump to the corresponding violation location in the document;

[0109] S705. Since rule detection and document rendering use the same unified document model data, the location of violations does not require runtime text matching, and the location accuracy is 100%.

[0110] Furthermore, to quantify the quality of system review, a complete evaluation index system based on a confusion matrix is ​​established; let the manually labeled set of violations be... The system outputs the set of violations as follows: The basic quantities in the four quadrants are defined as follows:

[0111] True positive: The number of violations correctly detected by the system;

[0112] False positive: The number of violations falsely reported by the system;

[0113] False negative: The number of violations that were missed by the system;

[0114] True negative: The system correctly identified the review points as compliant. For all sets that violate the rules.

[0115] Based on the above four-dimensional limit, the following core indicators are defined:

[0116] Precision rate measures the proportion of true violations in the system output: ;

[0117] Recall rate measures the extent to which a system covers actual violations: ;

[0118] Score, harmonic mean of precision and recall: ;

[0119] Accuracy measures the percentage of correct judgments made by the system as a whole. .

[0120] The Matthews correlation coefficient (MCC), which comprehensively considers the balance of the four quadrants, has a value range of [-1, 1] and is more robust to class imbalance scenarios. .

[0121] AUC measures the overall discrimination capability of a system across different confidence thresholds. For each violation output by the system... and its confidence level By traversing the threshold Calculate the true positive rate at different thresholds and false positive rate .

[0122] ,

[0123] ,

[0124] .

[0125] In addition, the following system-level metrics are defined:

[0126] Positioning accuracy measures the degree of match between the system's output of violation locations and manually marked locations.

[0127] Automatic conflict resolution rate measures the proportion of conflicts that a policy-resolving agent automatically handles.

[0128] Parallel acceleration efficiency measures the time savings of a parallel architecture compared to serial execution.

[0129] Another embodiment of the present invention relates to a document compliance review system based on a three-layer model collaborative verification decision-making system. The system includes a unified document parsing module, a document structure indexing module, a special large model intelligent agent, a global large model intelligent agent, a strategy adjudication intelligent agent, a document rendering and highlighting module, and a regulatory knowledge base.

[0130] Unified Document Parsing Module: Used to perform unified document parsing on uploaded procurement documents, build a unified document model, decompose the document content into a sequence of document blocks with stable block-level identifiers, and extract document structure information and key field metadata.

[0131] Document Structure Indexing Module: Used to build multi-level document structure indexes, identify the chapter hierarchy of documents, and support text retrieval with a range limit based on chapter path;

[0132] Specialized large-scale intelligent agent: Based on document structure indexing, it uses regularization methods and domain fine-tuning of the first large language model to perform deterministic compliance rule detection on documents, and retrieves relevant legal clauses from the legal knowledge base through a retrieval enhancement generation mechanism to assist in review and judgment, and outputs rule violation results with block-level identifier location information;

[0133] Global large model intelligent agent: Construct paragraph anchor indexes for the unified document model, embed paragraph numbers into the document text, and inject relevant legal clauses retrieved from the legal knowledge base into the review context. Then, send it to the second large language model in the same domain for fine-tuning. The second large language model performs in-depth semantic review of the document and outputs semantic violation results with paragraph anchor references and legal basis.

[0134] Strategy adjudication agent: Performs evidence-level conflict detection on the output results of the special large model agent and the global large model agent, identifies the differences in conclusions, evidence positions, and numerical extraction between the special large model agent and the global large model agent on the same review dimension, performs automatic adjudication on deterministic conflicts, and leaves high-risk uncertain conflicts to be adjudicated by the user.

[0135] Document rendering and highlighting module: The final violation results after adjudication are input into the document rendering engine along with the unified document model, and all violation locations are accurately highlighted in the document preview based on block-level identifiers and character offsets;

[0136] Legal knowledge base: Used to manage procurement-related laws, regulations, departmental rules, and local normative documents, supporting the classification management of knowledge items, vectorized index construction, and semantic similarity retrieval.

[0137] Furthermore, the system-integrated retrieval generation (RAG) mechanism, serving as the shared infrastructure of the three-layer engine, provides regulatory knowledge support for both the first-layer specialized large-scale model agent and the second-layer global large-scale model agent. The regulatory knowledge base stores procurement-related laws, regulations, departmental rules, and local normative documents, with each knowledge entry vectorized and indexed semantically.

[0138] For the first-layer specialized large-scale model agent, when the extended rule engine executes rules requiring legal basis, such as those concerning margin compliance and SME policies, it encodes the current review context as a query vector q and retrieves the K most relevant regulatory clauses from the regulatory knowledge base:

[0139] ,

[0140] ,

[0141] in, The cosine similarity is used as the basis for determining whether to trigger or not to issue a warning, and is also used to enrich the legal citations for violation results.

[0142] For the second-layer global large-scale model agent, in the structure-aware intelligent slicing stage, based on the content of each slice's chapters (such as qualification requirements, review methods, contract terms, etc.), the corresponding legal provisions are retrieved from the legal knowledge base, and the retrieval results are injected into the context of the slice's prompt words as a reference. This allows the large language model to not only rely on its own training knowledge when performing semantic review, but also to cite specific legal provisions, and the output violation results contain clear legal basis.

[0143] Since the two layers of intelligent agents share the same legal knowledge base, the third-layer policy adjudication intelligent agent can compare the quality of legal citations in the results of violations by both parties when performing semantic priority adjudication, and give priority to the conclusion of the party with more sufficient legal basis, thereby making the adjudication process more objective and reliable.

[0144] The document parsing stage of this invention takes time to complete. During the parallel review phase, A1 (specific large model agent) and A2 (global large model agent) execute the parsed documents independently and in parallel, with a processing time of [time value missing]. , For the time taken by A1, The time taken for A2; during the adjudication and rendering phases, the time taken for A3 (the policy adjudication agent) to output the maximum set of violations is... The total system time is + + .

[0145] Compared to the total time required to perform two layers of review sequentially + + + Parallel speedup ratio is ,because, and Parallelism typically constitutes the majority of system time consumption, and the two are of similar magnitude. The parallel speedup ratio approaches 2. In practice, for a typical 50-100 page procurement document, the parallel architecture reduces the total review time from 60-120 seconds in serial mode to 35-70 seconds.

[0146] The above description is merely a preferred embodiment of the present invention. It should be understood that the present invention is not limited to the forms disclosed herein and should not be construed as excluding other embodiments. It can be used in various other combinations, modifications, and improvements, and can be altered within the scope of the concept described herein through the above teachings or related technologies or knowledge. Modifications and variations made by those skilled in the art that do not depart from the spirit and scope of the present invention should be within the protection scope of the appended claims.

Claims

1. A document compliance review method based on a three-layer model for collaborative verification decision-making, characterized in that: The method includes: Step 1: Preprocess and parse the uploaded documents to build a unified document model and construct a multi-level document structure index; Step 2: A specialized large-scale model agent, fine-tuned and trained from the first large-scale language model, performs deterministic compliance checks on the document based on predefined rule configuration and document structure index. At the same time, a global large-scale model agent, fine-tuned and trained from the second large-scale language model, utilizes the native semantic reasoning capabilities of the second large-scale language model to perform in-depth semantic review of the document in parallel. The simultaneous use of a global large-scale model agent, fine-tuned and trained by the second large-scale language model, to perform deep semantic review of documents in parallel, leveraging the native semantic reasoning capabilities of the second large-scale language model, includes: The global large model agent constructs paragraph anchor indexes, assigns a unique number to each non-empty paragraph in the document, embeds the number into the document text, and generates review text with anchor indexes. Based on the document structure index, intelligent slicing is performed according to chapter boundaries. Each slice is injected with metadata summary. At the same time, legal clauses are retrieved from the legal knowledge base according to the chapter content involved in the slice and injected into the slice context. Each slice carries system prompts and regulatory references and is sent concurrently to the second language model via semaphore control. The second language model performs a ten-dimensional deep semantic review on each slice and returns a structured violation result. Parse paragraph numbers into block-level identifiers and perform a fallback search of normalized text for violations that cannot be located. Verify whether each violating matching text actually exists in the original document; extract alternative text from the evidence field for non-existent matching text; and perform rate limiting and deduplication on violating results with the same rule name. Step 3: Perform evidence-level conflict detection and automatic adjudication on the output results of the special large model agent and the global large model agent through the policy adjudication agent; Step 4: Render and highlight the final violation results after the ruling, and display the review results.

2. The document compliance review method based on a three-layer model collaborative verification decision-making according to claim 1, characterized in that: Step one specifically includes the following: A1. Users upload procurement documents in a set format through the front-end quick review page. The parser traverses the document's XML format, converts paragraph and table elements into a sequence of document blocks, assigns a stable block-level identifier to each block, and extracts key field metadata and table data. After parsing, it outputs the number of blocks, the number of tables, the text length, and the number of structural nodes. A2. Construct a multi-level document structure index based on the unified document model, identify chapter level, section level, subsection level and attachment title, establish a five-level structure tree, and convert the unified document model into a compatible format.

3. The document compliance review method based on a three-layer model collaborative verification decision-making according to claim 1, characterized in that: The specialized large-scale intelligent agent, fine-tuned and trained from the first large-scale language model, performs deterministic compliance checks on documents based on predefined rule configuration and document structure indexing, including: Based on the document structure index, regular expression methods and the first language model are used to perform deterministic compliance rule checks on the documents, including cross-chapter consistency checks, summary checks, working day calculation checks, threshold comparison checks, and required field integrity checks. By using a retrieval-enhanced generation mechanism, regulations are retrieved from the regulatory knowledge base to assist in review and judgment, and rule violation results with block-level identifier location information are output.

4. The document compliance review method based on a three-layer model collaborative verification decision-making according to claim 1, characterized in that: Step three includes: By using a rule identifier mapping table and a rule name alias table, the output results of the special large model intelligent agent and the global large model intelligent agent are associated and matched according to the review dimensions to detect differences in conclusions, differences in evidence location, and differences in numerical extraction. The four-level strategy is executed sequentially: numerical normalization decision, rule priority decision, semantic priority decision, and user decision retention. When making semantic-priority rulings, the quality of legal citations in the violation results is compared, and the conclusion of the party with more sufficient legal basis is adopted first. Conflicts that are automatically ruled are handled directly, while conflicts that require user ruling are marked and presented to the user.

5. The document compliance review method based on a three-layer model collaborative verification decision-making according to claim 2, characterized in that: A1 includes: A101. Parse the XML structure of the procurement document, traverse the paragraph and table elements in the document, and generate a stable block-level identifier for each element based on the path and sequence number of the element in the document using a hash algorithm. A102. For each document block, record its block type, region type, text content, normalized text, starting line number, parent node path, node number, starting character position, and ending character position; A103. Perform key field extraction on the entire document. Extract structured metadata such as project name, project number, procurement budget, maximum price, procurement method, contract performance period, bid deadline, bid security amount and announcement date through regular expressions and context analysis. Each extracted field is associated with its block-level identifier. A104. Perform enhanced parsing on the tables in the document to identify the procurement item table and the review criteria table, extract the table headers and row data, and map each cell in the table to an independent document block.

6. The document compliance review method based on a three-layer model collaborative verification decision-making according to claim 2, characterized in that: A2 includes: A201. Scan the document block sequence and use regular expressions to identify chapter-level headings, section-level headings, subsection-level headings, and attachment headings, and construct a document structure tree containing five levels: cover, chapter, section, subsection, and attachments. A202. Each structure node records its hierarchy, title, key value, starting block index, ending block index, and list of child nodes; A203 provides a path query interface, which supports obtaining a set of document blocks within a specified chapter range through expressions and performing regular expression searches within a specified range.

7. The document compliance review method based on a three-layer model collaborative verification decision-making according to claim 3, characterized in that: For cross-chapter consistency verification, target field values ​​are extracted from multiple specified chapters, and the extracted values ​​are normalized before consistency comparison is performed. An alert is triggered when any two valid extracted values ​​are inconsistent after normalization. For summary verification, extract the total amount and the list of itemized amounts respectively, convert all amounts to the same unit, and compare the difference between the itemized summary and the total amount. When the difference exceeds the preset tolerance threshold, trigger an alarm. For working day calculation verification, extract the date range from the document, calculate the number of working days, and compare it with the statutory minimum working day requirement; For threshold comparison verification, relevant numerical fields are extracted and their size is compared. When the value exceeds the statutory threshold, an alert is triggered.

8. The document compliance review method based on a three-layer model collaborative verification decision-making according to claim 1, characterized in that: The method further includes: accurately locating the violation results output by the first and second language models, specifically including: Problem type classification: Based on the verb characteristics in the violation description, violations are divided into two categories: existence violations and missing violation violations; For existing violations, a four-level progressive search is performed: the first level searches for matching text within the chapter range specified in the rules, the second level searches within the extended range, the third level searches within the entire text, and the fourth level falls back to the chapter node level for location. After each level of search is successful, a corresponding confidence limit is set. For missing violations, the search scope is determined based on a predefined mapping table of domain keywords to expected chapters, and the most relevant node-level position is located in the expected chapter; The output anchoring results include block-level identifier, character offset, matched text, confidence level, anchoring type, and search summary.

9. A document compliance review system based on a three-layer model for collaborative verification and decision-making, characterized in that: The system includes a unified document parsing module, a document structure indexing module, a specialized large model intelligent agent, a global large model intelligent agent, a policy adjudication intelligent agent, a document rendering and highlighting module, and a legal knowledge base. The unified document parsing module is used to perform unified document parsing on uploaded procurement documents, build a unified document model, decompose the document content into a sequence of document blocks with stable block-level identifiers, and extract document structure information and key field metadata. The document structure indexing module is used to construct a multi-level document structure index, identify the chapter hierarchy of a document, and support text retrieval with a range limit based on chapter paths. The specialized large-scale model intelligent agent: based on document structure indexing, it uses regularization methods and the first large language model to perform deterministic compliance rule detection on documents, and retrieves regulatory clauses from the regulatory knowledge base through a retrieval enhancement generation mechanism to assist in review and judgment, and outputs rule violation results with block-level identifier location information; The global large model intelligent agent: constructs paragraph anchor indexes for the unified document model, embeds paragraph numbers into the document text, injects review context by combining legal clauses retrieved from the legal knowledge base, and sends it to the second large language model. The second large language model performs in-depth semantic review of the document and outputs semantic violation results with paragraph anchor references and legal basis. The strategy adjudication agent performs evidence-level conflict detection on the output results of the special large model agent and the global large model agent, identifies the differences in conclusions, evidence positions, and numerical extraction between the special large model agent and the global large model agent in the same review dimension, performs automatic adjudication on deterministic conflicts, and reserves the decision-making of high-risk uncertain conflicts to the user. The document rendering and highlighting module: inputs the final violation result after adjudication and the unified document model into the document rendering engine, and accurately highlights all violation locations in the document preview based on block-level identifiers and character offsets; The aforementioned legal knowledge base is used to manage procurement-related laws, regulations, departmental rules, and local normative documents, and supports the classification management of knowledge items, the construction of vectorized indexes, and semantic similarity retrieval.

Images