Document element extraction method based on AI large model technology

Abstract

The invention discloses a document element extraction method based on an AI large model technology. The method comprises the following steps that various receipts input by a user are received; carrying out analysis and layout analysis on the document through a visual language model or a text analysis technology to generate a unified intermediate representation; a dynamic grading module is adopted to evaluate document complexity from three dimensions of format, structure and variation, and the document complexity is divided into a simple regular type, a structure complex type or a height variation type; adaptively selecting a processing flow according to a rating result, wherein the processing flow comprises high-speed template matching, multi-modal cognitive fusion or intelligent agent driving processing; the processing result is converted into an EasyEX format which is easy to process; the method comprises the following steps of: converting a natural language demand into an execution rule by extracting an Agent and utilizing a Prompt dynamic compiling technology, and positioning and extracting a target field by LLM (Logical Language Model); and finally, outputting structured data after type verification, knowledge graph verification and compliance review. According to the invention, the universality, the accuracy and the automation degree of receipt processing are improved, and the labor cost is greatly reduced.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and document processing, and in particular to a method for extracting document elements based on AI large model technology. Background Technology

[0002] Currently, element extraction for various documents (such as invoices, transaction records, and certificates) mainly employs traditional methods. These methods include rule-based systems based on template matching, where templates or regular expression rules are pre-designed for each document type. By recognizing text in images, key information is located and extracted according to the templates or rules. This method works when document formats are fixed and types are limited, but it requires tedious template design and rule writing for each document type, lacking versatility. With the development of deep learning, some traditional models based on Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs) have emerged, along with information extraction models combining layout analysis (such as LayoutLM). These models have improved the ability to understand complex document layouts to some extent. Furthermore, some Natural Language Processing (NLP) models are used to extract fields from text, such as Named Entity Recognition models based on Conditional Random Fields (CRFs) or BERT, used to extract information such as the names of the parties involved in a contract, the amount, etc.

[0003] Disadvantages of existing technologies: Although the above methods have achieved certain results in their respective fields, they still have significant limitations. First, traditional template- and rule-based methods have poor generalization ability: once the document format or type changes, the template and rules need to be redesigned, making them unable to adapt to diverse and dynamically changing document styles. Second, limited field coverage: templates and simple machine learning models can usually only extract a limited number of predefined fields, making it difficult to handle new fields or complex semantic relationships appearing in documents. Third, insufficient accuracy and robustness: traditional technologies are sensitive to image quality and layout complexity, and perform poorly in recognizing handwritten, blurred, or unconventional document layouts. Although deep learning outperforms traditional methods in complex scenarios, it still requires specific optimization for specific tasks, and its capabilities in layout analysis and cross-field reasoning are limited. Finally, high manual intervention and maintenance costs: both template adjustment and model training require a large amount of manual participation and professional knowledge, resulting in slow system iteration and high maintenance costs.

[0004] Case Comparison: Interbank transaction documents are diverse, including spot bond transactions, repurchase transactions, and other types such as lending and outright purchases. While traditional solutions can customize / pre-train recognition templates based on an exhaustive list of document types, today's business needs extend far beyond simply extracting surface-level information. Especially in financial transactions, it's crucial to accurately identify the trading party and the counterparty. The trading party is not always a fixed buyer / repurchase party, and based on this accurate identification, the remaining related fields must be extracted. In the open-ended fund statement scenario, the business user needs to extract relevant fields from statements from over 100 fund companies. The field descriptions vary among fund companies, such as: fund abbreviation / full fund name / securities abbreviation / holding name, etc. The order of fields in the table also differs, such as fund account / trading account / securities code / securities name, etc. Complex table structures such as merged cells and multi-page tables are also encountered. Even if traditional solutions can meet business needs, they lack advantages in terms of resource investment, document modification, and self-maintenance.

[0005] In summary, existing document element extraction technologies fall short of meeting the needs of enterprises in terms of versatility, accuracy, and efficiency when processing massive amounts of diverse documents. This solution achieves breakthroughs by leveraging Large Language Models (LLM) and Visual Language Models (VLM), demonstrating unprecedented performance in natural language understanding, image content awareness, contextual reasoning, and generation capabilities. These technologies are profoundly changing the field of document understanding and automated processing, particularly in handling massive amounts of diverse financial documents, showing disruptive potential. Summary of the Invention

[0006] In view of this, the present invention addresses the shortcomings of existing technologies by providing a document element extraction method based on AI large-scale model technology. This method aims to solve the core pain points of existing document element extraction technologies, such as poor generalization ability and inability to adapt to diverse and dynamically changing document formats. Its design aims to construct an end-to-end intelligent processing system by integrating the semantic understanding capabilities of large-scale language models (LLMs) and the visual parsing capabilities of visual language models (VLMs). Ultimately, this achieves high-precision, automated element extraction and verification for documents of any format, significantly improving the efficiency and reliability of document processing.

[0007] To achieve the above objectives, the present invention adopts the following technical solution: A document element extraction method based on AI large model technology includes the following steps: S001, Receive the document entered by the user; S002. The physical format of the document is identified through the type conversion module. After format identification and layout analysis, the system generates a structured intermediate representation. S003. The intermediate representation is sent to the dynamic grading module. The dynamic grading module calls the format analyzer, structure parser and variability detector in parallel to obtain the three-dimensional features of the document's format complexity, structural complexity and variability, and evaluates the grade in real time through a pre-trained lightweight decision tree model. S004. Based on the complexity level obtained in step S003, the corresponding differentiated processing flow is invoked; for structurally complex or highly variable documents, the processing flow uses a collaborative decision-making mechanism to integrate the layout analysis results of VLM and the semantic analysis results of LLM to generate processing instructions. S005. Convert the content processed in step S004 into EasyEX format. The EasyEX format is based on Markdown, with HTML tables and LaTeX tags added to form a unified intermediate document. S006. The extraction agent in the extraction engine receives the field requirements described by the user in natural language, uses the Prompt dynamic compilation technology to compile the field requirements described in natural language into an executable rule sequence in real time, and the LLM locates and extracts the corresponding fields and values ​​in the EasyEX format document. S007. The type validation module performs format and logic validation on the extraction results obtained in step S006 based on the pre-configured regular expressions, numerical ranges and business rules. If an error is found, the back pressure mechanism is triggered, and the document fragments corresponding to the error fields are re-sent into the extraction engine for iterative extraction. S008. The upstream and downstream verification modules query the association paths and attribute constraints of the extracted elements in the knowledge graph through the business knowledge graph engine, perform cross-field consistency verification, and the compliance semantic firewall performs deep semantic matching and reasoning with the built-in compliance rule library to output a comprehensive verification report containing error details, risk level and suggested handling measures. S009. Output the results of all structured elements that have passed all verifications in JSON, XML or Excel format, or write them directly to the downstream business system.

[0008] As a preferred option: In step S002, the layout analysis further includes: Recognizes the bounding boxes and type labels of text blocks, tables, charts, and images page by page; Visual topology analysis is used to detect cross-page or column tables, and the tables are merged based on header consistency and text coherence. Based on the semantic understanding of document layout using VLM, the reading priority between elements is predicted, and logical sorting is generated in scenarios with columns and nested tables. Based on the physical structure, LayoutLM identifies heading levels, paragraph categories, and table semantic roles, and outputs structured JSON or XML.

[0009] As a preferred embodiment: in step S003, the format analyzer is used to detect file metadata, including file type, resolution, and text optionality; The structure parser is used to quantify element density, table nesting levels, text-image ratio, and number of formulas. The variability detector is used to compare the current document with the built-in standard layout library, calculate the layout hash distance and semantic drift of key elements, and output the non-standard variability score. The lightweight decision tree model integrates the above three-dimensional features and outputs L1 / L2 / L3 levels in real time.

[0010] As a preferred option: In step S004, the collaborative decision-making mechanism is implemented through a real-time collaborative decision-making unit. This unit has a built-in gating mechanism to receive the semantic analysis results of LLM and the visual structural information of VLM, and dynamically generate fusion processing instructions based on the preset "visual-guided semantic" or "semantic-guided visual" collaborative rules to realize cross-modal information alignment, verification and reasoning.

[0011] As a preferred embodiment: Step S006, the Prompt dynamic compilation technology includes: The natural language compiler uses an LLM interpreter to compile user-input natural language instructions into a machine-executable sequence of rules containing multi-step logic in real time. The rule execution engine parses and executes the rule sequence in the sandbox environment, and schedules LLM and VLM tools to locate, extract and judge the target document; The online learning feedback loop captures users' corrective actions on the extracted results, transforming each correction behavior into a structured sample of "instruction-error result-correct result," which is stored in a dynamically updated error correction memory to guide the compiler in generating more accurate rules in subsequent tasks.

[0012] As a preferred embodiment: step S007, the back pressure mechanism includes: Immediately trigger single-field re-extraction for hard format errors; A re-extraction is triggered and the error is logged for numerical logic contradictions; After three semantic ambiguity errors are accumulated, a re-extraction will be triggered and manual intervention will be prompted. The type verification module also integrates a REACT mechanism, which enables LLM to generate and answer self-check questions based on the extraction results to verify the consistency between the extracted content and the original document context.

[0013] As a preferred solution: In step S008, the business knowledge graph engine stores entities, attributes, and relationships between entities within the document domain, and embeds business rules; The compliance semantic firewall has a built-in extensible compliance rule library that transforms legal provisions and risk control policies into machine-readable logic and performs compliance verification on the extracted results. The comprehensive verification report includes error details, risk level, and recommended handling measures.

[0014] As a preferred embodiment, the EasyEX format conversion step further includes: For text-based PDFs, extract text block coordinates and table structures through layout analysis, and convert headings into Markdown levels according to font size and indentation; Parse the DOCX document styles and correct style conflicts, and differentiate nested tables by border indentation to distinguish their hierarchical levels; Repair missing borders in Excel tables, identify weak styles for gray light borders, and generate a tree structure for multi-level headers; For TXT, the boundaries are inferred based on text density and spacing, and column names are located by combining semantic anchors. Logical borders are then added when generating Markdown tables. Cluster and concatenate multi-page tables according to the Y-axis coordinate and retain the multi-page continuation mark. Mark the range of rows and columns for merged cells.

[0015] As a preferred solution: the extracted Agent supports complex task decomposition. For multi-page documents, split them by chapter, extract the results separately, and then merge the results. The table data is first extracted using a table recognition model, and then the LLM interprets the meaning. For scenarios that require the integration of visual information, VLM is invoked to recognize seals and handwritten signatures, and the visual analysis results are integrated into the LLM inference process.

[0016] As a preferred option: In step S009, the output result is directly used for: intelligent renaming of interbank transaction orders. The system extracts the transaction date, counterparty, amount, and transaction type and generates standardized file names according to preset rules. The system extracts key elements from fund account statements and intelligently reconciles them with system data. The system automatically reconciles the extracted holdings table fields with the valuation system data and generates a discrepancy report. This method supports concurrent processing on ordinary hardware.

[0017] Compared with existing technologies, this invention has significant advantages and beneficial effects. Specifically, as can be seen from the above technical solution, firstly, it greatly improves the system's versatility and flexibility, completely eliminating the dependence on predefined templates. New document types or extracted fields can be adapted simply by modifying natural language instructions, effectively addressing the challenges of diverse and dynamically changing document formats. Secondly, it significantly improves the accuracy and robustness of extraction. The collaborative working mechanism of LLM and VLM enables it to deeply understand the semantic context and visual layout of documents. Its processing capabilities for complex tables, cross-page content, handwriting, and seals far surpass traditional solutions, and it can effectively suppress model "illusions" through REACT self-checking and knowledge graph verification. Thirdly, it achieves an optimal balance between processing efficiency and resource consumption. The dynamic hierarchical mechanism ensures that simple documents are processed quickly, while complex documents are allocated more computing resources for in-depth analysis. This allows the system to support high-concurrency processing even on ordinary hardware, with single-file processing time being only 1 / 3 of that required by manual processing. Fourth, it significantly reduces development and maintenance costs. The template-free design avoids tedious template creation and rule writing, and the online learning capability enables the system to optimize itself, greatly reducing the amount of manual intervention and maintenance work in the later stages. It provides enterprises with an efficient, economical and reliable automation solution for massive document processing scenarios.

[0018] To more clearly illustrate the structural features and effects of the present invention, a detailed description is provided below in conjunction with the accompanying drawings and specific embodiments. Attached Figure Description

[0019] Figure 1 This is a flowchart of the method and system for extracting elements from various documents using LLM-VLM according to the present invention. Figure 2 This is a schematic diagram of the AI-powered intelligent file renaming (interbank bill - transaction order) architecture of the present invention; Figure 3 This is a schematic diagram of the fund reconciliation statement structure of the present invention; Figure 4 This is a schematic diagram of the three-dimensional evaluation system architecture based on dynamic hierarchical classification of the present invention; Figure 5 This is a schematic diagram of the method steps of the present invention. Detailed Implementation

[0020] The present invention is as follows Figures 1 to 5 As shown, a document element extraction method based on AI large model technology includes the following steps: S001. Receiving Documents: The system provides a unified file entry point at the front end, supporting users to upload common file formats such as PDF, Word, JPG, PNG, TIF, and BMP in batches via drag-and-drop or dialog boxes. It also supports pushing enterprise content management systems, email attachments, and scanner output streams directly into the receiving queue via API. "Users input the documents to be extracted (which may be PDFs, Word files, scanned images, etc.) into the system." In this embodiment, the receiving module first performs virus scanning, size verification, and format sniffing on the files, generating a unique task ID and storing the original files in object storage. Subsequently, the task scheduler distributes tasks to idle nodes based on cluster load, ensuring sub-second response times even in high-concurrency scenarios. For extremely large files, the system employs a chunked upload and breakpoint resume mechanism to avoid retransmissions due to network jitter. For encrypted PDFs, the decryption SDK is called to complete decryption in memory, preventing plaintext from being stored. After upload, the front end immediately returns a progress bar, allowing users to view the entire "queue-parsing-extraction-verification-output" status in real time, achieving visual tracking.

[0021] S002, Type Conversion and Layout Restoration: The physical format of the document is identified through the type conversion module. After format identification and layout analysis, the system generates a structured intermediate representation.

[0022] After a file enters the type conversion module, the system first uses a double check of the file header's Magic Byte and the file extension to determine its true format. If an image or scanned document is detected, the system immediately invokes the Visual Language Model (VLM) to perform text recognition and layout analysis. "Using VLM, the following process can be used to achieve a comprehensive analysis of the structure of complex documents:" 1. Physical Object Detection: Leveraging VLM's image-text joint understanding capabilities, the system identifies physical elements such as text blocks, tables, charts, and images in a document page by page, labeling them with bounding boxes and type tags. In actual deployment, VLM employs an Encoder-Decoder architecture, using Swin-Transformer-large for image encoding and Qwen3-14B for text decoding. It is further pre-trained on a large-scale Chinese document dataset to improve its detection accuracy for elements such as table lines, handwriting, seals, and QR codes. The system first performs edge filling and distortion correction on each page's image before inputting it into VLM to obtain the bounding box coordinates, category labels, and reading order confidence scores for each text block, table, and image. For cross-page tables; 2. Cross-page (column) object merging: Through visual topology analysis, cross-page / column tables or long paragraphs are detected and merged based on position alignment and semantic similarity (such as header consistency and text coherence) to construct a global element relationship graph. In this embodiment, the system uses a Y-axis projection clustering algorithm to automatically merge tables on adjacent pages with overlapping vertical coordinates and header text similarity greater than 0.9, and adds a "cross-page continuation" tag to the JSON to ensure accurate row and column correspondence during subsequent extraction. For column layouts, the system detects column boundaries by using the width of blank gaps and vertical coordinate jumps, and then uses the Transformer language model to score the semantic coherence of the text within the column, automatically determining the reading order from left to right or top to bottom. Subsequently, the system calls LayoutLMv3-large to perform semantic role classification on the text blocks, identifying title level, paragraph affiliation, and table row and column titles, and outputs structured JSON with coordinates, level, and role labels for direct use by downstream modules. If the input is a native PDF or Word document, the OCR step is skipped, and the text layer and table coordinates are extracted directly. The same JSON structure is output, ensuring that the image and the electronic document continue to flow on the same data plane.

[0023] S003, Dynamic Hierarchical Decision-Making; After obtaining the structured JSON, the task immediately enters the dynamic hierarchical module. "This invention designs a dynamically hierarchical adaptive processing module. This module evaluates the complexity of the document through a three-dimensional evaluation system and dynamically adjusts the processing strategy accordingly to achieve hierarchical processing of documents with different complexities. The three-dimensional feature evaluation includes: format complexity evaluation, structural complexity evaluation, and variability evaluation, which are implemented by the format analyzer, structure parser, and variability detector, respectively." In this embodiment, the format analyzer first reads the file metadata: if it is a scanned document, it records the resolution, color depth, compression method, and presence of moiré patterns; if it is a PDF, it records text selectability, font embedding rate, and whether it contains transparent layers. The structure parser performs statistics on the JSON: total number of pages, text block density (average number of text blocks per page), table nesting level, maximum number of table rows and columns, image-to-text ratio, number of formulas, percentage of merged cells, number of tables spanning multiple pages, etc. The variability detector compares the current document with a built-in "standard layout library": the system pre-sets standard layout hashes for commonly used documents in the banking, fund, insurance, and government sectors; by calculating the layout hash distance and semantic drift of key elements (such as "amount" and "date"), it outputs a non-standard variation score between 0 and 1. The 3D evaluation engine uses a lightweight XGBoost decision tree model, integrating dozens of features including format, structure, and variability, and outputs one of three levels in real time: L1-simple regularity, L2-structural complexity, and L3-high variability. Model inference takes less than 5ms per iteration on the CPU and can scale linearly with workload.

[0024] The internal structure and working principle of the dynamic hierarchical module are as follows: Figure 4 As shown: This figure illustrates the workflow of the key pre-module for adaptive processing in this invention—the dynamic hierarchical system. After receiving the "input document," the system initiates three analyses in parallel via the "3D evaluation engine": 1. Format analysis: Detect file metadata (such as type and resolution).

[0025] 2. Structural analysis: quantifying element density, table nesting depth, etc.

[0026] 3. Variation detection: Compare with the standard layout library to calculate layout hash and semantic drift.

[0027] The engine integrates the results of the above "format, structure, and variability analysis" and uses a pre-trained lightweight decision tree model to output the "complexity level" in real time, classifying documents into three categories: L1 - Simple and Regular: Documents with fixed format and simple structure.

[0028] L2 - Complex Structure: Documents containing complex tables and mixed text and graphics.

[0029] L3 - Highly Variable: Documents with non-standard layouts and varied formats.

[0030] Based on the rating results, the system dynamically invokes differentiated processing flows through "policy routing": L1 level routing to "high-speed template matching process" (using preset rules and dictionaries for high-speed processing).

[0031] L2 routing leads to the "standard multimodal processing flow" (invoking LLM and VLM for collaborative processing).

[0032] L3 level routing leads to the "agent adaptive processing flow" (driven by an agent and capable of online learning).

[0033] This module is the core of the resource optimization and efficient processing of this invention, ensuring that the system can process simple documents at high speed as well as handle complex documents accurately.

[0034] S004, Differentiated Processing Chains: Based on the hierarchical results, the system calls different processing chains. "For L1 level documents, a high-speed template matching process is called; for L2 level documents, a standard multimodal cognitive fusion process is called; for L3 level documents, an agent-driven adaptive processing process is called." For L1, the system directly uses preset regular expressions and dictionaries for field capture, with CPU time typically <100ms and a single node supporting 200 QPS. For L2, "The real-time collaborative decision-making unit has a built-in dynamic fusion model based on a gating mechanism, used to receive the semantic analysis results from the LLM module and the visual structure information from the VLM module, and dynamically generate fusion processing instructions based on preset 'visual-guided semantics' or 'semantic-guided vision' collaborative rules to achieve cross-modal information alignment, verification, and reasoning." In this embodiment, the gating mechanism uses Sigmoid weighted fusion: when visual confidence > 0.9 and text confidence < 0.7, the "visual-guided semantics" branch is enabled, prioritizing table coordinate positioning; otherwise, the "semantic-guided vision" branch is enabled, prioritizing text context positioning. The fused feature vectors are fed into Qwen3-14B for field extraction. A single inference GPU session takes approximately 400ms, with a GPU memory usage of <12GB. For L3, the system initiates an intelligent agent chain: the natural language compiler compiles user prompts into executable rule sequences in real time; the rule execution engine schedules LLM and VLM tools within a sandbox; and the online learning feedback loop records user corrections and updates the error correction memory in real time, ensuring adaptive capabilities even under extreme conditions.

[0035] S005 and EasyEX format conversion: Regardless of whether the upstream result is OCR or text extraction, it must be converted to the EasyEX intermediate format. "EasyEX processing solution overview: 1. Multi-file type processing capability: For text-based PDFs, layout analysis extracts text block coordinates and table structures; nested headings are converted to Markdown levels based on font characteristics (font size / indentation) and semantic keywords (e.g., 'Chapter X'); out-of-order paragraphs are semantically rearranged using LLM and logical comments are added; for DOCX documents, document styles (e.g., heading 1 / 2 / 3) are parsed, style conflicts are automatically corrected, nested tables are differentiated by border indentation, and rowspan / colspan is marked during cell merging conversion; Excel repair..." For tables with missing borders, weak styles such as light gray borders are identified. Multi-level table headers are generated into a tree structure based on row height / column width. Merged cells retain their original range markers. Nested tables generate parent-child relationships. For TXT tables, boundaries are inferred based on text density and spacing, and column names are located using semantic anchors. Logical borders are added when generating Markdown tables. 2. Complex table processing solution: When borders are unclear, boundaries are inferred based on cell text density and spacing, and column names are located using semantic anchors such as 'Total'. Logical borders are added when generating Markdown tables. Cross-page tables are clustered and stitched according to Y-axis coordinates, retaining 'Total'.The '<!--Continued table across pages-->' tag enables automatic continuation of cell content with semantic coherence verification; nested / merged cells identify the indentation differences in the inner table, generating a nested JSON structure. When converting merged cells, the cross-row and column ranges are marked (such as A1:C3), preserving the original layout logic." In actual deployment, the system first runs PDFPlumber on the PDF to obtain vector lines, and then performs an intersection operation with the text block coordinates output by VLM to complete the missing borders; for Excel, it first reads the cell styles through OpenPyXL. If the wireframe color is <30% gray, it is regarded as a "weak border", and the logical boundary is automatically extended; for tables across pages, the system clusters and stitches them according to the Y-axis coordinates, and writes the '<!--Continued table across pages-->' comment in the JSON to ensure row and column alignment during subsequent extraction. The final output EasyEX document is a mixed text of Markdown + HTML table + LaTeX formula, which not only retains human readability but also facilitates extraction by regular expressions or LLMs.

[0036] S006, Prompt Dynamic Compilation and Extraction: Extraction Agent receives user field requirements described in natural language and generates executable rule sequences in real time through Prompt dynamic compilation technology. "Natural Language Compiler: The core is the Prompt dynamic compilation technology. This module takes the user-input natural language instructions (Prompt) and compiles them in real time into a structured, multi-step, machine-executable rule sequence through a Large Language Model (LLM) interpreter. Rule Execution Engine: Used in a secure sandbox environment to parse and execute the above rule sequence, scheduling the underlying LLM and VLM tools to perform operations such as locating, extracting, and judging the target document. Online Learning Feedback Loop: The core is an incremental learning mechanism based on error feedback. This module is used to capture the user's corrective actions on the agent's execution results (such as modification, deletion, and supplementation), and transform each error correction behavior into a structured learning sample of '(instruction, error result, correct result)', stored in a dynamically updated 'error correction memory', used to guide the compiler to generate more accurate rules in subsequent tasks." In this embodiment, the user inputs "Please extract the name of Party A, signing date, and contract amount (case-sensitive)," and the natural language compiler compiles it into a three-step rule: 1. Locate the title containing "Party A" or "Party". 1. Extract the paragraph with "A"; 2. Extract the first entity in the paragraph that matches the company name regular expression; 3. Locate the "Amount" row and extract the numerical and Chinese capitalized amounts. The rule execution engine calls Qwen3-14B to perform extraction within the sandbox, taking approximately 400ms per extraction. If the user corrects "ABC Co., Ltd." to "ABC Corporation Limited" in the result, the online learning feedback loop immediately records the triple and performs incremental fine-tuning of the compiler during off-peak hours at night via LoRA, so that "ABC Corporation Limited" is output first when extracting contracts from the same customer next time. In addition, the extraction agent also supports complex task decomposition: splitting multi-page documents by chapter, extracting separately, and then merging the results; first calling the table recognition model to extract data from tabular data, and then using LLM to understand the meaning; for scenarios that require combining visual information, calling VLM to recognize seals and handwritten signatures and integrating the visual analysis results into the LLM inference process.

[0037] S007, Type Validation and Backpressure + REACT; The type validation module checks each item according to pre-configured regular expressions, numerical ranges, and business rules. "Backpressure handling methods: 1. Hard format errors, level: high, such as: ID number, email, date, etc., not conforming to regular expression rules (e.g., date format '2025-02-30'), immediately triggering single-field re-extraction, without setting a retry interval; 2. Numerical logic contradictions, level: medium, such as: amount fields exceeding the reasonable business range (e.g., single bond transaction amount < 100 yuan) or inconsistent capitalization, triggering single-field re-extraction and recording the error; 3. Semantic ambiguity errors, level: low, such as: field names having ambiguity (e.g., 'Party A' is unclear) but the format is correct, triggering re-extraction after 3 similar errors, requiring manual intervention to supplement the context." Also integrates a REACT mechanism; REACT Template Library Examples: 1. Date / Time Field: 'Please verify that the [field value] is consistent with the time order of the [related fields, such as 'Effective Date'] in the document, and whether there are any contradictions across years / months?' 2. Amount / Numerical Field: 'Is the [field value] consistent with the calculation logic of the [related fields, such as 'Subtotal Amount'] in the document (e.g., quantity × unit price = total)? Please check whether the number of decimal places conforms to business specifications.' 3. Entity Name Field: 'Is the [field value, such as company name] consistent with the description in other places in the document (e.g., stamp, signature)? Are there any cases of confusion between abbreviations and full names?' 4. Logical Relationship Field: 'Does the relationship between [Field A value] and [Field B value] conform to business rules (e.g., when 'Transaction Type' is 'Repurchase', is the 'Term' within a reasonable range)?' In this embodiment, the system first performs regular expression validation on the extracted "Contract Amount" field. If there are more than two decimal places, it is marked as a hard format error, and back pressure is immediately triggered, so that the document fragment corresponding to the field is re-sent to the extraction engine. If the amount is inconsistent in capitalization, it is marked as a numerical logic contradiction, and re-extraction is triggered and logged. If there are multiple candidate entities for "Party A", the system will automatically pop up a manual review box after accumulating three similar errors, prompting the user to supplement the context.

[0038] S008, Upstream and Downstream Verification; The business knowledge graph engine and the compliance semantic firewall operate in conjunction. "Business Knowledge Graph Engine: Models and stores knowledge and rules within the business domain. The knowledge graph contains key entities (such as customers, products, projects, etc.), entity attributes, and relationships between entities, and also embeds business rules and constraints. Compliance Semantic Firewall: Connects in real-time with the business knowledge graph engine, and has a built-in scalable compliance rule library that transforms legal provisions and risk control policies into machine-readable logic. It outputs a comprehensive verification report containing error details, risk levels, and recommended handling measures." In this embodiment, the knowledge graph is stored using Neo4j, with nodes including "customer," "fund product," "counterpartie," and "contract," and edges including "signature," "holding," and "guarantee." The compliance rule library is written in Drools format, for example, "Repurchase transaction period must not exceed 365 days." When the system extracts "transaction type = repurchase" and "term = 400 days," the compliance semantic firewall immediately triggers, outputting a verification report: "High risk: Repurchase period exceeds regulatory limit, it is recommended to return to business personnel for verification."

[0039] S009. Result Output and Example Application: All structured elements that pass verification are output in JSON, XML, or Excel format, or directly written into downstream business systems. Example 1: The "Intelligent Renaming Application for Interbank Transaction Orders" is a document management solution based on artificial intelligence technology, primarily designed for financial institutions' standardized management needs for a large number of interbank transaction orders. This application automatically identifies and extracts key information from transaction orders using AI technology, such as transaction date, counterparty, transaction amount, and transaction type. The system first identifies the type of the input document; after confirming it is an interbank transaction order, it initiates in-depth content analysis to accurately extract key information fields. Subsequently, the system standardizes and converts the extracted information according to preset naming rules, generating a format conforming to the institution's requirements. The system names the file and performs a renaming operation. This process reduces manual intervention, improves efficiency, lowers the human error rate, and ensures standardized and consistent file management. For cases of incomplete information extraction, the system initiates a manual intervention mechanism to ensure the accuracy and completeness of the renaming results. Simple comparative testing shows that manually renaming a transaction document takes approximately 30-40 seconds per file, while the system takes only one-third of that time. It supports 10 concurrent requests on standard hardware, significantly improving processing efficiency. Furthermore, scheduled tasks and rule-based tasks can be set according to business needs, allowing for immediate processing of incoming documents and fully leveraging the advantages of automation. The specific implementation process of this invention in the intelligent renaming application of interbank transaction orders is as follows: Figure 2 As shown; This figure fully illustrates the end-to-end business process of the system described in this invention for processing interbank transaction orders and implementing intelligent renaming.

[0040] Detailed Explanation of Process and Principles: The entire process begins with the "file collection" phase. The system receives user-uploaded documents through the unified file entry point described in step S001, supporting batch processing.

[0041] The process then moves to the "file type recognition" stage. The system determines whether the input document is in PDF, Word, or image format by detecting the file header (Magic Byte) and extension, preparing for subsequent stream processing.

[0042] The next crucial step is "interbank transaction document recognition." This step goes far beyond simple format recognition; its core is determining the document's business type and complexity. The system needs to confirm that the document being processed is an "interbank transaction document," not a contract or invoice. This is typically achieved by combining LLM's semantic understanding with VLM's layout analysis. For example, it identifies whether the document contains keywords such as "transaction document," "counterparties," or "bond code," and whether it conforms to the typical layout characteristics of this type of document. Only when it is recognized as an "interbank transaction document" will the subsequent dedicated renaming process be triggered.

[0043] After confirming the document type, the system initiates multiple "key information extraction" tasks in parallel, including: 1. Transaction Date Recognition: Extract the core transaction date field.

[0044] 2. Counterparty Identification: Accurately identify the names of counterparty institutions that transact with us.

[0045] 3. Transaction Amount Recognition: Extract the transaction amount value.

[0046] 4. Transaction type identification: Identify and classify transaction types, such as spot bond trading, repurchase, and lending.

[0047] 5. Extraction of other key elements: Extract other important information besides the fields mentioned above.

[0048] 6. AI Document Content Analysis: This is a core component, referring to the use of artificial intelligence models to perform deep semantic analysis on the entire document. Its purpose is not merely OCR, but to understand the document's overall content, logical structure, and contextual relationships to ensure the completeness and accuracy of the extracted key information and to resolve ambiguities (e.g., distinguishing between "trading counterparty" and "settlement counterparty"). This directly demonstrates the powerful contextual understanding and reasoning capabilities of LLM (Local Management System).

[0049] After all information has been extracted, the system performs a core logical check: "Is the information extraction complete?". This step verifies whether the extracted key information (date, counterparty, amount, type, etc.) is complete, accurate, and of acceptable quality to meet the customization requirements of subsequent renaming rules. If the check is "no" (i.e., the information is incomplete or of poor quality), the process ends or the backpressure mechanism is triggered, sending the erroneous fields back to the extraction engine for iterative extraction. If the check is "yes," the process proceeds to the next stage.

[0050] The next step is to standardize and format the extracted raw information: Date formatting: Converts the recognized date into a standard format such as "YYYYMMDD".

[0051] Transaction type code mapping: Maps the identified transaction type text to the standard business code agreed upon by the organization.

[0052] Application of opponent data: Standardize the names of opponents, such as by using abbreviations or codes recognized by the internal system.

[0053] Amount formatting: Standardizes the format of monetary values.

[0054] Record serial number: Generate a unique serial number for batch file processing to avoid duplicate names and ensure orderly file management.

[0055] The formatted data is "applied naming rules". The system generates standardized filenames according to the naming rules preset by the user in Example 1 (e.g., "{Transaction Date}_{Counterparty Code}_{Transaction Type Code}_{Amount}").

[0056] Then, the system performs a file renaming operation and subsequently verifies the renaming result. Verification includes checking if the renaming was successful and whether the new filename is compliant. If the verification passes ("Verification Passed" in the diagram), the system updates the metadata records in the file management system, marking the task as "complete."

[0057] This complete process clearly demonstrates the specific implementation method of "the output results can be directly used for intelligent renaming of interbank transaction orders," covering the entire automated operation from file input to final file system update. The two features in the figure, "AI document content analysis" and "Is the information extraction complete?", particularly highlight the intelligence and robustness of this invention compared to traditional template technologies.

[0058] Example 2: "The application of open-ended fund statement element extraction and intelligent system data verification is an intelligent data processing system that combines large-scale language model (LLM) and visual language model (VLM). This system, targeting complex tabular documents such as open-ended fund statements, accurately parses complex table structures, including merged cells and nested tables, through powerful structure recognition capabilities." In this example, the system automatically retrieves email attachments from fund companies every morning, performs batch parsing, and then verifies the extracted results against the valuation system data line by line, generating a discrepancy report for accounting staff confirmation. The daily report processing volume reaches 30,000 pages, and the average reconciliation time per transaction is reduced from 15 minutes manually to 30 seconds by the system.

[0059] Appendix Figure 3 This invention provides a detailed description of the automated processing and verification workflow for highly complex and non-standardized documents such as open-ended fund statements. The entire process deeply integrates the layout understanding capabilities of a Visual Language Model (VLM), the semantic reasoning capabilities of a Large Language Model (LLM), and a verification mechanism based on business knowledge.

[0060] The process begins with "Entry of Fund Statement Documents". The system receives statement files (PDF, images, etc.) from hundreds of different fund companies in various formats.

[0061] The next stage is "document preprocessing." This stage is not just a simple format conversion, but a multimodal parsing process. The system calls a Visual Modeling Library (VLM) (such as LayoutLM) to perform detailed layout analysis, identify visual elements in the document such as text blocks, table areas, and seals, and output structured data with coordinates and type labels. For scanned documents, OCR text recognition is performed first.

[0062] The preprocessed data then enters the core stage of "document structure recognition and analysis." This is crucial for processing fund reconciliation statements. The system needs to deeply analyze its complex structure. Identify merged cells: Accurately determine the row span and column span of a cell to avoid misalignment during subsequent data extraction.

[0063] Concatenating cross-page tables: By clustering based on Y-axis coordinates and matching the consistency of table headers, table fragments scattered across multiple pages are logically reconstructed into a complete table.

[0064] Analyze nested structures: handle possible parent-child table relationships and clarify data hierarchy.

[0065] Semantic role labeling: Identifies and labels the semantic roles of key columns such as "fund account", "securities code", "securities name", "shares", "net asset value", and "market value", laying the foundation for subsequent accurate extraction. The output of this step is a highly structured, machine-understandable document representation.

[0066] With a clear document structure in place, the "core information extraction" process begins. This task is completed by the "Agent extraction" module. Based on user-defined requirements (such as "extract all holdings details"), the extraction agent uses Prompt dynamic compilation technology to generate execution instructions, scheduling the LLM to precisely locate and extract the values ​​of target fields from the parsed structured document. The "...XXXX" in the diagram indicates that this process can flexibly extract a large number of different fields, demonstrating the system's scalability.

[0067] The extraction results are immediately taken over by the "Agent Verification" module for a rigorous "data quality check." The verification includes: Format verification: Check whether the fund account number is a 12-digit number and whether the securities code conforms to the exchange's coding rules, etc.

[0068] Logical verification: Using the REACT mechanism, LLM is required to verify whether the calculation logic such as "shares × net value = market value" is self-consistent.

[0069] If the check is "unqualified", the system will "mark it as abnormal" and trigger the back pressure mechanism, sending the problematic data back to the extraction stage or prompting manual intervention. If the check is "qualified", it will proceed to the final stage.

[0070] "Intelligent Data Verification." The system does not process documents in isolation, but automatically compares extracted high-quality data (such as a holdings list) with downstream business databases such as the "valuation system" line by line. This process embodies the idea of ​​a "business knowledge graph engine," which performs consistency checks by querying entity relationships.

[0071] If the comparison results are "inconsistent", a "difference record" will be generated and "data verification" may be required to determine whether it is an extraction error or a system accounting problem.

[0072] Finally, the system generates a clear "verification result report" and "completes" the entire process.

[0073] The core principle of this process lies in the collaboration between VLM and LLM to transform unstructured, complex documents into machine-readable structured data. Then, through a three-stage pipeline of extraction, validation, and verification, it ultimately achieves seamless integration and value extraction with business systems. It not only solves the problem of "understanding," but also addresses the business challenges of "accurate usage" and "fast verification," reducing the average reconciliation time per transaction from 15 minutes manually to 30 seconds in the system, fully demonstrating the practicality and advanced nature of this invention.

[0074] like Figure 1As shown, the overall processing flow of the method described in this invention is as follows: This figure depicts the overall processing pipeline of the core method of this invention. The process begins with "Start" and receives "Input Document". First, the "Document Format Recognition" and "Invoice Type Recognition" modules parse the input document to determine its format (PDF / Word / image, etc.) and business document type (such as invoice, transaction slip, etc.). This process is completed by the "Document Conversion Module". If it is a scanned document or image, VLM is called for OCR and layout analysis; if it is a PDF / Word, the text and table structure are directly extracted, and a unified intermediate representation is output.

[0075] The process then proceeds to the "EasyEX format conversion" stage. This module converts the upstream parsing results into an intermediate format based on Markdown, with added HTML tables and LaTeX formulas, forming an "output JSON" structure based on Markdown + table + LaTeX. This step is performed by the "preprocessing module," which aims to generate a program-friendly and semantically structure-preserving unified document representation.

[0076] After format conversion, the "Extraction Agent" begins its work. It receives user requests and uses Prompt dynamic compilation technology to schedule the "LLM to execute extraction tasks," locating and extracting target fields and values ​​from the EasyEX document.

[0077] The extracted results then enter the verification phase. First, "type verification" is performed, which checks the field format according to pre-configured rules (such as regular expressions and numerical ranges). Simultaneously, the system integrates the "REACT module," which activates "LLM self-check mode," requiring the LLM to generate self-check questions based on the extraction results and answer them with reference to the original text to verify content consistency.

[0078] The decision point is checked for the "Validation Passed?" result. If it fails, feedback is sent to the front end or a re-examination is triggered; if it passes, the "Validation Agent" performs deeper "format and type validation + logic validation", that is, cross-field consistency validation is performed through the business knowledge graph, and compliance semantic firewall performs compliance verification.

[0079] Finally, after the "Final verification passed?" decision point is "yes", the system "outputs the results" and "ends" the process, outputting or writing the structured data to the downstream system in formats such as JSON, XML, or Excel.

[0080] The key design focus of this invention is to construct an intelligent processing framework that is multimodal, collaborative, dynamically adaptive, and possesses continuous learning capabilities. Its core lies in two aspects: First, a dynamic hierarchical system based on a lightweight decision tree model is used to perform a three-dimensional evaluation of the input document's format, structure, and variability, classifying it in real-time into three complexity levels: L1 (simple regularity), L2 (structural complexity), or L3 (high variability). Based on this, the system dynamically routes the data to differentiated processing flows (high-speed template matching, multimodal cognitive fusion, or agent-driven processing), thereby achieving optimized allocation of system computing resources and precise adaptation of processing strategies. Second, a unified intermediate representation format called EasyEX is designed. Based on Markdown, it integrates HTML tables and LaTeX formulas to convert documents from different sources into structured documents that retain human readability while also being program-friendly, laying the foundation for subsequent high-precision extraction. Finally, an innovative agent-driven architecture is adopted, the core of which is "Prompt dynamic compilation technology" and "online learning feedback loop". This allows users to flexibly configure extraction requirements through natural language commands, and the system can compile them into executable rules in real time. It can also continuously learn and evolve from the user's corrective behavior. Finally, combined with business knowledge graph and compliance semantic firewall, it can achieve comprehensive verification from the field level to the business logic level, ensuring the high reliability of the output results.

[0081] The above description is merely a preferred embodiment of the present invention and does not constitute any limitation on the technical scope of the present invention. Therefore, any minor modifications, equivalent changes, and alterations made to the above embodiments based on the technical essence of the present invention shall still fall within the scope of the technical solution of the present invention.

Claims

1. A document element extraction method based on AI large model technology, characterized in that, Includes the following steps: S001, Receive the document entered by the user; S002. The physical format of the document is identified through the type conversion module. After format identification and layout analysis, the system generates a structured intermediate representation. S003. The intermediate representation is sent to the dynamic grading module. The dynamic grading module calls the format analyzer, structure parser and variability detector in parallel to obtain the three-dimensional features of the document's format complexity, structural complexity and variability, and evaluates the grade in real time through a pre-trained lightweight decision tree model. S004. Based on the complexity level obtained in step S003, the corresponding differentiated processing flow is invoked; for structurally complex or highly variable documents, the processing flow uses a collaborative decision-making mechanism to integrate the layout analysis results of VLM and the semantic analysis results of LLM to generate processing instructions. S005. Convert the content processed in step S004 into EasyEX format. The EasyEX format is based on Markdown, with HTML tables and LaTeX tags added to form a unified intermediate document. S006. The extraction agent in the extraction engine receives the field requirements described by the user in natural language, uses the Prompt dynamic compilation technology to compile the field requirements described in natural language into an executable rule sequence in real time, and the LLM locates and extracts the corresponding fields and values ​​in the EasyEX format document. S007. The type validation module performs format and logic validation on the extraction results obtained in step S006 based on the pre-configured regular expressions, numerical ranges and business rules. If an error is found, the back pressure mechanism is triggered, and the document fragments corresponding to the error fields are re-sent into the extraction engine for iterative extraction. S008. The upstream and downstream verification modules query the association paths and attribute constraints of the extracted elements in the knowledge graph through the business knowledge graph engine, perform cross-field consistency verification, and the compliance semantic firewall performs deep semantic matching and reasoning with the built-in compliance rule library to output a comprehensive verification report containing error details, risk level and suggested handling measures. S009. Output the results of all structured elements that have passed all verifications in JSON, XML or Excel format, or write them directly to the downstream business system.

2. The document element extraction method based on AI large model technology according to claim 1, characterized in that: In step S002, the layout analysis further includes: Recognizes the bounding boxes and type labels of text blocks, tables, charts, and images page by page; Visual topology analysis is used to detect cross-page or column tables, and the tables are merged based on header consistency and text coherence. Based on the semantic understanding of document layout using VLM, the reading priority between elements is predicted, and logical sorting is generated in scenarios with columns and nested tables. Based on the physical structure, LayoutLM identifies heading levels, paragraph categories, and table semantic roles, and outputs structured JSON or XML.

3. The document element extraction method based on AI large model technology according to claim 1, characterized in that: In step S003, the format analyzer is used to detect file metadata, including file type, resolution, and text selectability; The structure parser is used to quantify element density, table nesting levels, text-image ratio, and number of formulas. The variability detector is used to compare the current document with the built-in standard layout library, calculate the layout hash distance and semantic drift of key elements, and output the non-standard variability score. The lightweight decision tree model integrates the above three dimensions of features and outputs L1 / L2 / L3 levels in real time.

4. The document element extraction method based on AI large model technology according to claim 1, characterized in that: In step S004, the collaborative decision-making mechanism is implemented through a real-time collaborative decision-making unit. This unit has a built-in gating mechanism to receive the semantic analysis results of LLM and the visual structural information of VLM, and dynamically generate fusion processing instructions based on the preset "visual-guided semantic" or "semantic-guided visual" collaborative rules to realize cross-modal information alignment, verification and reasoning.

5. The document element extraction method based on AI large model technology according to claim 1, characterized in that: In step S006, the Prompt dynamic compilation technology includes: The natural language compiler uses an LLM interpreter to compile user-input natural language instructions into a machine-executable sequence of rules containing multi-step logic in real time. The rule execution engine parses and executes the rule sequence in the sandbox environment, and schedules LLM and VLM tools to locate, extract and judge the target document; The online learning feedback loop captures users' corrective actions on the extracted results, transforming each correction behavior into a structured sample of "instruction-error result-correct result," which is stored in a dynamically updated error correction memory to guide the compiler in generating more accurate rules in subsequent tasks.

6. The document element extraction method based on AI large model technology according to claim 1, characterized in that: In step S007, the back pressure mechanism includes: Immediately trigger single-field re-extraction for hard format errors; A re-extraction is triggered and the error is logged for numerical logic contradictions; After three semantic ambiguity errors are accumulated, a re-extraction will be triggered and manual intervention will be prompted. The type verification module also integrates a REACT mechanism, which enables LLM to generate and answer self-check questions based on the extraction results to verify the consistency between the extracted content and the original document context.

7. The document element extraction method based on AI large model technology according to claim 1, characterized in that: In step S008, the business knowledge graph engine stores entities, attributes, and relationships between entities within the document domain, and embeds business rules. The compliance semantic firewall has a built-in extensible compliance rule library that transforms legal provisions and risk control policies into machine-readable logic and performs compliance verification on the extracted results. The comprehensive verification report includes error details, risk level, and recommended handling measures.

8. The document element extraction method based on AI large model technology according to claim 1, characterized in that: The EasyEX format conversion step further includes: For text-based PDFs, extract text block coordinates and table structures through layout analysis, and convert headings into Markdown levels according to font size and indentation; Parse the DOCX document styles and correct style conflicts, and differentiate nested tables by border indentation to distinguish their hierarchical levels; Repair missing borders in Excel tables, identify weak styles for gray light borders, and generate a tree structure for multi-level headers; For TXT, the boundaries are inferred based on text density and spacing, and column names are located by combining semantic anchors. Logical borders are then added when generating Markdown tables. Cluster and concatenate multi-page tables according to the Y-axis coordinate and retain the multi-page continuation mark. Mark the range of rows and columns for merged cells.

9. The document element extraction method based on AI large model technology according to claim 1, characterized in that: The Agent extraction supports complex task breakdown: For multi-page documents, split them by chapter, extract the results separately, and then merge the results. The table data is first extracted using a table recognition model, and then the LLM interprets the meaning. For scenarios that require the integration of visual information, VLM is invoked to recognize seals and handwritten signatures, and the visual analysis results are integrated into the LLM inference process.

10. The document element extraction method based on AI large model technology according to claim 1, characterized in that: In step S009, the output results are directly used for: intelligent renaming of interbank transaction orders, where the system extracts the transaction date, counterparty, amount, and transaction type and generates standardized file names according to preset rules; intelligent verification of fund account statement elements with system data, where the system automatically verifies the extracted holdings table fields with the valuation system data and generates a difference report; the method supports concurrent processing on ordinary hardware.

Images