Document element extraction method based on AI large model technology

Abstract

The application discloses a document element extraction method based on AI large model technology, comprising the following steps: receiving various kinds of documents input by users; analyzing and layout analyzing the document through a visual language model or a text analysis technology to generate a unified intermediate representation; adopting a dynamic grading module to evaluate the document complexity from three dimensions of format, structure and variability, and dividing the document into a simple and regular type, a structural complex type or a high variability type; adaptively selecting a processing flow according to the grading result, including high-speed template matching, multi-modal cognitive fusion or agent-driven processing; converting the processing result into an EasyEX format; converting a natural language requirement into an execution rule by an extraction agent using a Prompt dynamic compilation technology, positioning and extracting a target field by an LLM; and finally outputting structured data after type checking, knowledge graph verification and compliance review. The application improves the universality, accuracy and automation degree of document processing, and greatly reduces the labor cost.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence and document processing technology, in particular to a document element extraction method based on AI large model technology. BACKGROUND

[0002] Current element extraction for various types of documents (such as invoices, transaction logs, certificates, etc.) mainly uses traditional methods. Traditional methods include rule systems based on template matching, i.e., pre-designing templates or regular rules for each type of document, recognizing the text in the image, and then positioning and extracting key information according to the template or rule. This method can work in cases where the document format is fixed and the type is limited, but it requires tedious template design and rule writing for each document type, and lacks universality. With the development of deep learning, some traditional models based on convolutional neural networks (CNN) and recurrent neural networks (RNN) have emerged, as well as information extraction models combined with layout analysis (such as LayoutLM), which have improved the understanding ability of complex layout documents to some extent. In addition, some natural language processing (NLP) models are used to extract fields from text, such as conditional random field (CRF) or BERT-based named entity recognition models, which are used to extract information such as Party A, Party B, and amount in contracts.

[0003] Limitations of existing technology: Although the above methods have achieved certain results in their respective fields, they still have obvious limitations. First, traditional template and rule-based methods have poor generalization ability: once the document format or type changes, templates and rules need to be redesigned, and they cannot adapt to diverse and dynamic document styles. Second, the coverage of fields is limited: templates and simple machine learning models can usually only extract a limited number of predefined fields, making it difficult to handle new fields or complex semantic relationships in documents. Third, accuracy and robustness are insufficient: traditional techniques are sensitive to image quality and layout complexity, and perform poorly on handwritten, blurred, or unconventional layout documents. Although deep learning performs better than traditional methods in complex scenarios, it still needs to be optimized for specific tasks, and has limited ability in layout analysis and cross-field reasoning. Finally, the cost of manual intervention and maintenance is high: whether it is template adjustment or model training, it requires a lot of human involvement and professional knowledge, resulting in slow system iteration and high maintenance costs.

[0004] Case comparison: The types of transaction documents between banks are diverse, including physical bond transaction documents, repo transaction documents, and other types such as lending and buyout. Although the traditional solution can customize / pre-train the recognition template based on the exhaustive document type, today's business needs have long gone beyond the content extraction of the appearance. Especially for financial businesses, it is necessary to accurately identify the party and the counterparty, and based on the accurate identification of the party, to identify and extract the remaining fields related to the party. In the opening account statement scenario, the business user needs to extract the field data related to the holding table in the account statement from more than 100 fund companies. The field expressions of each fund company are different. For example, the security name: fund nickname / fund full name / security nickname / holding name, etc. The order of the fields in the table may also be different, such as fund account number / transaction account number / security code / security name. It will also encounter complex table structures such as merged cells and cross-page tables. The traditional solution can meet the business needs, but it does not have advantages in resource investment, document changes, and self-maintenance.

[0005] In summary, the existing document element extraction technology is difficult to meet the needs of enterprises in handling massive and diversified documents in terms of generality, accuracy, and efficiency. The present solution has made breakthrough progress in natural language understanding, image content perception, context reasoning, and generation capabilities using large language models (LLM) and visual language models (VLM). These technologies are deeply changing the field of document understanding and automated processing, especially in handling massive and diversified financial documents, showing revolutionary potential. SUMMARY

[0006] Therefore, the present application aims to solve the core pain points of the existing document element extraction technology, which has poor generalization ability and cannot adapt to diversified and dynamically changing document formats. The design purpose is to build an end-to-end intelligent processing system by integrating the semantic understanding ability of large language models (LLM) and the visual analysis ability of visual language models (VLM). Finally, high-precision and automated element extraction and verification of any version of the document are realized, greatly improving the efficiency and reliability of document processing.

[0007] To achieve the above-mentioned purpose, the present application adopts the following technical solutions:

[0008] A document element extraction method based on AI large model technology, comprising the following steps:

[0009] S001, receiving a document input by a user;

[0010] S002, the physical format of the document is identified by the type conversion module, after format recognition and layout analysis, the system generates a structured intermediate representation;

[0011] S003, the intermediate representation is sent to the dynamic grading module, the dynamic grading module calls the format analyzer, structure parser and variability detector in parallel to obtain the three dimensional features of the document format complexity, structure complexity and variability, and evaluates the grade in real time through the pre-trained lightweight decision tree model;

[0012] S004, according to the complexity level obtained in step S003, the corresponding differential processing flow is called; for the structure complex or high variability document, the processing flow fuses the layout analysis result of VLM and the semantic analysis result of LLM through the cooperative decision mechanism to generate the processing instruction;

[0013] S005, the content processed by step S004 is converted into EasyEX format, which is based on Markdown, with HTML table and LaTeX mark added, forming a unified intermediate document;

[0014] S006, the extraction agent in the extraction engine receives the field requirement described in natural language by the user, and uses the Prompt dynamic compilation technology to compile the natural language description of the field requirement into an executable rule sequence in real time, and locates and extracts the corresponding field and value in the EasyEX format document by LLM;

[0015] S007, the type checking module checks the extraction result obtained in step S006 according to the preconfigured regular expression, numerical range and business rules for format and logic checking, and if an error is found, the back pressure mechanism is triggered to re-input the document segment corresponding to the error field into the extraction engine for iterative extraction;

[0016] S008, the upstream and downstream verification module queries the associated path and attribute constraint of the extracted element in the knowledge graph through the business knowledge graph engine, performs cross-field consistency verification, and the compliance semantic firewall performs deep semantic matching and reasoning on the extracted result and the built-in compliance rule library, and outputs a comprehensive verification report containing error details, risk level and recommended treatment measures;

[0017] S009, the structured element result passing through all the verification is output in JSON, XML or Excel format, or directly written into the downstream business system.

[0018] As a preferred scheme: in step S002, the layout analysis further includes:

[0019] The boundary box and type label of the text block, table, chart and picture are identified page by page;

[0020] Detect cross-page or columnar tables through visual topology analysis, merge with table header consistency and text coherence;

[0021] Based on the semantic understanding of document layout by VLM, predict the reading priority between elements, and generate logical sorting in columnar and nested table scenarios;

[0022] On the basis of physical structure, identify title level, paragraph attribution and table semantic role through LayoutLM, and output structured JSON or XML.

[0023] As a preferred solution: in step S003, the format analyzer is used to detect file metadata, including file type, resolution, and text selectivity;

[0024] The structure parser is used to quantify element density, table nesting level, image-text ratio, and formula quantity;

[0025] The variability detector is used to compare the current document with the built-in standard version library, calculate the layout hash distance and key element semantic drift, and output the non-standard variability score;

[0026] The lightweight decision tree model integrates the above three-dimensional features and outputs L1 / L2 / L3 levels in real time.

[0027] As a preferred solution: in step S004, the collaborative decision mechanism is realized through a real-time collaborative decision unit, which is built-in with a gating mechanism for receiving semantic analysis results from LLM and visual structure information from VLM, and based on the preset "visual guided semantics" or "semantic guided vision" collaborative rules, dynamically generates fusion processing instructions to realize cross-modal information alignment, verification and reasoning.

[0028] As a preferred solution: in step S006, the Prompt dynamic compilation technology includes:

[0029] The natural language compiler converts the user's natural language instructions into a machine-executable rule sequence containing multiple-step logic in real time through the LLM interpreter;

[0030] The rule execution engine parses and executes the rule sequence in a sandbox environment, schedules LLM and VLM tools to locate, extract and judge the target document;

[0031] The online learning feedback loop captures the user's correction operations on the extraction results, converts each correction behavior into a "instruction-error result-correct result" structured sample, and stores it in a dynamically updated correction memory library to guide the compiler to generate more accurate rules in subsequent tasks.

[0032] As a preferred solution: step S007, the back pressure mechanism comprises:

[0033] Trigger single-field re-extraction immediately for format hard errors;

[0034] Trigger re-extraction and record errors for numerical logical contradictions;

[0035] Trigger re-extraction and prompt manual intervention after accumulating three times of semantic ambiguity errors;

[0036] The type verification module also integrates the REACT mechanism, so that the LLM generates self-check questions and answers based on the extraction results, to verify the consistency of the extraction content and the context of the original document.

[0037] As a preferred solution: step S008, the business knowledge graph engine stores entities, attributes and relationships between entities in the single document field, and embeds business rules;

[0038] The compliance semantic firewall has an extensible compliance rule library built-in, which converts legal provisions and risk control policies into machine-readable logic to perform compliance verification on the extraction results;

[0039] The comprehensive verification report contains error details, risk levels and recommended treatment measures.

[0040] As a preferred solution: the EasyEX format conversion step further comprises:

[0041] For text-based PDF, extract text block coordinates and table structure through layout analysis, and convert titles to Markdown levels according to font size and indentation;

[0042] For DOCX, parse document styles and correct style conflicts, and distinguish levels according to border and indentation for nested tables;

[0043] For Excel, repair line frame missing tables, identify gray light style, and generate tree structure for multi-level table headers;

[0044] For TXT, infer boundaries based on text density and spacing, combine semantic anchor points to locate column names, and generate Mark

[0045] down table with complete logical borders;

[0046] For cross-page tables, cluster and splice them according to Y-axis coordinates and keep cross-page continuation table markers, and label cross-row and column ranges for merged cells.

[0047] As a preferred solution: the extraction agent supports complex task decomposition:

[0048] Split multi-page single documents by chapters, extract them separately, and then merge the results;

[0049] First, call the table recognition model to extract data from the table data, and then use LLM to understand the meaning.

[0050] For scenes that need to combine visual information, call VLM to recognize seals and handwritten signatures and integrate the visual analysis results into the LLM inference process.

[0051] As a preferred solution: in step S009, the output result is directly used for: interbank transaction sheet intelligent renaming, the system extracts the transaction date, counterparty, amount, and transaction type, and generates a standardized file name according to the preset rules;

[0052] The fund reconciliation statement element extraction and system data intelligent checking system automatically checks the extracted position table fields with the valuation system data and generates a difference report; the method supports concurrent processing under ordinary hardware.

[0053] Compared with the prior art, the present application has obvious advantages and beneficial effects. Specifically, from the above technical solution, first, the universality and flexibility of the system are greatly improved, and the dependence on predefined templates is completely eliminated. By modifying the natural language instructions, the new document type or extraction field can be adapted, effectively addressing the challenges of document format diversification and dynamic changes. Second, the accuracy and robustness of the extraction are greatly improved. The cooperative working mechanism of LLM and VLM enables it to deeply understand the semantic context and visual layout of the document, and its processing capacity for complex tables, cross-page content, handwritten elements, and seals is far superior to traditional solutions. It can effectively suppress the model "illusion" through REACT self-checking and knowledge graph verification. Third, the optimal balance between processing efficiency and resource consumption is achieved. The dynamic grading mechanism ensures that simple documents can be processed quickly, while complex documents are allocated more computing resources for in-depth analysis, enabling the system to support high-concurrency processing under ordinary hardware, with a single file processing time of only 1 / 3 of manual processing. Fourth, the development and maintenance costs are significantly reduced. The template-free design avoids the tedious template creation and rule writing, and the online learning capability enables the system to optimize itself, greatly reducing the need for manual intervention and maintenance workload, providing an efficient, economical, and reliable automated solution for enterprises in mass document processing scenarios.

[0054] To make the structure and function of the application clearer, specific embodiments will be described in detail below with reference to the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0055] Figure 1 The processing flowchart of the LLM-VLM method and system for extracting elements from various documents of the present application;

[0056] Figure 2 The AI intelligent file renaming (interbank bill - transaction sheet) architecture diagram of the present application;

[0057] Figure 3 A schematic diagram of a fund statement architecture of the present application;

[0058] Figure 4 A schematic diagram of a three-dimensional evaluation system architecture based on dynamic grading of the present application;

[0059] Figure 5 A schematic diagram of the method steps of the present application. DETAILED DESCRIPTION

[0060] As shown in the present application Figures 1 to 5 A document element extraction method based on AI large model technology, comprising the following steps:

[0061] S001, receiving a document; the system provides a unified file entry in the front end, supports users to upload PDF, Word, JPG, PNG, TIF, BMP and other common format files in batch at one time by dragging or dialog box, also supports pushing enterprise content management system, mail attachments, scanner output stream directly into the receiving queue through API interface. "The user inputs the document to be extracted (which may be a PDF, Word file, picture scan, etc.) into the system." In this embodiment, the receiving module first performs virus scanning, size checking and format sniffing on the file, generates a unique task ID, and stores the original file in the object storage; then, the task scheduler distributes the task to the idle node according to the cluster load, ensuring that the response can still be maintained in seconds under high concurrency scenarios. For large files, the system uses sharding upload and breakpoint resume mechanism to avoid network jitter caused by retransmission; for encrypted PDF, call decryption SDK to complete decryption in memory to avoid plaintext landing. After uploading, the front end returns a progress bar immediately, and the user can view the "queuing-analysis-extraction-verification-output" whole link status in real time to realize visual tracking.

[0062] S002, type conversion and layout restoration; the type conversion module is used to identify the physical format of the document, and after format identification and layout analysis, the system generates a structured intermediate representation.

[0063] After the file enters the type conversion module, the system first determines its real format through the double verification of the file header Magic Byte and the extension name; if a picture or scan is detected, the visual language model VLM is immediately called to perform character recognition and layout analysis. "By using VLM (visual language model), the panoramic structure analysis of complex documents can be realized through the following process:

[0064] 1. Physical object detection: Utilize the joint understanding of text and image of VLM to identify the bounding boxes and type labels of text blocks, tables, charts, pictures, etc. in the document page by page. In actual deployment, VLM adopts an Encoder-Decoder architecture, with Swin-Transformer-large for image encoding and Qwen3-14B for text decoding, and continues pre-training on large-scale Chinese document datasets to improve the detection accuracy of elements such as table lines, handwritten text, seals, and QR codes. The system first performs edge filling and distortion correction on each page image, then inputs it into VLM to obtain the bounding box coordinates, class labels, and reading order confidence of each text block, table, and picture. For cross-page tables,

[0065] 2. Cross-page (column) object merging: Through visual topology analysis, detect cross-page / column tables or long paragraphs, and combine position alignment and semantic similarity (such as table header consistency and text coherence) to construct a global element relationship graph. In this embodiment, the system uses a Y-axis projection clustering algorithm to automatically merge tables with overlapping vertical coordinates and table header text similarity greater than 0.9 in adjacent pages, and adds a "cross-page continuation table" marker in JSON to ensure accurate row and column correspondence during subsequent extraction. For columnar layout, the system detects column boundaries by analyzing the width of the blank gap and the vertical coordinate jump, and then uses a Transformer language model to score the semantic coherence of the text within the column to automatically determine the reading order of left-to-right or top-to-bottom. Subsequently, the system calls LayoutLMv3-large to perform semantic role classification on the text blocks to identify title levels, paragraph ownership, and table row and column titles, and outputs a structured JSON with coordinates, levels, and role labels for direct use by downstream modules. If the input is a native PDF or Word, the OCR step is skipped, and the text layer and table coordinates are directly extracted, and the same JSON structure is output to ensure that pictures and electronic files continue to flow on the same data plane.

[0066] S003、Dynamic classification decision; after obtaining the structured JSON, the task enters the dynamic classification module. The application designs a dynamic classification adaptive processing module. The module evaluates the complexity of the document through a three-dimensional evaluation system, dynamically adjusts the processing strategy accordingly, and realizes the hierarchical processing of documents with different complexities. The three-dimensional feature evaluation includes: format complexity evaluation, structure complexity evaluation and variation degree evaluation, which are realized by format analyzer, structure parser and variation degree detector. In this embodiment, the format analyzer first reads the file metadata: if it is a scan, record the resolution, color depth, compression method, and whether there are moire patterns; if it is a PDF, record the text selectivity, font embedding rate, and whether it contains a transparent layer. The structure parser performs statistics on JSON: total number of pages, text block density (average number of text blocks per page), table nesting level, maximum table row and column number, image-text ratio, formula number, merged cell ratio, and cross-page table number. The variation degree detector compares the current document with the built-in "standard format library": the system preloads the standard layout hash of common documents in the banking, fund, insurance and government industries; by calculating the layout hash distance and the semantic drift of key elements (such as "amount" and "date"), a non-standard variation score between 0 and 1 is output. The three-dimensional evaluation engine uses a lightweight XGBoost decision tree model, which integrates dozens of features such as format, structure and variation degree, and outputs one of L1-simple and regular type, L2-structure complex type and L3-high variation type in real time. The model inference takes <5ms on CPU, which can be linearly expanded with task volume.

[0067] The internal structure and working principle of the dynamic classification module are as shown in Figure 4 The figure illustrates the working process of the key pre-module of the application, the dynamic classification system, which realizes adaptive processing. After the system receives the "input document", the "three-dimensional evaluation engine" initiates three analyses in parallel:

[0068] 1. Format analysis: detect file metadata (such as type, resolution).

[0069] 2. Structure analysis: quantify element density, table nesting depth, etc.

[0070] 3. Variation degree detection: compare with the standard format library, calculate layout hash and semantic drift.

[0071] The engine integrates the results of the above "format, structure, and variation degree analysis", and through a pre-trained lightweight decision tree model, it outputs the complexity level in real time, and classifies the document into three categories:

[0072] L1-simple and regular type: documents with fixed format and simple structure.

[0073] L2-structure complex type: documents containing complex tables and image-text mixed layout.

[0074] L3 - Highly Variable: Documents with non-standard layouts and varied formats.

[0075] Based on the rating results, the system dynamically invokes differentiated processing flows through "policy routing":

[0076] L1 level routing to "high-speed template matching process" (using preset rules and dictionaries for high-speed processing).

[0077] L2 routing leads to the "standard multimodal processing flow" (invoking LLM and VLM for collaborative processing).

[0078] L3 level routing leads to the "agent adaptive processing flow" (driven by an agent and capable of online learning).

[0079] This module is the core of the resource optimization and efficient processing of this invention, ensuring that the system can process simple documents at high speed as well as handle complex documents accurately.

[0080] S004, Differentiated Processing Chains: Based on the hierarchical results, the system calls different processing chains. "For L1 level documents, a high-speed template matching process is called; for L2 level documents, a standard multimodal cognitive fusion process is called; for L3 level documents, an agent-driven adaptive processing process is called." For L1, the system directly uses preset regular expressions and dictionaries for field capture, with CPU time typically <100ms and a single node supporting 200 QPS. For L2, "The real-time collaborative decision-making unit has a built-in dynamic fusion model based on a gating mechanism, used to receive the semantic analysis results from the LLM module and the visual structure information from the VLM module, and dynamically generate fusion processing instructions based on preset 'visual-guided semantics' or 'semantic-guided vision' collaborative rules to achieve cross-modal information alignment, verification, and reasoning." In this embodiment, the gating mechanism uses Sigmoid weighted fusion: when visual confidence > 0.9 and text confidence < 0.7, the "visual-guided semantics" branch is enabled, prioritizing table coordinate positioning; otherwise, the "semantic-guided vision" branch is enabled, prioritizing text context positioning. The fused feature vectors are fed into Qwen3-14B for field extraction. A single inference GPU session takes approximately 400ms, with a GPU memory usage of <12GB. For L3, the system initiates an intelligent agent chain: the natural language compiler compiles user prompts into executable rule sequences in real time; the rule execution engine schedules LLM and VLM tools within a sandbox; and the online learning feedback loop records user corrections and updates the error correction memory in real time, ensuring adaptive capabilities even under extreme conditions.

[0081] S005, EasyEX format conversion; whether the upstream is an OCR result or a text layer extraction, it needs to be converted into an EasyEX intermediate format. "EasyEX processing scheme brief: 1. Multi-file type processing capability: text type PDF through layout analysis, extract text block coordinates and table structure, convert nested titles to Markdown levels according to font features (font size / indentation) and semantic keywords (such as 'Chapter X'), and rearrange disordered paragraphs through LLM semantics and add logical annotations; DOCX parses document styles (such as headings 1 / 2 / 3), automatically corrects style conflicts, nested tables are distinguished by border indentation, and merged cells are converted to mark rowspan / colspan; Excel fixes missing line boxes, identifies weak styles such as gray light boxes, generates tree structures based on row height / column width, preserves original range markers for merged cells, and generates parent-child relationships for nested tables; TXT based on text density and spacing to infer boundaries, combined with semantic anchor positioning column names, and complete logical borders when generating Markdown tables. 2. Complex table processing scheme: when the border is not clear, infer the boundary based on cell text density and spacing, combined with semantic anchors such as 'total' to locate column names, and complete the logical border when generating Markdown tables; cross-page tables are clustered and spliced based on Y-axis coordinates, with 'cross-page continuation table' markers, cell content is automatically continued and semantic coherence is verified; nested / merged cells identify indentation differences in inner tables, generate nested JSON structures, and mark cross-row and column ranges (such as A1:C3) when converting merged cells, preserving the original layout logic." In actual deployment, the system first runs PDFPlumber to obtain vector lines on PDF, then performs intersection operation with the text block coordinates output by VLM, and fills in the missing border; for Excel, first read the cell style through OpenPyXL, if the line color is less than 30% gray, it is considered as a "weak border", and the logical boundary is automatically expanded; for cross-page tables, the system clusters and splices them based on Y-axis coordinates, and writes a "cross-page continuation table" note in JSON, ensuring that the rows and columns are aligned during subsequent extraction. The final output of the EasyEX document is a mixed text of Markdown+HTML table+LaTeX formula, which not only retains human readability, but also facilitates regular or LLM extraction.

[0082] ​​S006, Prompt dynamic compilation and extraction; extraction agent receives user's field requirements described in natural language, and generates executable rule sequence in real time through prompt dynamic compilation technology. "Natural language compiler: the core is prompt dynamic compilation technology. This module will interpret the natural language instructions (prompt) input by the user through a large language model (LLM) interpreter, and compile it into a structured, multi-step logic, machine executable rule sequence in real time. Rule execution engine: used to parse and execute the above rule sequence in a secure sandbox environment, schedule the underlying LLM and VLM tools to locate, extract, judge, etc. on the target document. Online learning feedback loop: the core is an incremental learning mechanism based on error feedback. This module is used to capture the user's correction operations (such as modification, deletion, supplement) on the agent's execution results, and convert each correction behavior into a structured learning sample of'(instruction, error result, correct result) ', and store it in a dynamically updated 'error correction memory bank' for guiding the compiler to generate more accurate rules in subsequent tasks. In this embodiment, the user inputs "please extract Party A name, signing date, contract amount (case sensitive)", the natural language compiler compiles it into three rules: 1. Locate the paragraph containing "Party A" or "Party A" in the title; 2. Extract the first entity in the paragraph that meets the company name regular expression; 3. Locate the "amount" line and extract the number and Chinese capital amount. The rule execution engine calls Qwen3-14B in the sandbox to perform extraction, and the single time consumption is about 400ms; if the user corrects "ABC Limited Company" in the result to "ABC Company Limited", the online learning feedback loop immediately records the triple, and through LoRA, it performs incremental fine-tuning on the compiler during the night low peak period, so that the next time the contract of the same customer is extracted, "ABC Company Limited" is output preferentially. In addition, the extraction agent also supports complex task decomposition: split the multi-page document by chapter, then extract and merge the results; call the table recognition model to extract data from the table data first, and then understand the meaning by LLM; call VLM to recognize seal and handwritten signature in the scene that needs to combine visual information, and fuse the visual analysis result into the LLM reasoning process.

[0083] S007, Type verification and back pressure + REACT; Type verification module checks item by item according to pre-configured regular, numerical range and business rules. Back pressure processing mode: 1. Format hard error, level: high, such as ID number, email, date field does not conform to regular rules (such as date format is '2025-02-30'), immediately trigger single field re-drawing, do not set retry interval; 2. Numerical logic contradiction, level: medium, such as: the amount field exceeds the reasonable range of business (such as bond single transaction amount <100 yuan) or case insensitive, trigger single field re-drawing and record error; 3. Semantic ambiguity error, level: low, such as: field name has ambiguity (such as 'party A' is not clear) but the format is correct, trigger re-drawing after accumulating 3 times of the same error, manual intervention is required to supplement the context. At the same time, integrate REACT mechanism;

[0084] REACT template library example: 1. Date / time field: 'Please verify whether [field value] is consistent with the time sequence of [associated field such as 'effective date'] in the document, whether there is a cross-year / cross-month contradiction?' 2. Amount / value field: '[field value] is consistent with the calculation logic of [associated field such as'subtotal amount'] in the document (such as quantity * unit price = total)? Please check whether the number of decimal places meets the business specifications.' 3. Entity name field: '[field value, such as company name] is consistent with the expression in other positions (such as seal, seal) in the document? Is there a case of confusion between the abbreviation and the full name?' 4. Logical relationship field: '[field A value] and [field B value] are associated with business rules (such as 'transaction type' is'repurchase', 'term' is within a reasonable range)?'; In this embodiment, the system runs regular verification on the extracted "contract amount" field first, and if there are more than two decimal places, it is marked as a format hard error, which immediately triggers back pressure and re-sends the document segment corresponding to the field to the extraction engine; If the amount is case insensitive, it is marked as a numerical logic contradiction, which also triggers re-drawing and records the log; If there are multiple candidate entities for 'party A', the system automatically pops up a manual review box after accumulating three times of the same error, prompting the user to supplement the context.

[0085] S008, upstream and downstream verification; business knowledge graph engine and compliance semantic firewall joint operation. "Business knowledge graph engine: model and store the knowledge and rules of the business domain. The knowledge graph contains key entities in the domain (such as customers, products, projects, etc.), entity attributes, and relationships between entities, and also embeds business rules and constraints. Compliance semantic firewall: real-time connection with business knowledge graph engine, which has an extensible compliance rule base that converts legal provisions and risk control policies into machine-readable logic. The output includes a comprehensive verification report containing error details, risk levels, and recommended handling measures." In this embodiment, the knowledge graph uses Neo4j storage, nodes include "customer", "fund product", "counterparty", "contract", etc., and edges include "sign", "position", "guarantee", etc.; The compliance rule base is written in Drools format, for example, "The term of repurchase transaction shall not be > 365 days". When the system extracts "transaction type = repurchase" and "term = 400 days", the compliance semantic firewall triggers immediately, outputting the verification report "high risk: repurchase term exceeds regulatory upper limit, suggest returning to business personnel for verification".

[0086] S009, result output and example application; output the structured elements that pass all verification in JSON, XML, or Excel, or directly write to the downstream business system. Example 1: "Bank transaction sheet intelligent renaming application as a document management solution relying on artificial intelligence technology, mainly designed for the standardized management needs of financial institutions for a large number of bank transaction sheets. The application uses AI technology to automatically identify and extract key information from the transaction sheet, such as transaction date, counterparty, transaction amount, transaction type, and other core elements. The system first identifies the type of input document and confirms that it is a bank transaction sheet, then starts deep content analysis to accurately extract key information fields. Subsequently, the system standardizes the extracted information according to the pre-set naming rules, converts the format, generates a file name that meets the institutional specifications, and performs the renaming operation. The entire process not only reduces manual intervention, improves work efficiency, but also reduces the error rate, ensuring the standardization and consistency of file management. In the case of incomplete information extraction, the system will start the manual intervention mechanism to ensure the accuracy and integrity of the renaming results. After a simple comparison test, it takes about 30-40 seconds per file for a person to complete a transaction sheet renaming, while the system only takes 1 / 3 of the time. With 10 concurrent support at ordinary hardware level, the processing efficiency is greatly improved. In addition, you can set up timed tasks and rule tasks according to business needs to process documents as they come, fully leveraging automation advantages."

[0087] The specific implementation process of the application in the bank transaction sheet intelligent renaming application is as shown in Figure 2

[0088] ​This figure fully demonstrates the end-to-end business process of the system processing interbank trading orders and realizing intelligent renaming.

[0089] Process and principle detailed description: The whole process starts from the "file collection" stage. The system receives user uploaded documents through the unified file entrance described in step S001, supporting batch processing.

[0090] Subsequently, the process enters the "file type identification" stage. The system judges whether the input document is PDF, Word or image format by detecting the file header (Magic Byte) and extension, preparing for subsequent shunt processing.

[0091] The next step is a key step: "interbank trading order identification". This step goes far beyond simple format identification. Its core is to judge the business type and complexity of the document. The system needs to confirm that the current document being processed is an "interbank trading order", not a contract or an invoice. This is usually achieved by combining the semantic understanding of LLM and the layout analysis of VLM, such as identifying whether the keywords in the document such as "trading order", "counterparty", "bond code" appear, and whether it conforms to the typical layout features of this type of document. Only if it is identified as an "interbank trading order", will it trigger the subsequent special renaming process.

[0092] After confirming the type of the document, the system starts multiple "key information extraction" tasks in parallel, including:

[0093] 1. Transaction date identification: extract the core transaction date field.

[0094] 2. Counterparty identification: accurately identify the counterparty name that transacts with the party.

[0095] 3. Transaction amount identification: extract the transaction amount value.

[0096] 4. Transaction type identification: identify and classify transaction types such as spot buying and selling, repurchase, and lending.

[0097] 5. Other key element extraction: extract other important information in addition to the above fields.

[0098] 6. AI document content analysis: This is a core step, which refers to calling an artificial intelligence model to perform deep semantic analysis on the entire document. Its purpose is not only OCR, but also to understand the overall content, logical structure and context relationship of the document to ensure the completeness and accuracy of the extraction of the above key information and to solve ambiguities (for example, distinguishing between "counterparty" and "settlement counterparty"). This directly reflects the powerful context understanding and reasoning ability of LLM.

[0099] After all information extraction is completed, the system will make a core logical judgment: "Is the information extraction complete?" This node is used to check whether the extracted key information (date, counterparty, amount, type, etc.) is complete, accurate and of qualified quality to meet the customization requirements of subsequent renaming rules. If the judgment is "no" (i.e. the information is incomplete or of poor quality), the process ends or triggers the back pressure mechanism, and the error field is sent back to the extraction engine for iterative extraction. If the judgment is "yes", the next stage is entered.

[0100] Next, the extracted raw information is standardized and formatted:

[0101] Date formatting: Convert the recognized date to a standard format such as "YYYYMMDD".

[0102] Transaction type code mapping: Map the recognized transaction type text to the standard business code agreed upon by the institution.

[0103] Counterparty data application: Standardize the counterparty name, such as using the internal system-recognized abbreviation or code.

[0104] Amount representation formatting: Uniformly format the amount value.

[0105] Record serial number: Generate a unique serial number for batch file processing to avoid duplication and ensure orderly file management.

[0106] The formatted data is "applied to the naming rule". The system generates a standardized file name based on the naming rule preset by the user in Example 1 (e.g. "{transaction date}_{counterparty code}_{transaction type code}_{amount}").

[0107] Then, the system "performs file renaming" operation, and then "verifies the renaming result". The verification includes whether the renaming is successful, whether the new file name is compliant, etc. If the verification is "passed" ( "verification passed" in the figure), the metadata record in the "file management system" is updated to indicate that the task is "completed".

[0108] This complete process clearly demonstrates the specific implementation of the "output result can be directly used for interbank transaction sheet intelligent renaming", covering the full-link automated operation from file input to final file system update. The two features "AI document content analysis" and "is the information extraction complete?" in the figure highlight the intelligence and robustness of the invention compared to traditional template technology.

[0109] Embodiment 2: "Kaiji Bill Factor Extraction and System Data Intelligent Verification Application is a set of intelligent data processing system combining Large Language Model (LLM) and Visual Language Model (VLM). The system accurately parses the complex table structure including merged cells and nested tables for open-end fund bills and other complex table documents through powerful structure recognition capability", in this embodiment, the system automatically pulls fund company email attachments every morning, batch parses and checks the extraction results with the valuation system data, generates a difference report for accountants to confirm, and processes 30,000 pages of reports per day. The average single reconciliation time is reduced from 15 minutes to 30 seconds.

[0110] Attached Figure 3 This embodiment describes in detail the automatic processing and verification process of the open-end fund bill, a highly complex and non-standardized document. The entire process deeply integrates the layout understanding capability of the visual language model (VLM), the semantic reasoning capability of the large language model (LLM), and the verification mechanism based on business knowledge.

[0111] The process begins with "Kaiji Bill Document Input". The system receives bill files (PDF, images, etc.) from hundreds of different fund companies with various formats.

[0112] The next step is the "document preprocessing" stage. This stage is not just a simple format conversion, but also a multi-modal analysis process. The system calls VLM (such as LayoutLM) for detailed layout analysis to identify text blocks, table areas, seals, and other visual elements in the document, and outputs structured data with coordinates and type labels. For scanned documents, OCR text recognition is performed first.

[0113] The preprocessed data enters the "document structure recognition and analysis" core link. This is the key to processing fund bills. The system needs to deeply analyze the complex structure:

[0114] Identify merged cells: accurately determine the row span and column span of the cells to avoid misplacement during subsequent data extraction.

[0115] Splice cross-page tables: through Y-axis coordinate clustering and table header consistency matching, the scattered table fragments in multiple pages are logically reconstructed into a complete table.

[0116] Parse nested structure: handle possible parent-child table relationships and clarify data hierarchy.

[0117] Semantic Role Labeling: Identify and label the semantic roles of key columns such as "fund account number", "stock code", "stock name", "shares", "net value", and "market value". This step outputs a highly structured and machine-readable document representation.

[0118] On top of the clear document structure, the "Core Information Extraction" process is initiated. This task is completed by the "Agent Extraction" module. The extraction agent generates execution instructions using Prompt dynamic compilation technology based on user-predefined requirements (such as "extract all holdings details") and schedules LLM to accurately locate and extract the values of target fields in the parsed structured document.

[0119] The extraction results are immediately taken over by the "Agent Verification" module for strict "data quality checks". The verification includes:

[0120] Format verification: Check if the fund account number is 12 digits, the stock code conforms to the exchange coding rules, etc.

[0121] Logical verification: Use the REACT mechanism to require LLM to verify that the calculation logic such as "shares x net value = market value" is self-consistent.

[0122] If the check is "unqualified", the system will "mark the exception" and trigger the back pressure mechanism to send the problem data back to the extraction link or prompt manual intervention. If the check is "qualified", it flows to the final link.

[0123] "Data Intelligent Check". Instead of processing documents in isolation, the system automatically compares the extracted high-quality data (such as the holdings list) with downstream business databases such as the "valuation system". This process embodies the idea of the "business knowledge graph engine", which checks consistency by querying entity association relationships.

[0124] If the comparison result is "inconsistent", a "difference record" is generated and "confirmation data" may be required to determine whether the extraction is incorrect or there is a system accounting problem.

[0125] Finally, the system generates a clear "check result report" and "completes" the entire process.

[0126] The core principle of this process is to convert unstructured complex documents into machine-readable structured data through the cooperation of VLM and LLM, and then through the three-level pipeline of extraction, verification, and check, to finally achieve seamless integration with business systems and value mining. It not only solves the problem of "understanding", but also solves the business problems of "accurate use" and "fast checking", reducing the average single account reconciliation time from 15 minutes manually to 30 seconds systemically, fully embodying the practicality and advancement of the invention.

[0127] As Figure 1 shown, the overall processing flow of the method of the present application is as follows: the figure depicts the overall processing pipeline of the core method of the present application. The flow starts with "Start" and receives "Input Document". First, the "Document Format Identification" and "Invoice Type Identification" modules parse the input document to determine its format (PDF / Word / Picture, etc.) and business document type (such as invoice, transaction sheet, etc.). This process is completed by the "Document Conversion Module", if it is a scanned copy or picture, VLM is called for OCR and layout analysis; if it is PDF / Word, the text and table structure are directly extracted, and a unified intermediate representation is output.

[0128] Subsequently, the flow enters the "EasyEX Format Conversion" link. This module converts the parsing results from the upstream into an intermediate format based on Markdown, with additional HTML tables and LaTeX formulas, forming an "output json" structure based on "markdown+table+latex". This step is performed by the "Preprocessing Module", which aims to generate a unified document representation that is friendly to programs and preserves semantic structure.

[0129] After format conversion, the "Extraction Agent" begins to work. It receives user requirements and uses Prompt dynamic compilation technology to schedule "LLM to perform extraction tasks" to locate and extract target fields and values from EasyEX documents.

[0130] The extraction results then enter the verification stage. First, "type verification" is performed, i.e., according to pre-configured rules (such as regular expressions, numerical ranges) to check the field format. At the same time, the system integrates the "REACT module" and starts the "LLM self-check mode", i.e., requires LLM to generate self-check questions based on the extraction results and answer them by referring to the original text, to verify the consistency of the content.

[0131] The decision point "Verification Passed?" determines the result. If not, feedback to the front end or trigger re-extraction; if yes, hand over to the "Verification Agent" to perform deeper "format type verification + logical verification", i.e., cross-field consistency verification through business knowledge graph, and compliance verification by compliance semantic firewall.

[0132] Finally, after the "Final Verification Passed?" decision point is "Yes", the system "outputs the results" and "ends" the flow, outputting or writing structured data in JSON, XML, or Excel format to the downstream system.

[0133] The design of the present application focuses on constructing an intelligent processing framework with multi-modal collaboration, dynamic self-adaptation and continuous learning ability. The core is that first, through a dynamic grading system based on a lightweight decision tree model, the format, structure and variability of the input document are evaluated in three dimensions, and it is divided into L1 (simple and regular type), L2 (complex structure type) or L3 (highly variable type) three complexity levels in real time, and then routed to the differentiated processing flow (high-speed template matching, multi-modal cognitive fusion or agent-driven processing) accordingly, so as to realize the optimal allocation of system computing resources and the precise adaptation of processing strategy. Secondly, a unified intermediate representation format named EasyEX is designed, which is based on Markdown, combines HTML tables and LaTeX formulas, and converts documents of different sources into structured documents that are both human-readable and program-friendly, laying a foundation for subsequent high-precision extraction. Finally, an agent (Agent) driven architecture is innovatively adopted, the core of which is the "Prompt dynamic compilation technology" and the "online learning feedback loop", so that users can flexibly configure extraction requirements through natural language instructions, the system can compile them into executable rules in real time, and can continuously learn and evolve from the user's correction behavior, finally combined with the business knowledge graph and the compliance semantic firewall, realize the all-round check from the field level to the business logic level, and ensure the high reliability of the output results.

[0134] The above is only a preferred embodiment of the present application, and does not limit the technical scope of the present application in any way. Therefore, any minor modification, equivalent change and modification made according to the technical essence of the present application to the above embodiment are still within the scope of the technical solution of the present application.

Claims

1.A document element extraction method based on AI large model technology, characterized by, The method comprises the following steps: S001, receiving a document input by a user; S002, identifying the physical format of the document through a type conversion module, and generating a structured intermediate representation after format identification and layout analysis; S003, sending the intermediate representation to a dynamic grading module, which calls a format analyzer, a structure parser, and a variability detector in parallel to obtain three-dimensional features of the document, including format complexity, structure complexity, and variability, and assesses the level in real time through a pre-trained lightweight decision tree model, such as L1 simple and regular type, L2 structure complex type, or L3 high variability type; S004, calling a corresponding differential processing procedure according to the complexity level obtained in step S003; for structure complex type or high variability type documents, the processing procedure fuses the layout analysis results of VLM and the semantic analysis results of LLM through a collaborative decision mechanism to generate processing instructions; S005, converting the content processed in step S004 into EasyEX format, which is based on Markdown, with additional HTML tables and LaTeX tags, forming a unified intermediate document; S006, an extraction agent in the extraction engine receives a field requirement described in natural language by a user, and uses Prompt dynamic compilation technology to compile the field requirement described in natural language into an executable rule sequence in real time, and locates and extracts corresponding fields and values in the EasyEX format document by LLM; The Prompt dynamic compilation technology comprises: A natural language compiler compiles user input natural language instructions into a machine executable rule sequence containing multi-step logic in real time through an LLM interpreter; A rule execution engine parses and executes the rule sequence in a sandbox environment, schedules LLM and VLM tools to locate, extract, and judge target documents; An online learning feedback loop captures user correction operations on extraction results, converts each correction behavior into a "instruction-error result-correct result" structured sample, and stores it in a dynamically updated correction memory library for guiding the compiler to generate more accurate rules in subsequent tasks; S007, a type checking module checks the extraction results obtained in step S006 according to preconfigured regular expressions, value ranges, and business rules for format and logic verification, and triggers a back pressure mechanism if errors are found to re-enter the document segment corresponding to the error field into the extraction engine for iterative extraction; S008, an upstream and downstream verification module queries the associated path and attribute constraints of the extracted elements in the knowledge graph through a business knowledge graph engine, performs cross-field consistency verification, and outputs a comprehensive verification report containing error details, risk levels, and recommended treatment measures through a compliance semantic firewall for deep semantic matching and reasoning with the built-in compliance rule library; S009, outputting the structured element results that pass all verifications in JSON, XML, or Excel format, or directly writing them into the downstream business system. 2.The AI large model technology-based document element extraction method of claim 1, wherein: In step S002, layout analysis further comprises: Identify the bounding boxes and type labels of text blocks, tables, charts, and pictures page by page; Detect cross-page or columnar tables through visual topology analysis, and merge them based on table header consistency and text coherence; Predict the reading priority between elements based on the semantic understanding of document layout by VLM, and generate the logical order in the context of columnar and nested tables; On the basis of physical structure, identify the title level, paragraph ownership, and table semantic role through LayoutLM, and output structured JSON or XML. 3.The AI large model technology-based document element extraction method of claim 1, wherein: In step S003, the format analyzer is used to detect file metadata, including file type, resolution, and text selectivity; The structure parser is used to quantify element density, table nesting level, text-to-picture ratio, and formula quantity; The variability detector is used to compare the current document with the built-in standard version library, calculate the layout hash distance and key element semantic drift, and output the non-standard variability score; The lightweight decision tree model integrates the above three dimensional features and outputs L1 / L2 / L3 levels in real time. 4.The AI large model technology-based document element extraction method of claim 1, wherein: In step S004, the real-time collaborative decision unit is used to realize the collaborative decision mechanism. The unit is built-in with a gating mechanism, which is used to receive the semantic analysis results of LLM and the visual structure information of VLM, and based on the preset "visual guided semantics" or "semantic guided vision" collaborative rules, dynamically generates fusion processing instructions to realize cross-modal information alignment, verification and reasoning. 5.The AI large model technology-based document element extraction method of claim 1, wherein: In step S007, the back pressure mechanism includes: Immediately trigger single field re-drawing for format hard errors; Trigger re-drawing and record errors for numerical logic contradictions; Trigger re-drawing and prompt manual intervention after accumulating three times of semantic ambiguity errors; The type verification module also integrates the REACT mechanism, which makes LLM generate self-check questions and answers based on the extraction results to verify the consistency of the extraction content and the context of the original document. 6.The AI large model technology-based document element extraction method of claim 1, wherein: In step S008, the business knowledge graph engine stores entities, attributes and relationships between entities in the field of single documents, and embeds business rules; The compliance semantic firewall has a built-in extensible compliance rule library, which converts legal provisions and risk control policies into machine-readable logic to perform compliance verification on the extraction results; The comprehensive verification report includes error details, risk levels and recommended treatment measures. 7.The AI-large model technology-based document element extraction method of claim 1, wherein: The EasyEX format conversion step further includes: For text PDF, extract text block coordinates and table structure through layout analysis, and convert titles to Markdown levels according to font size and indentation; For DOCX, parse document styles and correct style conflicts, and generate tree structure for multi-level table headers based on border and indentation; For Excel, repair missing line boxes, identify gray light style, and generate tree structure for multi-level table headers; For TXT, infer boundaries based on text density and spacing, locate column names based on semantic anchors, and complete logical borders when generating Markdown tables; For cross-page tables, cluster and splice them according to Y-axis coordinates and retain cross-page continuation table markers. Mark the cross-row and column range of merged cells. 8.The AI large model technology-based document element extraction method of claim 1, wherein: The extraction agent supports complex task decomposition: Split multi-page documents by chapters, extract them separately, and then merge the results; First call the table recognition model to extract data, and then use LLM to understand the meaning of the table data; For scenarios that require visual information, VLM is called to recognize seals and handwritten signatures and integrate visual analysis results into LLM inference process. 9.The AI-large model technology-based document element extraction method of claim 1, wherein: In step S009, the output result is directly used for: inter-bank transaction sheet intelligent renaming, the system extracts the transaction date, counterparty, amount, and transaction type to generate a standardized file name according to a preset rule; fund reconciliation statement element extraction and system data intelligent checking, the system automatically checks the extracted position table field with the valuation system data and generates a difference report; the method supports concurrent processing under ordinary hardware.

Images