A method and system for extracting information from templated documents based on multi-modal data extraction

Abstract

The application discloses a kind of based on multi-modal data extraction's templated document information extraction method and system, it is related to information processing and intelligent identification technical field, document attachment is converted into the standard document of pre-set format;According to document structure, standard document is partitioned and identified, the text content in each text area is fragmented and text is identified, the text fragment obtained is vectorized coding and is stored in vector database;Then according to the field requirement description of extraction template, retrieve the corresponding text fragment from vector database as candidate fragment;Several candidate fragments are input to large language model by using reordering model screening to extract key field value;The key field value extracted is filled in corresponding position according to the field mapping relationship defined in output file template, and structured extraction file is generated. Through structure perception fragment and reordering fragment screening, the accuracy and processing efficiency of field extraction are improved.

Description

Technical Field

[0001] This invention relates to the field of information processing and intelligent recognition technology, specifically to a method and system for extracting information from templated documents based on multimodal data extraction. Background Technology

[0002] In document processing in corporate finance, legal, and other fields, a large amount of critical information is contained in long, unstructured documents such as contracts, reports, and tender documents. These documents are lengthy and complex in structure, and in actual business operations, only a few key fields often need to be extracted.

[0003] Existing methods primarily follow the technical approach of "OCR recognition + full-text traversal + rule matching". First, each page of the document is converted into an image for OCR recognition to obtain the full text. Then, the entire text is sequentially matched based on keywords or regular expression rules in a preset template to locate the target field. This approach has significant bottlenecks in long document scenarios: First, the full text of long documents often exceeds the context length limit of large language models, making direct input processing impossible; second, key fields usually only appear in specific paragraphs, and a large amount of irrelevant content in the full text constitutes noise interference, reducing matching efficiency and easily leading to misjudgments due to multiple instances of the same keyword; third, long documents generally have complex structures such as nested multi-level headings, cross-page tables, and repeated information in headers and footers, making it difficult for traditional methods to accurately distinguish the ownership of fields with the same name, splice cross-page tables, and eliminate interference from repeated information; fourth, abnormal page orientation may occur during document scanning and conversion, and the tilted features of diagonal watermarks can interfere with orientation detection, leading to misjudgments of rotation.

[0004] The aforementioned shortcomings result in insufficient extraction accuracy and low efficiency when processing long documents, making it difficult to meet the requirements of high accuracy and high efficiency for industrial applications. Summary of the Invention

[0005] The purpose of this invention is to provide a method and system for extracting templated document information based on multimodal data extraction. By using structure-aware fragmentation and reordering fragmentation filtering, the accuracy and processing efficiency of field extraction are improved.

[0006] To achieve the above objectives, the following plan is adopted: On the one hand, this invention provides a method for extracting information from templated documents based on multimodal data extraction, specifically including the following steps: S1. Obtain the document attachments uploaded by the user, perform orientation anomaly detection and rotation correction on the document attachments, and convert them into standard documents in a preset format; S2. Responding to the extraction template selected by the user, the standard document is partitioned and identified according to the document structure to obtain several text regions. The text content in each text region is segmented and identified to obtain text fragments. S3. Vectorize and encode the segmented text fragments and store them in the vector database; based on the field requirement description of the extracted template, retrieve the text fragments corresponding to the field requirement description from the vector database as candidate fragments; S4. Use a re-ranking model to score and rank the candidate segments based on their relevance. Input the top-ranked candidate segments as effective information context into the large language model, and the large language model will perform the location and extraction of key field values. S5. Fill the extracted key field values ​​into the corresponding positions according to the field mapping relationship defined in the output file template to generate a structured extraction file.

[0007] In some specific implementations, the specific process of anomaly detection and rotation correction is as follows: Render each page of the document attachment as a preview image, input it sequentially into the trained orientation detection OCR model, and output the classification result of the page rotation angle; When the classification result is page tilt, the page is judged to be abnormal and its page number and rotation angle are recorded, and a page-level orientation abnormality mapping table is established. According to the orientation abnormality mapping table, each abnormal page is rotated in the opposite direction at the corresponding angle so that all pages are unified to the normal reading direction.

[0008] In some specific implementations, the orientation detection OCR model includes a spatial attention module. The spatial attention module distinguishes between the main text region and the specific label region based on the density and continuity of the text, and assigns a higher attention weight to the main text region than to the specific label region. Based on the main text region, it determines the normal reading direction of the text and the classification result of the page tilt angle.

[0009] In some specific implementations, when the classification result is that the page is not tilted, the feature vector of the main text area is extracted, and its cosine similarity with the standard forward text distribution reference vector is calculated. When the cosine similarity is lower than a preset threshold, the page is marked as a page with questionable orientation and a secondary verification is triggered. The secondary verification involves calling the text connectivity analysis algorithm to detect the baseline tilt angle of the text lines within the page. If the detected baseline tilt angle exceeds the preset tolerance range, a fine-grained rotation correction is performed.

[0010] In some specific implementations, the specific process of obtaining text fragments is as follows: The standard document is partitioned and identified based on the density and continuity of the text to obtain several text regions; Each text region is segmented according to the document structure, and each structural unit is treated as a text block. The size of each text block is checked to see if it exceeds the preset range. If it does, the text block is recursively split and then merged. Metadata information is preserved during the recursive splitting.

[0011] In some specific implementations, the recursive decomposition process is as follows: Construct block-level metadata for each text block. The block-level metadata includes the page number to which the text block belongs, the parent title, a list of candidate field key names contained in the text block, whether it is a table block, the table row and column range, and cross-page markers. For each text block, an initial token count is performed. If the initial token count exceeds a preset limit, the text block will be split. If the initial number of tokens in a text block is lower than a preset limit, then the adjacent text blocks of that text block are obtained and merged based on the consistency of the parent title and semantic continuity in the block-level metadata.

[0012] In some specific implementations, text blocks are divided into paragraph blocks and table blocks according to the document structure. The splitting of text blocks includes: Perform semantically preserving splitting operations based on sentence boundaries on paragraph blocks; For table blocks, if the table in the table block is a cross-page table, it will be split naturally along the page boundary; otherwise, the table will be kept intact and only the header row and data row will be split. After splitting, the child blocks will inherit the block-level metadata of the parent block and be marked with the split sequence number.

[0013] In some specific implementations, the process of obtaining candidate fragments includes: Generate query text based on the field requirements description of the extracted template, encode the query text into query vector through the embedding model, and encode the fragmented text fragments into fragment vectors. Perform a hybrid search in the vector database. The hybrid search performs two search paths simultaneously: the first is vector similarity search, which calculates the cosine similarity between the query vector and each piece vector, and returns the top K1 pieces with the highest similarity. The second method is full-text search, which constructs full-text search conditions based on keywords in the query text, matches the text content in each segment vector, and returns the top K2 segments with the highest matching degree. The results from the two retrieval paths are merged and deduplicated. The inverse ranking fusion algorithm is used to weight and merge the rankings of the same fragment vector in different retrieval paths to generate a unified candidate fragment list.

[0014] In some specific implementations, the process of using a re-ranking model to score and rank candidate slices based on relevance is as follows: The field requirement description is paired with each candidate slice to form a text pair, which is then input into a reordering model based on a cross-encoder architecture. The reordering model performs joint encoding on the text pairs and outputs a relevance score through a fully connected layer, representing the degree of semantic matching between the slice content and the field requirement. All candidate segments are sorted in descending order based on their relevance scores, and a segment sequence is formed by retaining several candidate segments with scores higher than a preset threshold and ranking higher as the fine screening result. For candidate segments in the screening results, read the parent title information in their block-level metadata. If the parent title block corresponding to the candidate segment is not selected in the screening results, the parent title block is added to the segment sequence in read-only context form to ensure that the large language model obtains a complete structural context.

[0015] Secondly, this application provides a templated document information extraction system based on multimodal data extraction, comprising: The preprocessing module is used to obtain the document attachments uploaded by the user, perform orientation anomaly detection and rotation correction on the document attachments, and convert them into standard documents in a preset format. The information extraction module includes: The text segmentation unit is used to respond to the extraction template selected by the user, divide the standard document into several text regions according to the document structure, and perform segmentation and text recognition on the text content in each text region to obtain text fragments. The text filtering unit is used to vectorize and encode the segmented text fragments and store them in the vector database; then, based on the field requirement description of the extraction template, it retrieves the text fragments corresponding to the field requirement description from the vector database as candidate fragments. The location and extraction unit is used to score and rank the candidate segments using a re-ranking model, and input the top-ranked candidate segments as effective information context into the large language model, which then performs the location and extraction of key field values. The output extraction module is used to fill the extracted key field values ​​into the corresponding positions according to the field mapping relationship defined in the output file template, and generate a structured extraction file.

[0016] The beneficial effects of this invention are as follows: This application reconstructs the long document information extraction process into a retrieval enhancement generation paradigm of "structure-aware fragmentation - vectorized storage - on-demand retrieval - re-sorting and filtering - precise extraction from large models". This transforms long document processing from "full-volume traversal matching" to "on-demand retrieval and focusing", achieving precise extraction of key fields from documents of any length within a limited large model context window. At the same time, in the document preprocessing stage, the robustness of document correction is significantly improved through the collaborative design of a lightweight direction detection model and an anti-watermark interference mechanism.

[0017] This application balances semantic integrity and length control through a structure-aware fragmentation and token-aware recursive splitting mechanism. It segments the document based on inherent structural units such as heading levels, chapters, paragraphs, and table boundaries. Large blocks exceeding the token limit are further recursively split according to sentence or table boundaries. Adjacent blocks that are too short are merged. Simultaneously, each fragment is associated with metadata such as its page number, parent heading, table row and column information, and cross-page markers. This ensures that each fragment maintains semantic integrity without fragmentation while remaining within the length that a large model can process. The metadata association provides a structural foundation for subsequent accurate localization and contextual reconstruction.

[0018] Furthermore, a hybrid retrieval mechanism is employed to combine the advantages of vector semantic recall and keyword precision matching. Vector similarity retrieval can recall semantically related segments that may not contain precise keywords, while full-text retrieval can precisely match segments containing specific field names. A weighted merging algorithm using inverse ranking is used to combine the results of the two paths, significantly improving the recall and relevance of candidate segments compared to a single retrieval method, and reducing the omission of effective segments due to differences in keyword expression or semantic ambiguity. Attached Figure Description

[0019] Figure 1 A flowchart illustrating the intelligent extraction method for multimodal image information provided in this embodiment of the invention. Detailed Implementation

[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit the present invention or its application or use. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] Unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps described in these embodiments do not limit the scope of the invention.

[0022] Furthermore, for clarity and brevity, descriptions of well-known functions and methods may have been omitted. Those skilled in the art will recognize that various changes and modifications can be made to the examples described herein without departing from the spirit and scope of this disclosure.

[0023] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and equipment should be considered part of the specification.

[0024] In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values.

[0025] Example 1 like Figure 1 As shown in the figure, this embodiment 1 provides a method for extracting information from templated documents based on multimodal data extraction, which specifically includes the following steps: S1. Obtain the document attachments uploaded by the user, perform orientation anomaly detection and rotation correction on the document attachments, and convert them into standard documents in a preset format; The specific process of anomaly detection and rotation correction is as follows: S11. Render each page of the document attachment as a preview image, input it into the trained orientation detection OCR model in sequence, and output the classification result of the page rotation angle; Specifically, the orientation detection OCR model includes a spatial attention module, which distinguishes between the main text region and the specific label region based on the density and continuity of the text, and assigns a higher attention weight to the main text region than to the specific label region. Based on the main text region, it determines the normal reading direction of the text and the classification result of the page tilt angle.

[0026] For example, specific identifiers include watermarks, seals, etc. Generally, the text in these identifiers is quite different from the text content and is easy to distinguish. However, the text within them, such as watermarks, may be large and printed at an angle, filling the entire document page, which can interfere with the judgment of the normal direction of the document text. Therefore, during the training phase of the direction detection OCR model, document samples containing identifiers such as watermarks can be used. Through the collaborative design of data construction, model architecture, and training strategies, the model can learn to distinguish between two modes: "the entire page of text is tilted" and "the text is normal and the watermark is tilted". For the latter, the direction of the text lines is used as the judgment criterion. For example, at the training data construction level, contrast samples with "positive documents overlaid with slanted watermarks" and labeled as positive are injected into the training set, so that the model learns to use the direction of the text lines as the reference and ignore the direction of the slanted watermark during training. At the model architecture level, a spatial attention module is introduced after the feature extraction layer. The attention weight of each spatial location is learned through 1×1 convolution. Based on the visual feature differences between the text region and the watermark region in terms of text density, distribution uniformity and contrast, the module automatically outputs high weight for the text region and low weight for the watermark region. At the training strategy level, a joint loss function is used for training, which includes direction classification cross-entropy loss and text region attention guidance loss. The text region attention guidance loss extracts the main direction features of the text region through a pre-trained text line direction detector, calculates the direction consistency between the main direction features and the regional direction features in the spatial attention weighted feature map, and guides the model to gradually reduce the attention weight of the watermark region during backpropagation.

[0027] The orientation detection OCR model can use a lightweight convolutional neural network model with fewer than 10M parameters, and employs depthwise separable convolution and channel pruning for joint compression. Under GPU acceleration, the inference time per page is less than 3 milliseconds.

[0028] S12. When the classification result is page tilt, determine the page is abnormal and record its page number and rotation angle, and establish a page-level orientation abnormality mapping table; according to the orientation abnormality mapping table, perform the corresponding angle reverse rotation on each abnormal page to unify all pages to the normal reading direction.

[0029] S13. When the classification result is that the page is not tilted, extract the feature vector of the main text area and calculate its cosine similarity with the standard forward text distribution reference vector. When the cosine similarity is lower than the preset threshold, mark the page as a page with questionable orientation and trigger a secondary verification. The secondary verification is to call the text connectivity analysis algorithm to detect the baseline tilt angle of the text lines in the page. If the baseline tilt angle is detected to exceed the preset tolerance range, then perform a fine-grained rotation correction.

[0030] To prevent the lightweight model from giving incorrect orientation classifications due to watermark interference (such as misjudging a normal forward-facing page as needing rotation), a secondary verification mechanism is set up. When entering the secondary verification stage, the text connectivity analysis algorithm will detect the actual tilt angle of the text line baseline—the baseline of the main text is still horizontal (the tilt angle is within the tolerance range), thereby correcting the model's misjudgment and avoiding unnecessary rotation operations.

[0031] The default format for extracting templates is PDF. Therefore, the original format type of the uploaded document is parsed. For PDF documents, it is determined whether the internal pages are scanned image pages: if they are text-based PDF pages, high-fidelity images are generated using vector rendering; if they are scanned PDF pages, the embedded bitmap images are directly extracted. For Word documents, they are first converted to PDF intermediate format and then converted to images. Resolution adaptive adjustment is performed on all converted images, and the optimal resolution is calculated based on the input Token budget required by the downstream visual language model.

[0032] S2. Responding to the extraction template selected by the user, the standard document is partitioned and identified according to the document structure to obtain several text regions. The text content in each text region is segmented and identified to obtain text fragments. The specific process of obtaining a text fragment: S21. Based on the density and continuity of the text, the standard document is partitioned and identified to obtain several text regions; S22. Divide each text region into segments according to the document structure, treat each structural unit as a text block, and check whether the size of each text block exceeds the preset range. If so, recursively split and merge the text blocks, retaining metadata information during recursive splitting.

[0033] The document structure is parsed after the text content is recognized and sorted by OCR. The document structure includes heading levels, paragraph boundaries, table areas, list structures, and headers and footers. Based on the recognized structural units, the document is divided into several initial text blocks, each corresponding to a complete structural unit. The size of each text block is generally limited to a maximum of 8k, which can be determined according to the size supported by the vectorized storage model. If the block size limit is exceeded, the large block will be split into several smaller blocks. Each smaller block retains some metadata information, such as the original page number, parent heading, and table header (e.g., split tables).

[0034] The specific process of recursive decomposition is as follows: S221. Construct block-level metadata for each text block. The block-level metadata includes the page number to which the text block belongs, the parent title, a candidate list of field key names contained in the text block, whether it is a table block, the table row and column range, and cross-page markers. S222. Perform an initial token count for each text block. If the initial token count exceeds the preset limit, the text block will be split. The text block is divided into paragraph blocks and table blocks according to the document structure. The splitting of the text block includes: performing a semantically preserved splitting operation based on sentence boundaries for paragraph blocks; for table blocks, if the table in the table block is a cross-page table, it will be split naturally along the page boundary. Otherwise, the table will be kept intact and only the title row and data row will be split. After splitting, the child blocks will inherit the block-level metadata of the parent block and be marked with the splitting sequence number.

[0035] S223. If the initial number of tokens of a text block is lower than a preset lower limit, then obtain the adjacent text blocks of the text block and merge them based on the consistency of the parent title and semantic continuity in the block-level metadata.

[0036] S3. Vectorize and encode the segmented text fragments and store them in the vector database; then, based on the field requirement description of the extracted template, retrieve the text fragments corresponding to the field requirement description from the vector database as candidate fragments. The specific process of obtaining candidate slices includes: S31. Generate query text based on the field requirements description of the extracted template, encode the query text into a query vector through the embedding model, and encode the fragmented text segments into fragment vectors. S32. Perform a hybrid search in the vector database. The hybrid search performs two search paths simultaneously: the first is vector similarity search, which calculates the cosine similarity between the query vector and each piece vector, and returns the top K1 pieces with the highest similarity. The second method is full-text search, which constructs full-text search conditions based on keywords in the query text, matches the text content in each segment vector, and returns the top K2 segments with the highest matching degree. S33. Merge and deduplicate the results from the two retrieval paths, and use the inverse ranking fusion algorithm to weight and merge the rankings of the same fragment vector in different retrieval paths to generate a unified candidate fragment list.

[0037] S4. Use a reranking model (such as the reranker model) to score and rank the candidate segments based on their relevance. Then, use the top-ranked candidate segments as effective information context to input into the large language model, which will then locate and extract the key field values. The specific process of using the re-ranking model to score and rank the candidate slices based on their relevance is as follows: S41. Combine the field requirement description with each candidate segment to form a text pair, input it into a reordering model based on a cross-encoder architecture, the reordering model performs joint encoding on the text pairs, and outputs a relevance score through a fully connected layer, which represents the degree of semantic matching between the segment content and the field requirement. S42. Based on the correlation score, all candidate segments are sorted in descending order, and a number of candidate segments with scores higher than the preset threshold and rankings are retained to form a segment sequence as the fine screening result. S43. For candidate segments in the screening results, read the parent title information in their block-level metadata. If the parent title block corresponding to the candidate segment is not selected in the screening results, the parent title block is added to the segment sequence in the form of read-only context to ensure that the large language model obtains a complete structural context.

[0038] When using a large language model to locate and extract key field values, effective information segments are concatenated according to the document reading order to form contextual prompt text. The contextual prompt text is then merged with the field extraction prompt word template defined in the extraction template to construct a complete extraction instruction. The extraction instruction includes the field name, field type, extraction range description, and output format requirements for each field to be extracted. The large language model is then invoked to perform semantic understanding and reasoning based on the extraction instruction, locating the segment and specific position of the target field from the contextual prompt text and extracting the corresponding field value. Constraint decoding verification is performed on the extraction results: if the extracted field value does not conform to the field type or format defined in the template, a secondary extraction is automatically triggered. During the secondary extraction, the attention weight of the segment where the target field is located is increased, causing the model to focus on that segment area for re-extraction.

[0039] S5. Fill the extracted key field values ​​into the corresponding positions according to the field mapping relationship defined in the output file template to generate a structured extraction file.

[0040] As can be seen, the processing flow of steps S2-S5 above constitutes a retrieval enhancement generation process with closed-loop feedback. The document to be extracted undergoes structure-aware segmentation after OCR recognition → segment vectorization encoding and metadata association storage → adaptive hybrid retrieval recall of candidate segments after receiving field requirements → effective information segments are filtered and structural context is completed after re-sorting → the large language model extracts fields based on the complete context and template instructions, and undergoes constraint decoding verification, forming a complete retrieval enhancement generation closed loop. When the verification finds that the extraction result does not meet the template constraints, the feedback loop is automatically triggered, adjusting the retrieval parameters or segment weights and re-executing the retrieval and extraction. This achieves adaptive optimization from retrieval to generation, reducing reliance on manual review and improving the system's intelligence level and industrial applicability.

[0041] Example 2 This embodiment provides a templated document information extraction system based on multimodal data extraction, implementing the method of Embodiment 1, including: The preprocessing module is used to obtain the document attachments uploaded by the user, perform orientation anomaly detection and rotation correction on the document attachments, and convert them into standard documents in a preset format. The information extraction module includes: The text segmentation unit is used to respond to the extraction template selected by the user, divide the standard document into several text regions according to the document structure, and perform segmentation and text recognition on the text content in each text region to obtain text fragments. The text filtering unit is used to vectorize and encode the segmented text fragments and store them in the vector database; then, based on the field requirement description of the extraction template, it retrieves the text fragments corresponding to the field requirement description from the vector database as candidate fragments. The location and extraction unit is used to score and rank the candidate segments using a re-ranking model, and input the top-ranked candidate segments as effective information context into the large language model, which then performs the location and extraction of key field values. The output extraction module is used to fill the extracted key field values ​​into the corresponding positions according to the field mapping relationship defined in the output file template, and generate a structured extraction file.

[0042] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Based on the technical essence of the present invention, any simple modifications, equivalent substitutions, and improvements made to the above embodiments within the spirit and principles of the present invention shall still fall within the protection scope of the present invention.

Claims

1. A method for extracting information from templated documents based on multimodal data extraction, characterized in that, Specifically, the following steps are included: S1. Obtain the document attachments uploaded by the user, perform orientation anomaly detection and rotation correction on the document attachments, and convert them into standard documents in a preset format; S2. Responding to the extraction template selected by the user, the standard document is partitioned and identified according to the document structure to obtain several text regions. The text content in each text region is segmented and identified to obtain text fragments. S3. Vectorize and encode the segmented text fragments and store them in the vector database; based on the field requirement description of the extracted template, retrieve the text fragments corresponding to the field requirement description from the vector database as candidate fragments; S4. Use a re-ranking model to score and rank the candidate segments based on their relevance. Input the top-ranked candidate segments as effective information context into the large language model, and then the large language model will perform the location and extraction of key field values. S5. Fill the extracted key field values ​​into the corresponding positions according to the field mapping relationship defined in the output file template to generate a structured extraction file.

2. The method for extracting templated document information based on multimodal data extraction according to claim 1, characterized in that, The specific process of anomaly detection and rotation correction is as follows: Render each page of the document attachment as a preview image, input it sequentially into the trained orientation detection OCR model, and output the classification result of the page rotation angle; When the classification result is page tilt, the page is judged to be abnormal and its page number and rotation angle are recorded, and a page-level orientation abnormality mapping table is established. According to the orientation abnormality mapping table, each abnormal page is rotated in the opposite direction at the corresponding angle so that all pages are unified to the normal reading direction.

3. The method for extracting templated document information based on multimodal data extraction according to claim 2, characterized in that, The orientation detection OCR model includes a spatial attention module, which distinguishes between the main text region and the specific label region based on the density and continuity of the text, and assigns a higher attention weight to the main text region than to the specific label region. Based on the main text region, it determines the normal reading direction of the text and the classification result of the page tilt angle.

4. The method for extracting templated document information based on multimodal data extraction according to claim 3, characterized in that, When the classification result indicates that the page is not tilted, the feature vector of the main text region is extracted, and its cosine similarity with the standard forward text distribution reference vector is calculated. When the cosine similarity is lower than the preset threshold, the page is marked as a page with questionable orientation and a secondary verification is triggered. The secondary verification involves calling the text connectivity analysis algorithm to detect the baseline tilt angle of the text lines within the page. If the baseline tilt angle exceeds the preset tolerance range, a fine-grained rotation correction is performed.

5. The method for extracting templated document information based on multimodal data extraction according to claim 1, characterized in that, The specific process of obtaining a text fragment: The standard document is partitioned and identified based on the density and continuity of the text to obtain several text regions; Each text region is segmented according to the document structure, and each structural unit is treated as a text block. The size of each text block is checked to see if it exceeds a preset range. If it does, the text blocks are recursively split and then merged. Metadata information is preserved during the recursive splitting.

6. The method for extracting templated document information based on multimodal data extraction according to claim 5, characterized in that, The specific process of recursive decomposition is as follows: Construct block-level metadata for each text block. The block-level metadata includes the page number to which the text block belongs, the parent title, a list of candidate field key names contained in the text block, whether it is a table block, the table row and column range, and cross-page markers. For each text block, an initial token count is performed. If the initial token count exceeds a preset limit, the text block will be split. If the initial number of tokens in a text block is lower than a preset limit, then the adjacent text blocks of that text block are obtained and merged based on the consistency of the parent title and semantic continuity in the block-level metadata.

7. The method for extracting templated document information based on multimodal data extraction according to claim 6, characterized in that, Text blocks are divided into paragraph blocks and table blocks according to the document structure. The breakdown of text blocks includes: Perform semantically preserving splitting operations based on sentence boundaries on paragraph blocks; For table blocks, if the table in the table block is a cross-page table, it will be split naturally along the page boundary; otherwise, the table will be kept intact and only the header row and data row will be split. After splitting, the child blocks will inherit the block-level metadata of the parent block and be marked with the split sequence number.

8. The method for extracting templated document information based on multimodal data extraction according to claim 1, characterized in that, The specific process of obtaining candidate slices includes: Generate query text based on the field requirements description of the extracted template, encode the query text into query vector through the embedding model, and encode the fragmented text fragments into fragment vectors. Perform a hybrid search in the vector database. The hybrid search performs two search paths simultaneously: the first is vector similarity search, which calculates the cosine similarity between the query vector and each piece vector, and returns the top K1 pieces with the highest similarity. The second method is full-text search, which constructs full-text search conditions based on keywords in the query text, matches the text content in each segment vector, and returns the top K2 segments with the highest matching degree. The results from the two retrieval paths are merged and deduplicated. The inverse ranking fusion algorithm is used to weight and merge the rankings of the same fragment vector in different retrieval paths to generate a unified candidate fragment list.

9. The method for extracting templated document information based on multimodal data extraction according to claim 6, characterized in that, The specific process of using the re-ranking model to score and rank the candidate slices based on their relevance is as follows: The field requirement description and each candidate slice are combined into a text pair, which is then input into a reordering model based on a cross-encoder architecture. The reordering model performs joint encoding on the text pairs and outputs a relevance score through a fully connected layer, representing the degree of semantic matching between the slice content and the field requirement. All candidate segments are sorted in descending order based on their relevance scores, and a segment sequence is formed by retaining several candidate segments with scores higher than a preset threshold and ranking higher as the fine screening result. For candidate segments in the screening results, read the parent title information in their block-level metadata. If the parent title block corresponding to the candidate segment is not selected in the screening results, the parent title block is added to the segment sequence in read-only context form to ensure that the large language model obtains a complete structural context.

10. A templated document information extraction system based on multimodal data extraction, characterized in that, include: The preprocessing module is used to obtain the document attachments uploaded by the user, perform orientation anomaly detection and rotation correction on the document attachments, and convert them into standard documents in a preset format. The information extraction module includes: The text segmentation unit is used to respond to the extraction template selected by the user, divide the standard document into several text regions according to the document structure, and perform segmentation and text recognition on the text content in each text region to obtain text fragments. The text filtering unit is used to vectorize and encode the segmented text fragments and store them in the vector database; then, based on the field requirement description of the extraction template, it retrieves the text fragments corresponding to the field requirement description from the vector database as candidate fragments. The location and extraction unit is used to score and rank the candidate segments using a re-ranking model, and input the top-ranked candidate segments as effective information context into the large language model, which then performs the location and extraction of key field values. The output extraction module is used to fill the extracted key field values ​​into the corresponding positions according to the field mapping relationship defined in the output file template, and generate a structured extraction file.

Images