Bank statement form recognition method and system fusing multi-path heterogeneous model

Abstract

This invention provides a method and system for recognizing bank statement forms by fusing multiple heterogeneous models, belonging to the field of data processing technology. It includes: receiving a form image to be recognized; inputting the form image into multiple heterogeneous form recognition sub-modules for recognition, each sub-module including at least a computer vision model and a multimodal large model; standardizing the recognition results from the multiple heterogeneous form recognition sub-modules to unify the data format, obtaining multiple standardized recognition results; and fusing the multiple standardized recognition results to output a unified structured recognition result. This invention overcomes the problems of existing technologies that can only handle regular forms, have limited scene adaptability, poor accuracy, and lack flexible scalability.

Description

Technical Field

[0001] This invention belongs to the field of data processing technology, and specifically relates to a method and system for recognizing bank statement forms by integrating multiple heterogeneous models. Background Technology

[0002] In recent years, with the continuous advancement of computer technology, algorithms, and big data, artificial intelligence (AI) technology has developed rapidly and is being implemented across various industries, driving social development. OCR refers to Optical Character Recognition (OCR), which is the ability to convert text in images into editable and searchable computer text. OCR technology first involves image processing and preprocessing, then uses pattern recognition, machine learning, and other techniques to recognize characters into a computer-readable and editable text format. OCR technology is an application of artificial intelligence, and its algorithms incorporate many achievements of AI technology. Document table recognition combines OCR technology with layout analysis to extract table content from documents and perform appropriate table conversion. In the financial field, there are various types of tabular data. Bank statements are electronic or paper documents provided by banks to customers, recording all account activities, including deposits, expenditures, and transfers. They are an important supplement and basis for users' bookkeeping and tax reporting. To use bank statement data, traditional financial processes require financial personnel to manually enter bank statements into the financial system, which is time-consuming, labor-intensive, and seriously affects the work efficiency of financial personnel. How to automatically input users' bank statement data into the financial system is a problem that urgently needs to be solved.

[0003] Several technical solutions and products already exist for extracting table content, mainly falling into the following two categories: 1) Traditional Computer Vision (CV) + OCR Solution This solution locates the table structure and divides cells through line detection and contour analysis, then uses OCR to recognize the text. Its core problems are poor robustness and weak adaptability. It is highly dependent on table format and can only handle well-defined, regular tables (such as standard bank statements). When encountering tables without borders or tables with incomplete borders due to blurry scanning, the positioning accuracy drops sharply, and problems such as cell misalignment and missed field recognition may occur. Furthermore, in this solution, text and structure are disconnected. If the OCR shows confusion between characters such as "8" and "B", or "6" and "G", the system cannot correct the errors by combining them with table logic (e.g., no letters in the amount field), and the influx of erroneous data increases the cost of subsequent verification.

[0004] 2) End-to-end solution for large-scale multimodal model (VLM) This solution relies on a large visual language model to directly output structured results. While it can handle complex layouts, it suffers from uncontrollable results and insufficient accuracy. First, the risk of "recognition illusion" is critical; when processing blurry images or complex cells, the model may fabricate non-existent fields. Second, accuracy is insufficient in small sample scenarios; when fine-tuning is needed and the sample size is small, the accuracy of the large model drops rapidly and cannot be quickly optimized, failing to adapt to the personalized needs of enterprises. Third, the output results of the large model (such as HTML) require secondary parsing to match the business data dictionary and cannot be linked to system verification rules (such as retaining two decimal places for amounts), still requiring manual verification and failing to achieve an automated closed loop. Summary of the Invention

[0005] Therefore, the technical problem to be solved by the present invention is to provide a bank statement form recognition method and system that integrates multiple heterogeneous models, which overcomes the problems of existing technologies that can only process regular forms, have limited scene adaptability, poor accuracy, and lack flexible expansion capabilities.

[0006] In a first aspect, the present invention provides a method for recognizing bank statement forms by fusing multiple heterogeneous models, comprising: S1, Receive the table image to be identified; S2, the table image is input to multiple heterogeneous table recognition sub-modules for recognition, and the heterogeneous table recognition sub-modules include at least a computer vision model, a multimodal large model and a domain-specific model; S3, standardize the recognition results of the multiple heterogeneous table recognition sub-modules, unify the data format, and obtain multiple standardized recognition results; S4, the multiple standardized recognition results are fused and a decision is made to output a unified structured recognition result, including: S41, Cell-level cross-validation and conflict set generation: Using the output of the multimodal large model as the reference grid, the recognition results of different heterogeneous table recognition sub-modules are mapped to the reference grid by calculating the intersection-union ratio of the cell center points or bounding boxes in the normalized multiple recognition results, and the cells recognized by the heterogeneous table recognition sub-modules are aligned; the differences are obtained by comparison and classified into conflict sets. S42, Dynamic confidence assessment: Based on the differences, the recognition results of the multiple heterogeneous table recognition sub-modules are quantitatively scored to obtain the confidence of the recognition results of the multiple heterogeneous table recognition sub-modules. S43, hierarchical decision conflict arbitration: based on the confidence level of the recognition results of the multiple heterogeneous table recognition sub-modules, a preset rule engine is used to make a decision and output the unified structured recognition result.

[0007] Further, in S41, the conflict set includes structural conflicts, content conflicts, and existence conflicts; wherein, the structural conflict is a merging / splitting conflict of one-to-many / many-to-one matching; the content conflict is a conflict in which cells are successfully mapped one-to-one but the text strings are different; and the existence conflict is a dangling conflict in which any cell identified by the heterogeneous table recognition submodule has no corresponding cell in the reference grid.

[0008] Furthermore, in step S42, the recognition results of the multiple heterogeneous table recognition sub-modules are quantitatively scored, and the indicators used include image quality indicators, confidence scores output by each heterogeneous table recognition sub-module, cross-validation consistency ratio, and domain knowledge / business logic features.

[0009] Furthermore, in S2, when the table image is input to the multiple heterogeneous table recognition sub-modules for recognition, an intelligent scheduling rule is adopted: in the initial stage, the computer vision model and the domain-specific model are called first. If the proportion of cells that fail the business logic verification in the output result exceeds the preset proportion threshold, or the table line detection completeness is lower than the preset threshold, it is determined to be a complex and difficult sample, and then the multimodal large model is called for recognition.

[0010] Furthermore, the image quality metrics include local sharpness, local contrast, and binarization quality; the local sharpness is evaluated by calculating the variance of the cell image region using the Laplacian operator; the local contrast is evaluated by calculating the grayscale histogram of the cell image region; and the binarization quality is evaluated by the morphological features of the connected components of the binarized cell image region. The confidence scores output by each heterogeneous table recognition submodule include the average or minimum confidence scores of the OCR character-level and word-level of the computer vision model, the average log probability of the token sequence of the multimodal large model, and the normalized cross-correlation score or feature point matching quality score of the domain-specific model. The cross-validation consistency ratio is the proportion of the number of heterogeneous table recognition submodules that output the same result. The domain knowledge / business logic features include data format validation based on regular expressions, numerical range and relationship validation, and transaction terminology dictionary matching validation.

[0011] Furthermore, the scoring function for the dynamic confidence assessment is: S=w_img×F_img+w_model×F_model+w_cross×F_cross+w_logic×F_logic; Where: S is the overall confidence score; F_img is the normalized value of the image quality feature; F_model is the normalized value of the model's intrinsic confidence feature; F_cross is the normalized value of the cross-validation consistency feature; F_logic is the normalized value of the domain knowledge / business logic characteristics; w_img, w_model, w_cross, and w_logic are weight coefficients obtained by training a logistic regression or gradient boosting decision tree model on a labeled dataset, with w_logic having a higher weight than the other coefficients.

[0012] Furthermore, in S43, the preset rule engine includes: High-confidence consensus: If the recognition results of the multiple heterogeneous table recognition submodules in a certain cell are consistent and the confidence level is higher than the preset first threshold, then the result is directly adopted; Structural conflict arbitration: Calculate the overall confidence S_merged of the merged cell scheme and the average confidence S_split_avg of the multiple independent cell schemes respectively; if the preset conditions are not met, the structural judgment of the multimodal large model shall be adopted first. The preset conditions are that S_merged is lower than S_split_avg, and the content of each independent cell passes the business logic validation. Content-based conflict arbitration: Adopt the candidate result with the highest overall confidence score S; Existence conflict arbitration: For dangling results without corresponding cells in the baseline grid, the results are determined to be either missed or misidentified based on image quality features and business logic verification, and are either removed or supplemented. Manual verification trigger: If the confidence level of the recognition results of multiple heterogeneous table recognition sub-modules is lower than the preset second threshold, the manual verification process will be initiated.

[0013] Furthermore, the unified data format in S3 is a JSON structure based on cells, with each cell containing the following fields: cell position coordinates, cell text content, cell merge status, and identification source module identifier.

[0014] Secondly, the present invention provides a bank statement form recognition system that integrates multiple heterogeneous models, comprising: The input module receives the table image to be recognized; The recognition module inputs the table image into multiple heterogeneous table recognition sub-modules for recognition. The heterogeneous table recognition sub-modules include at least a computer vision model, a multimodal large model, and a domain-specific model. The standardization module standardizes the recognition results of the multiple heterogeneous table recognition sub-modules, unifies the data format, and obtains multiple standardized recognition results. The decision module integrates the multiple standardized recognition results and outputs a unified structured recognition result.

[0015] Beneficial effects: The accuracy of complex layout (borderless, merged cells) recognition in this invention has been improved from 75% to 92% in the prior art, and the error rate of key fields such as "amount" and "tax" in the statement has been reduced from 5% to 0.8%, meeting the "accounts and actual situation consistency" requirement of the ERP financial module. This invention supports the recognition of 12 common document types (reconciliation statements, purchase orders, expense reports, etc.) in ERP systems, expanding the scenario adaptability from 3 types in traditional solutions to 12 types. Furthermore, if a single submodule fails, other modules can continue to operate, achieving a system availability of 99.9%. Using an ERP system reduces manual verification costs by 70% (due to the high accuracy of the integrated results, no secondary manual verification is required). The pluggable domain-specific model allows enterprises to customize templates based on new document types added to ERP business (such as customs declarations), shortening the access cycle from 15 days in the existing technology to 3 days, thus adapting to the needs of enterprise business iteration. Attached Figure Description

[0016] To make the content of this invention easier to understand, the invention will be further described in detail below with reference to specific embodiments and accompanying drawings.

[0017] Figure 1 This is a schematic diagram of the method flow of Embodiment 1 of the present invention. Detailed Implementation

[0018] The present invention will now be described in detail with reference to the accompanying drawings and embodiments. The principles and features of the present invention are described below with reference to the accompanying drawings. It should be noted that, unless otherwise specified, the embodiments and features described in these embodiments can be combined with each other. The embodiments given are only for explaining the present invention and are not intended to limit the scope of the present invention.

[0019] This invention provides a method and system for recognizing bank statement forms by integrating multiple heterogeneous models. Its core focus is on "structured data extraction from tabular documents in ERP system scenarios." Through multi-model collaborative optimization, it achieves accurate and efficient conversion of unstructured tabular images (such as business documents and financial vouchers) into structured data that can be directly accessed by the ERP system. It primarily serves data entry, process automation, and business collaboration within enterprise ERP systems. Specific application scenarios are as follows: 1. Automated data entry for the ERP financial module This invention addresses the high-frequency processing of tabular documents in enterprise ERP financial systems, such as bank statements, VAT invoices, expense reimbursement forms, and purchase payment orders. It automatically identifies and extracts core fields like account details, amounts, taxes, supplier information, and payment details from these documents, directly synchronizing them to the ERP financial module's general ledger, accounts payable, and expense management subsystems, replacing traditional manual data entry. This solves the problems of low data entry efficiency and discrepancies between accounts and actual records caused by human error in ERP financial data entry. It also automates processes such as financial closing and tax filing, reducing repetitive labor costs for financial personnel.

[0020] 2. Collaborative processing of documents in the ERP supply chain module This system automatically extracts key data such as product name, specifications, quantity, unit price, warehouse location, and supplier code from purchase orders, packing lists, receiving slips, and supplier reconciliation reports in ERP supply chain management. This data is then synchronized in real-time to the ERP procurement and inventory management subsystems, enabling automatic data matching and verification between purchase orders, receiving slips, and reconciliation reports. This avoids problems such as inaccurate inventory information and low procurement reconciliation efficiency caused by delays in manual entry of supply chain documents, improving the ERP system's dynamic control over supply chain operations and shortening the procurement settlement cycle.

[0021] 3. ERP cost accounting and report data support This invention addresses the data required by enterprise ERP cost modules, such as production timesheets, material consumption details, and departmental expense allocation tables. It precisely extracts fields like timesheets, material usage, and expense amounts, automatically importing them into the ERP cost accounting model to assist in product cost calculation, expense allocation, and cost analysis. Furthermore, it allows for reverse data verification and structured archiving of monthly operating reports and inventory count sheets generated by the ERP system, ensuring consistency between ERP system data and actual business operations and providing reliable data support for enterprise business decisions.

[0022] 4. Integration and adaptation between the ERP system and external documents In business collaborations between enterprises and external partners, this invention addresses the challenges of recognizing different formatted forms such as customer reconciliation letters, logistics waybills (including cargo detail tables), and third-party audit schedules. It breaks down the barriers to identifying external document data by structuring the data and importing it into ERP systems (such as customer management and sales management subsystems). This solves the data "gap" between the ERP system and external documents, enabling efficient data exchange between internal and external entities and supporting automated processing of customer reconciliation, sales settlement, and other business processes.

[0023] Example 1 This embodiment addresses the shortcomings of existing technologies by focusing on resolving the core issue of balancing "accuracy, robustness, and cost" in table recognition systems. Specifically, it aims to overcome the following pain points: 1. Traditional CV-OCR solutions are not robust to complex layouts such as those without borders and merged cells, and can only handle regular tables, limiting their adaptability to various scenarios; 2. Single multimodal large model solutions suffer from "recognition illusions" (such as the generation of fake cells), and accuracy is limited in small sample scenarios; 3. The existing system lacks flexible scalability and cannot be customized to meet the enterprise's ERP business scenarios (such as reconciliation statements and purchase orders). Furthermore, when multiple solutions are deployed independently, data cannot be shared, making it difficult to form a collaborative advantage.

[0024] like Figure 1 As shown, the workflow of this embodiment is as follows: Existing technology: The individual OCR sub-modules used in this embodiment (such as sub-modules A, B, and C in the schematic diagram) are based on existing technologies. For example, sub-module A can be a mature open-source CV-OCR engine, and sub-module B can be a pre-trained multimodal large model.

[0025] Improvements and novel aspects: The novelty of this invention lies in organizing these independent technical components into a new parallel, redundant, and complementary architecture, and inventing an "intelligent fusion module" at the core of the system. The "dynamic confidence assessment model" and "hierarchical decision arbitration mechanism" within this module are the key innovations of this embodiment.

[0026] When the system receives an image of a bank statement form to be recognized, it determines the timing of calling each submodule according to intelligent scheduling rules. Initially, the image is sent to some heterogeneous table recognition submodules (such as computer vision models and domain-specific models); when a complex or difficult sample is identified, the multimodal large model is then called for recognition. Each submodule is an "expert," leveraging its strengths to independently complete the recognition task. Submodule A (traditional CV model): It excels at handling regular tables with clear borders. It uses line detection algorithms to quickly and cost-effectively analyze the row and column structure of the table and recognize the text. Submodule B (multimodal large model): It excels at understanding complex tables without borders and with flexible layouts. Through its powerful visual language understanding capabilities, it can directly generate the HTML structure of the table, compensating for the shortcomings of submodule A. Submodule C (domain-specific model): This is a pluggable module that can be customized according to specific business needs. It is a high-speed template matching model that can complete tasks with extremely low cost and extremely high accuracy.

[0027] Because the data formats output by each submodule (such as coordinate lists and HTML code) are different, they cannot be directly compared. Therefore, the result normalization module will parse all the original output into a standard intermediate data format (e.g., a JSON list with cells as the smallest unit), laying the foundation for subsequent intelligent fusion.

[0028] The intelligent fusion decision module is the key to this invention; it is responsible for fusing the results from various identification methods into a single final solution. The steps of intelligent fusion are as follows: Step 1: Cell-level cross-validation and conflict set generation. The goal of this step is to accurately align the recognition results from different sub-modules (A: CV+OCR, B: VLM, C: template matching) and construct a "conflict set" to be arbitrated.

[0029] Baseline Grid Mapping: To avoid confusion, the system first selects the "most reliable" structure as the baseline. Typically, the HTML or JSON output of the multimodal large model (sub-module B) is more likely to have a correct macro-table structure (total number of rows and columns, merged cell relationships) due to its stronger understanding of complex layouts. Therefore, the system uses the output of sub-module B as the baseline grid. The recognition results of sub-module A (coordinate list) and sub-module C (predefined region) are mapped by calculating the Intersection over Union (IOU) ratio between the cell center point or bounding box and the baseline grid cells. A high IOU threshold is set (e.g., ...). 0.7) links cells from different sources that point to the same physical location.

[0030] Precise classification and labeling of conflict types: After mapping, the system categorizes all inconsistent cells and stores them in a "conflict set", and gives each conflict point a precise label so that subsequent modules can handle them in a targeted manner.

[0031] 1. Structural Conflict (One-to-Many / Many-to-One Matching): When a cell in the baseline grid (e.g., a merged cell) has a high IOU with multiple cells in another module, it is marked as a "Merge / Split Conflict". Example: Submodule B identifies a merged cell "Transaction Time" spanning two columns, while submodule A identifies two separate cells, "Transaction Date" and "Transaction Time". The system packages these three cells (one merged and two separate) into a structural conflict group.

[0032] 2. Content conflict (text difference): One-to-one cell mapping successful (IOU) (0.7), but the internally recognized text strings are different. Example: For the same amount cell, submodule A recognizes it as "8,000.00", while submodule B recognizes it as "B,000.00". The system marks these two results as content conflicts.

[0033] 3. Existence Conflict (Dangling Cell): A cell identified by a module cannot find any corresponding cell with a sufficiently high IOU in the baseline grid. This usually indicates a missed or false identification. Example: Due to scanning artifacts, submodule A identifies a meaningless text box in the footer blank area, but this area is empty in the result of submodule B. This cell is marked as an existence conflict.

[0034] Step 2: Dynamic confidence assessment. For each candidate result in the conflict set (for example, for content conflict, there are two candidate options: "8,000.00" and "B,000.00"), the system will extract features from multiple dimensions to construct a multi-dimensional feature vector for subsequent quantitative scoring.

[0035] Image quality characteristics: 1. Local Sharpness: The variance of the cell image region is calculated using the Laplacian operator. The larger the variance, the sharper the image edges, the higher the sharpness, and the more reliable the recognition result for that region.

[0036] 2. Local Contrast: Calculate the grayscale histogram of the cell region to evaluate its dynamic range. Contrast that is too low or too high can affect OCR accuracy.

[0037] 3. Binarization Quality: For traditional OCR, the binarization effect in the preprocessing step is crucial. The morphological features (such as aspect ratio and area) of connected components within the binarized region can be evaluated. Distorted or broken connected components indicate low-quality recognition.

[0038] Model intrinsic confidence features: 1. Submodule A (CV+OCR): Extracts the average or minimum of character-level confidence and phrase-level confidence returned by the OCR engine (such as Tesseract).

[0039] 2. Submodule B (VLM): Extracts the average log probability of the token sequence that generated the text in this cell. The higher the probability, the more confident the model is in the content it generates.

[0040] 3. Submodule C (Template Matching): Extracts the normalized cross-correlation (NCC) score of template matching or the quality score of feature point matching.

[0041] Cross-validation consistency characteristics: Consistency breadth: For a given point of conflict, how many submodules (≥2) output the same result. For example, if A and C both identify as "8,000.00", while B identifies as "B,000.00", then the former's consistency breadth is 2 / 3.

[0042] Domain knowledge / business logic characteristics: 1. Data format validation: Apply regular expressions to validate the cell content based on the column headings (such as "Transaction Date" and "Transaction Amount").

[0043] Example: In the "Transaction Date" column, the format validation score for "2023-10-27" is 1, while the score for "2023-10-27" is 0. In the "Amount" column, "8,000.00" matches the amount format and scores 1; while "B,000.00" does not match and scores 0.

[0044] 2. Numerical Range and Relationship Validation: Balance Validation: Check if "Previous Balance" + / - "Income and Expenditure Amount" is approximately equal to "Current Balance". For rows that satisfy this relationship, the confidence level of all cells within that row should be increased. Dictionary Matching: Check if the summary column contains common banking transaction terms (such as "transfer", "interest", "spending", etc.).

[0045] Step 3: Hierarchical Decision Making and Conflict Arbitration. This step introduces a lightweight decision model (rather than a simple rule engine) to comprehensively evaluate the feature vectors generated in Step 2 and output the final result.

[0046] Confidence scoring model: Weighted fusion formula: For each candidate result, its final confidence score S can be calculated using a linear weighted model: S=w_img×F_img+w_model×F_model+w_cross×F_cross+w_logic×F_logic Here, F represents the normalized value of each feature dimension, and w is the corresponding weight. These weights can be obtained by training models such as Logistic Regression or Gradient Boosting Decision Trees (GBDT) on a small, labeled dataset, making the weight allocation more scientific and adaptive. The weights (w_logic) of business logic features are usually set higher because they are directly related to the final usability of the data.

[0047] Hierarchical decision-making arbitration logic: 1. High-Confidence Consensus (Fast Track): If the results of multiple sub-modules are consistent, and their calculated comprehensive confidence scores S are all higher than a high threshold (such as 0.95), the result is directly adopted without going through complex arbitration, thus improving processing efficiency.

[0048] 2. Structural Conflict Arbitration (Merged Cells vs. Multiple Independent Cells): Calculate the "overall confidence" under both structural assumptions. For example, calculate the confidence S_merged for the merged cell scheme and the average confidence S_split_avg for all cells in the multiple independent cell scheme. Simultaneously, prioritize the structural judgment of VLM (submodule B), assigning it an additional weight or directly adopting it, unless its S_merged is significantly lower than S_split_avg, and the content of all independent cells has passed rigorous business logic validation.

[0049] 3. Content-based conflict arbitration (same structure, different text. For example, "8" vs. "B"): Directly select the candidate result with the highest overall confidence score S. Example: For the conflict between "8,000.00" and "B,000.00", although model B (VLM) itself may have a high intrinsic confidence in its output "B" (high F_model), since "B,000.00" scores 0 on the business logic feature F_logic, its final overall score S will be much lower than "8,000.00". Therefore, the system will resolutely adopt "8,000.00".

[0050] 4. Intelligent Scheduling Optimization: In the initial stage, the computer vision model and domain-specific model are prioritized. When a large number of low-confidence samples appear in the recognition results, for example, if the proportion of cells failing business logic verification exceeds a preset threshold, or if the table line detection completeness is below a preset threshold, the system classifies this as a complex and difficult sample. Only when a sample is classified as complex and difficult will the multimodal large model be called for recognition. In this way, high accuracy is maintained while achieving low-cost and high-efficiency processing of most simple tables.

[0051] Through the above process, a more accurate and complete structured result is ultimately output than any single submodule.

[0052] Example 2 This embodiment discloses a bank statement form recognition system that integrates multiple heterogeneous models, including: The input module receives the table image to be recognized; The recognition module inputs the table image into multiple heterogeneous table recognition sub-modules for recognition. The heterogeneous table recognition sub-modules include at least a computer vision model and a multimodal large model. The standardization module standardizes the recognition results of the multiple heterogeneous table recognition sub-modules, unifies the data format, and obtains multiple standardized recognition results. The decision module integrates the multiple standardized recognition results and outputs a unified structured recognition result.

[0053] This embodiment has made significant progress in extracting bank statement forms. The recognition accuracy of complex bank statement formats (without borders and merged cells) has reached 92%, and the error rate for key fields such as amount has been reduced to 0.8%.

[0054] Obviously, the above embodiments are merely illustrative examples for clear explanation and are not intended to limit the implementation. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all possible implementations here. However, obvious variations or modifications derived therefrom are still within the scope of protection of this invention.

Claims

1. A method for recognizing bank statement forms by integrating multiple heterogeneous models, characterized in that, include: S1, Receive the table image to be identified; S2, the table image is input to multiple heterogeneous table recognition sub-modules for recognition, and the heterogeneous table recognition sub-modules include at least a computer vision model, a multimodal large model and a domain-specific model; S3, standardize the recognition results of the multiple heterogeneous table recognition sub-modules, unify the data format, and obtain multiple standardized recognition results; S4, the multiple standardized recognition results are fused and a decision is made to output a unified structured recognition result, including: S41, Cell-level cross-validation and conflict set generation: Using the output of the multimodal large model as the reference grid, the recognition results of different heterogeneous table recognition sub-modules are mapped to the reference grid by calculating the intersection-union ratio of the cell center points or bounding boxes in the normalized multiple recognition results, and the cells recognized by the heterogeneous table recognition sub-modules are aligned; the differences are obtained by comparison and classified into conflict sets. S42, Dynamic confidence assessment: Based on the differences, the recognition results of the multiple heterogeneous table recognition sub-modules are quantitatively scored to obtain the confidence of the recognition results of the multiple heterogeneous table recognition sub-modules. S43, hierarchical decision conflict arbitration: based on the confidence level of the recognition results of the multiple heterogeneous table recognition sub-modules, a preset rule engine is used to make a decision and output the unified structured recognition result.

2. The method according to claim 1, characterized in that, In step S41, the conflict set includes structural conflicts, content conflicts, and existence conflicts; wherein, the structural conflict is a merging / splitting conflict of one-to-many / many-to-one matching; the content conflict is a conflict in which cells are successfully mapped one-to-one but the text strings are different; and the existence conflict is a dangling conflict in which any cell identified by the heterogeneous table recognition submodule has no corresponding cell in the reference grid.

3. The method according to claim 1, characterized in that, In step S42, the recognition results of the multiple heterogeneous table recognition sub-modules are quantitatively scored. The indicators used include image quality indicators, confidence scores output by each heterogeneous table recognition sub-module, cross-validation consistency ratio, and domain knowledge / business logic features.

4. The method according to claim 1, characterized in that, In step S2, when the table image is input to the multiple heterogeneous table recognition sub-modules for recognition, an intelligent scheduling rule is adopted: in the initial stage, the computer vision model and the domain-specific model are called first. If the proportion of cells that fail the business logic verification in the output result exceeds the preset proportion threshold, or the table line detection completeness is lower than the preset threshold, it is determined to be a complex and difficult sample, and then the multimodal large model is called for recognition.

5. The method according to claim 3, characterized in that, The image quality metrics include local sharpness, local contrast, and binarization quality; local sharpness is evaluated by calculating the variance of a cell image region using the Laplacian operator; local contrast is evaluated by calculating the grayscale histogram of the cell image region; and binarization quality is evaluated by the morphological features of the connected components of the binarized cell image region. The confidence scores output by each heterogeneous table recognition submodule include the average or minimum confidence scores of the OCR character-level and word-level of the computer vision model, the average log probability of the token sequence of the multimodal large model, and the normalized cross-correlation score or feature point matching quality score of the domain-specific model. The cross-validation consistency ratio is the proportion of the number of heterogeneous table recognition submodules that output the same result. The domain knowledge / business logic features include data format validation based on regular expressions, numerical range and relationship validation, and transaction terminology dictionary matching validation.

6. The method according to claim 1, characterized in that, The scoring function for the dynamic confidence assessment is: S=w_img×F_img+w_model×F_model+w_cross×F_cross+w_logic×F_logic; Where: S is the overall confidence score; F_img is the normalized value of the image quality feature; F_model is the normalized value of the model's intrinsic confidence feature; F_cross is the normalized value of the cross-validation consistency feature; F_logic is the normalized value of the domain knowledge / business logic characteristics; w_img, w_model, w_cross, and w_logic are weight coefficients obtained by training a logistic regression or gradient boosting decision tree model on a labeled dataset, with w_logic having a higher weight than the other coefficients.

7. The method according to claim 1, characterized in that, In S43, the preset rule engine includes: High-confidence consensus: If the recognition results of the multiple heterogeneous table recognition submodules in a certain cell are consistent and the confidence level is higher than the preset first threshold, then the result is directly adopted; Structural conflict arbitration: Calculate the overall confidence S_merged of the merged cell scheme and the average confidence S_split_avg of the multiple independent cell schemes respectively; if the preset conditions are not met, the structural judgment of the multimodal large model shall be adopted first. The preset conditions are that S_merged is lower than S_split_avg, and the content of each independent cell passes the business logic validation. Content-based conflict arbitration: Adopt the candidate result with the highest overall confidence score S; Existence conflict arbitration: For dangling results without corresponding cells in the baseline grid, the results are determined to be either missed or misidentified based on image quality features and business logic verification, and are either removed or supplemented. Manual verification trigger: If the confidence level of the recognition results of multiple heterogeneous table recognition sub-modules is lower than the preset second threshold, the manual verification process will be initiated.

8. The method according to claim 1, characterized in that, The unified data format in S3 is a JSON structure based on cells, with each cell containing the following fields: cell position coordinates, cell text content, cell merge status, and identification source module identifier.

9. A bank statement form recognition system integrating multiple heterogeneous models, characterized in that, include: The input module receives the table image to be recognized; The recognition module inputs the table image into multiple heterogeneous table recognition sub-modules for recognition. The heterogeneous table recognition sub-modules include at least a computer vision model, a multimodal large model, and a domain-specific model. The standardization module standardizes the recognition results of the multiple heterogeneous table recognition sub-modules, unifies the data format, and obtains multiple standardized recognition results. The decision module integrates the multiple standardized recognition results and outputs a unified structured recognition result.

Images