Intelligent recognition and structure restoration method for large-scale and high-complexity table

Abstract

This invention provides an intelligent recognition and structural reconstruction method for large and highly complex tables, belonging to the field of computer vision and document image recognition technology. It includes the following steps: PDF rendering and image processing, content-aware image preprocessing and segmentation, local cell detection and candidate structure construction, cross-block splicing and redundancy elimination, text direction awareness and multilingual OCR recognition, and structural content mapping and standardized export. This invention designs an adaptive segmentation strategy based on layout features and structural cues, as well as a cell intelligent detection algorithm for large and highly complex tables—TableVerseNet. It also designs a redundancy elimination strategy based on spatial alignment and edge overlap determination. This invention can achieve high-precision detection of table content and character recognition, as well as complete reconstruction and restoration of semantic structure, in various practical application scenarios such as large tables, multilingual mixed text, multi-directional text, and highly complex table structures.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and document image recognition technology, and in particular to a method for intelligent recognition and structural reconstruction of large and highly complex tables. Background Technology

[0002] In the field of document image recognition, existing technologies have developed relatively mature methods for text and table recognition, which can reliably achieve the recognition and extraction of text from printed tables. The core technologies mainly include the following categories: (1) Image preprocessing techniques: Image quality is improved and the accuracy of subsequent recognition algorithms is enhanced through operations such as image binarization, noise reduction, edge enhancement, and character region segmentation.

[0003] (2) Character recognition technology: Early methods relied on template matching, feature engineering and traditional classifiers (such as SVM, random forest, etc.), while the mainstream solutions in recent years have adopted convolutional neural networks (CNN) or Transformer structures based on deep learning to achieve end-to-end character-level recognition.

[0004] (3) Table recognition and structure reconstruction technology: This direction is mainly based on image processing and layout analysis methods, and restores the two-dimensional structure of the table through steps such as row and column detection, cell location, border extraction and text attribution judgment. In recent years, deep learning models (such as TableNet and CascadeTabNet) have been used to improve the robustness of table detection and structure modeling in complex scenarios.

[0005] (4) PDF document parsing technology: The common method is to convert the PDF into an image and then perform OCR recognition; some commercial tools (such as Adobe Acrobat) can directly parse the text layer, but for scanned PDFs (i.e. image-type documents without text layers), it is still necessary to rely on visual recognition and structural analysis algorithms to complete the content extraction.

[0006] Currently, there are several OCR systems on the market that have table recognition and structured reconstruction functions. Typical examples include: Google Doc AI and Amazon Textract can recognize tables and output structured results (such as JSON or Excel format), but their recognition and structure restoration accuracy drops significantly when faced with long documents, complex merged cells, nested structures, and mixed multilingual text.

[0007] Azure AI Document Intelligence is suitable for parsing structured template documents such as invoices and resumes, but it performs poorly in scenarios with free layout or non-standardized tables.

[0008] Open source tools (such as Tesseract, Camelot, PdfPlumber): Tesseract supports multilingual text recognition, while Camelot and PdfPlumber can extract regular border-type tables. However, in scanned images, with blurred boundaries and misaligned structures, the structure recognition accuracy is low and it is difficult to automatically generate a complete Excel format.

[0009] Research-oriented solutions (such as TableNet and DeepDeSRT) achieve pixel-level segmentation and structural modeling of tables through deep neural networks. They perform well on public datasets (such as PubTabNet), but their generalization and interpretability are insufficient, making them difficult to apply directly to complex real-world scenarios.

[0010] In summary, the existing technology still has the following prominent problems: (1) It is difficult to reliably support the complete recognition and restoration of complex tables in long documents; (2) The accuracy rate of recognition for mixed multilingual text, different writing directions and diverse font styles is low; (3) Insufficient structural restoration capability, especially lack of accurate modeling in structural scenarios such as merged cells, blank cells, nested or cross headers; (4) The structured results are not accurate enough, and the generated Excel tables often need to be manually corrected and processed.

[0011] When the recognition target expands from ordinary text documents and simple table images to large, multilingual, and highly complex table images, the performance of traditional OCR technologies and table recognition algorithms still has significant limitations. Especially in scenarios involving large tables (more than 100 rows and columns), multilingual text (simultaneous text in multiple languages ​​and with multiple text directions), and highly complex structures (including multi-level row and column headers, merged cells, nested headers, and cross-tabulations), existing methods struggle to simultaneously achieve detection accuracy, character recognition accuracy, and structural integrity, resulting in unsatisfactory table content recognition and reconstruction. Specifically, existing technologies mainly suffer from the following shortcomings: (1) Insufficient table detection accuracy. When the table is large (more than 100 rows and more than 100 columns) and has a complex structure (containing multiple levels of row and column headers, merged cells, nested headers and cross-tabs, etc.), that is, when the table image contains large-scale rows and columns, merged cells across rows and columns, nested headers or cross-tabs, the existing detection algorithms have difficulty accurately identifying all cell boundaries. They often miss detections, mis-merge cells or incomplete segmentation. The phenomenon of missed detections and false detections is significant, and the overall recognition accuracy is low. This leads to subsequent processing based on incorrect detection results, thereby undermining the reliability of the overall recognition chain.

[0012] (2) Character recognition models have poor versatility. In table scenarios with mixed languages ​​or diverse text orientations, existing OCR models often have high false recognition rates and character omissions. In particular, when dealing with vertical, rotated, or special font English and symbols, omissions, character loss, or recognition errors are likely to occur. The robustness to heterogeneous fonts, vertical text, rotated text, and language-specific symbols is insufficient. Therefore, the versatility of character recognition models is limited.

[0013] (3) Weak ability to restore table structure. When faced with large and complex table images, existing methods often cannot fully reconstruct the hierarchical relationship and semantic structure of the table. They are unable to correctly restore the original nested table headers, merged cells, cross tables and other structural features, and cannot maintain the alignment and hierarchical correspondence between cells, resulting in distorted restoration results and loss of semantics, which seriously affects the accuracy of subsequent data parsing and structured processing.

[0014] (4) Existing algorithms have weak structural restoration capabilities. Many solutions are unable to restore the multi-level semantic relationships and original hierarchical information of tables. When encountering irregular merging, nested headers or cross tables, they often cannot accurately reconstruct the row and column correspondences and cannot output an Excel file that is semantically consistent with the input image content.

[0015] In summary, existing methods have significant shortcomings in detection accuracy, multilingual compatibility, structural modeling, and automated export, hindering their effective application in industrial and large-scale document processing scenarios. Therefore, there is an urgent need for a novel technical solution with high robustness and scalability, capable of achieving high-precision content recognition, semantic structure reconstruction, and editable format output in real-world applications involving large amounts of text, multiple languages, and highly complex tables, thus constructing a complete technical loop from image input to structured table output. Summary of the Invention

[0016] In view of this, the present invention aims to propose an intelligent recognition and structure restoration method for large and highly complex tables. This method is a table recognition and structure restoration method with high robustness and good scalability. It can achieve high-precision detection of table content and character recognition, complete reconstruction and restoration of semantic structure in various practical application scenarios such as large tables, multilingual mixed text, multi-directional text, and highly complex table structures. It can also output editable format files (such as Excel). Thus, it constructs a complete processing closed loop from image recognition, structure analysis to data restoration, significantly improving the accuracy and practicality of table recognition.

[0017] To achieve the above objectives, the technical solution of the present invention is implemented as follows: a method for intelligent recognition and structure restoration of large and highly complex tables, comprising the following steps: Step 1: PDF Rendering and Image Processing: Receive the PDF document to be processed, and use the PDF rendering module to convert each page into a bitmap image; ensure the preservation of table details through adaptive resolution and anti-aliasing strategies, laying the foundation for subsequent visual processing; Step 2: Content-aware image preprocessing and segmentation: Preprocess the rendered full-page bitmap image and perform adaptive segmentation based on layout features and structural clues; record the absolute position and relative offset information of each generated sub-block in the original page coordinate system to form an indexable position mapping table to ensure that local processing does not destroy global spatial semantics.

[0018] Step 3: Local cell detection and candidate structure construction: For each sub-plot, the TableVerseNet algorithm is called to detect cells, and the corner coordinates and confidence attributes of each cell are output. Based on the attributes output by each cell, a candidate table structure of cells in each sub-plot is constructed at the local scale to describe the relative row and column relationships and preliminary topological connections of the cells, providing a candidate basis for subsequent cross-block merging and hierarchy determination. Step 4: Cross-block stitching and redundancy elimination: Project the candidate cell images of each sub-block back to the original coordinate system through the position mapping table, and merge the cell instances of cross-blocks using spatial position alignment, boundary overlap judgment and edge redundancy adaptive elimination strategies. Then, use graph analysis methods to restore the global hierarchical relationships such as multi-level nested headers, cross tables and irregularly merged cells to generate a consistent and unambiguous full-page table structure diagram. Step 5: Text Direction Awareness and Multilingual OCR Recognition: For each defined cell region, text direction detection and language category prediction are performed first to determine whether the text is in horizontal or vertical format and to identify the possible language. Based on the direction and language information, the corresponding multilingual OCR sub-model is dispatched for accurate recognition, and the accurate bounding box and confidence of the recognized text are recorded at the same time. Step 6: Structure, Content Mapping, and Standardized Export: The full-page table structure diagram generated in Step 4 is matched cell by cell with the OCR recognition results in Step 5 to establish a one-to-one correspondence between cells and text content. For mappings with ambiguity or low confidence, rule-based validation and contextual consistency strategies are used for correction. Finally, based on the restored row and column hierarchy and merging relationships, cell merging, border styles, and formatting attributes are set according to standard Excel file specifications, and exported as an editable Excel document. This ensures that the output file is highly consistent with the original table in terms of visual presentation and semantic structure and can be directly used for subsequent manual or automated processing.

[0019] Furthermore, in step 1, a PDF rendering module with 500 dots per inch is used to convert each page of the PDF document into a bitmap image with a resolution of 9925×7017. The goal is to preserve the minute structure and font details of the table with minimal information loss, including details such as thin lines, dotted lines, and tiny annotations. During the rendering process, adaptive resolution and anti-aliasing strategies are enabled to maintain line continuity. The output adopts a lossless or near-lossless format, laying the foundation for subsequent visual processing.

[0020] Furthermore, step 2 includes four sub-processes: (1) Image preprocessing: Before image segmentation, a series of preprocessing steps are performed on the input image to improve the clarity and separation of table lines and content areas; First, the original image is subjected to grayscale conversion and adaptive normalization to balance the overall brightness distribution and alleviate uneven lighting in scanned or photographed documents; the formula is as follows: In the formula, and Thresholds The pixel ratio of the background to the foreground; for The variance is dimensionless; for The variance is dimensionless; Indicates that T is the minimum value. Subsequently, using The original image is binarized using a threshold to enhance the contrast between the table lines and the background, making the line structure more prominent in grayscale projection. To address noise and fracture lines, a median filter combined with morphological closing operations is used. This effectively removes isolated noise points and repairs virtual fracture edges, ensuring the stability of subsequent projection analysis. The processing formula is as follows: in, The image to be processed is dimensionless. It is a rectangular core, dimensionless; This is an expansion operation, dimensionless; This is a corrosion operation, dimensionless. (2) Content-aware layout projection and outline analysis: An adaptive projection and contour fusion strategy based on content distribution is introduced. By calculating the gray-level average projection curves of the image in the row and column directions, the content density distribution features are obtained. The gray-level average projections of the image in the row and column directions are as follows: in, and These are the image width and height, respectively; Represents the pixel values ​​of an image; The average pixel value along the row direction is dimensionless. The pixel mean along the column direction is dimensionless; the low-density areas correspond to the row spacing or column spacing of the table, calculated from the average projection of rows and columns. Combining the above contour detection to extract the bounding rectangle region of the valid content, the extracted bounding rectangle region is the contour positioning information. Non-table edge noise or title area is excluded to ensure that the cutting line is consistent with the actual table boundary. (3) Adaptive block segmentation strategy design: Based on the projection curve results and contour positioning information in the row and column directions, a dynamic cutting grid is constructed. This step automatically identifies horizontal and vertical low-density bands as candidate segmentation boundaries, generates a preliminary cutting scheme, and fine-tunes it through boundary expansion and alignment mechanisms to prevent characters, borders, or table lines from being truncated. To this end, the edge of each cutting block is extended outward by several pixels to align with the adjacent table edge or page edge, and uniformly corrected into an approximately rectangular area. The adaptive cutting strategy can avoid semantic loss caused by over-segmentation while maintaining the integrity of the internal structure of the cutting block. (4) Mapping of location and index information: To ensure the traceability of subsequent table reconstruction and spatial positioning, each sub-tile is generated with complete spatial metadata, including: Absolute coordinate information: Records the coordinates (x, y) of the top left corner of the sub-plot in the original image and its width and height (w, h) to ensure that any sub-plot can be restored to the full image coordinate system; Relative index information: Based on the cutting grid structure, assign a relative position index (i,j) to each sub-tile and identify its row and column position in the global structure; Adjacency mapping: Constructing topological relationships between sub-tiles and recording the top, bottom, left, and right adjacency information of each sub-tile for spatial stitching during reconstruction; Unique Identifier: A globally unique ID is generated for each sub-plot to ensure consistency and traceability during data flow.

[0021] Furthermore, step 3 includes: (1) Cell detection and boundary extraction: Each sub-plot is fed as input into TableVerseNet, a deep learning-based network for recognizing table structures. TableVerseNet consists of four core sub-modules: First, the FusionBackbone feature extractor: This feature extractor achieves multi-scale feature aggregation based on the ResNet and FPN network structures, and introduces deformable convolution to enhance the model's adaptability to different shaped table lines. At the same time, it improves the attention distribution in the intersection region through a lightweight Transformer module. Second, the bounding box detection branch: The bounding box detection branch borrows DBNet's differentiable binarization mechanism, which not only outputs the overall table frame, but also refines and generates inner cell frames and weak bounding boxes to distinguish between valid cell boundaries and noisy edges. Third, the structure mask branch: The structure mask branch uses Mask R-CNN to achieve pixel-level structure segmentation. It generates Row Mask and Column Mask through projection constraints in the row and column directions, which are used to assist in the hierarchical modeling of merged cells and nested headers. Fourth, Cell structure modeler: The Cell structure modeler uses a DETR-style Transformer decoder to decode each query vector into a cell, directly outputting the coordinates of the four corner points of its Bounding Box and the confidence score, providing a geometric basis for subsequent structure construction; (2) Cell deduplication and optimization: After the model outputs preliminary detection results, optimization and filtering are performed on duplicate detections, abnormal small boxes, and non-standard rectangles. First, the number of pixels A in each candidate cell is calculated. When A < ε, it is considered a noise region and is removed. ε is an empirical threshold. Second, the intersection-union ratio (IUU) is calculated for any two cells. The calculation formula is as follows: In the formula, express and The number of pixels in the intersection of the two regions; express and The number of pixels after merging the two regions; This represents the intersection, union, and ratio of numbers; it is a floating-point number and dimensionless. If IOU>θ, it is determined to be a duplicate detection and boundary merging is performed. For the detected non-standard quadrilateral cells Q={x1,y1,x2,y2,x3,y3,x4,y4}, the approximate rectangular boundary is refitted by the minimum bounding rectangle algorithm to ensure the geometric consistency of subsequent row and column clustering. All corner coordinates are uniformly retained to two decimal places in the final output to improve the accuracy of subsequent structure splicing. (3) Logical row and column clustering and index generation After obtaining the optimized set of cells, extract the center coordinates (x, y) of each cell. c y c), according to y c Vertical clustering is performed from smallest to largest to form logical rows, according to x c Horizontal clustering is performed from smallest to largest to form logical columns; adjacent center points are considered to be at the same logical level when the distance between them is less than a set threshold δ; after clustering, a unique logical index (row) is assigned to each cell. i ,col i ), and construct a preliminary table matrix structure: In the formula, It is a unique cell number, unique within the block; for cells that span multiple logical rows or columns in space, their structural range is determined by the row and column indices corresponding to the top left and bottom right corners, forming the description information of the merged cells, ensuring the complete expression of complex headers and multi-layered structures; (4) Structural candidate graph representation: Based on logical index relationships, a local structure candidate graph is constructed, with each cell as a node in the graph, and directed edges are established according to spatial adjacency relationships; (5) Metadata output: The module outputs standardized structured data for cross-block splicing and global structure fusion.

[0022] Furthermore, step 4 includes: (1) Global coordinate relocation and spatial mapping: First, project the cell coordinates of all sub-tiles from the local coordinate system to the global coordinate system. For any sub-tile... cell Its global corner coordinates Calculate as follows: In the formula, This indicates the local coordinates of the cell within the current slice. and These are the width and height of the slice in column m and row n, respectively. To avoid splicing errors, the offset is dynamically adjusted based on the overlapping boundary record during the image cutting stage. Sub-pixel level alignment correction is performed on the overlapping area. After sub-pixel level alignment correction, all cells are mapped to a unified coordinate system, which has geometric consistency that enables direct global structural fusion. (2) Global logical row and column normalization and continuity correction: After spatial alignment, the logical rows and columns of cells spanning multiple blocks are uniformly clustered to restore the complete row and column sequence, based on the coordinates of the center point of each cell in the global coordinate system (x, y, y). c y c Automatic clustering is performed according to the following rules: Row clustering rule: If If so, then both are considered to belong to the same logical line; Column clustering rules: If If they are in the same logical column, then they are considered to belong to the same logical column. in =0.3, This indicates the average height of the global cells in pixels. This indicates the average width of all cells in pixels. After clustering is completed, the global row and column indexes are reallocated, and the transition across tile boundaries is automatically smoothed to ensure that the table remains visually and logically continuous and consistent. (3) Boundary overlap detection and redundant cell elimination: Calculate the intersection-union ratio (IUR) of any two cells C1 and C2 in the full-map coordinate system: When IOU > θ and cell C1 and cell C2 have the same global row and column index, they are determined to be duplicate cells. It is necessary to make a comprehensive judgment based on area size and confidence level, automatically retain cells with more complete boundaries or larger areas, and delete redundant items. For cells with a partial overlap ratio of less than 5% but not completely duplicated, calculate the boundary similarity to determine whether edge fusion operation needs to be performed, thereby minimizing boundary cracks and misalignment at cross-block splicing points. (4) Global structure topology restoration and hierarchical consistency: After redundancy elimination, the structural-level splicing stage begins. Based on the global row and column indices, the adjacency relationships between cells are reconstructed, and logical breaks caused by block segmentation are repaired. For any cell A, if there is a cell B adjacent to the global row and column indices and A and B are adjacent, then an edge in the corresponding direction is added to the adjacency list. At the same time, based on the adjacency graph and cell semantic similarity, a graph matching algorithm is executed to structurally merge multi-level nested headers, cross-tab structures, and irregularly merged cells, restoring the row and column hierarchy and merging relationship information, ensuring that the output global table structure is topologically complete, hierarchically clear, and unambiguous.

[0023] Furthermore, step 5 is implemented in five sub-stages: text region refinement, text direction detection, language category prediction, OCR model scheduling and recognition, boundary fine-tuning and confidence estimation, specifically as follows: (1) Text area refinement: Using the cell corner coordinates output from the previous steps as initial input, the cell area is first fine-tuned locally: the table lines, cell fills or background interference within the cell are masked / removed, and local contrast enhancement and local noise reduction are performed to improve the robustness of orientation detection and OCR recognition. (2) Text direction detection: For each refined text sub-image, orientation detection is performed. A lightweight regression network is used first to directly regress the rotation angle (angle) of the text paragraph relative to the horizontal axis. The rotation angle (angle) is a continuous value in degrees, ranging from [-180, 180) or [0, 360). At the same time, a binary classifier is run in parallel to determine the main horizontal / vertical orientation to process vertical text. If a vertical label or rotation angle (angle) is detected that is close to ±90°, it will be processed first according to the vertical recognition process. The angle detection output includes the confidence score (score_angle). (3) Language category prediction: A lightweight language recognizer is used to predict the most likely set of language categories for the normalized text subgraphs, and it is labeled whether mixed scripts are included. The prediction results are used to schedule the corresponding OCR sub-models and guide subsequent regularization. (4) OCR sub-model scheduling and recognition: Dynamically select the appropriate OCR model and inference process based on the rotation angle and language prediction results: If the rotation angle ≠ 0 or the rotation angle ≈ 90°, first perform an affine rotation on the text sub-image to place the text in the desired orientation of the model. During the rotation process, retain the original coordinate mapping for back projection. If the text is determined to be vertical, the dedicated vertical text model is used or the character order of the horizontal text model is adjusted. Based on the Top-K language candidates, the corresponding multilingual OCR sub-models are tried in order of confidence. When the first round of recognition is lower than the confidence threshold, it falls back to the suboptimal candidate or hybrid model. The OCR output includes the recognized text string, the precise bounding box of each word or character, and the model confidence score, while also recording the model identifier and version number used for traceability; (5) Boundary fine-tuning and confidence fusion For the bounding boxes returned by OCR, perform boundary fine-tuning: based on text pixel density and connected component analysis, refine the four corner coordinates of each word / character and project them back to the global coordinate system. For the recognition results from different models or different rotation attempts within the same cell, use confidence weighting and character-level consistency checks to fuse the final text and confidence, and record the overall confidence and character-level confidence distribution of each text segment.

[0024] Furthermore, step 6 specifically involves: extracting the four corner coordinates of each cell region in the full-page table structure diagram generated in step 4 to form the polygon boundary P. i,j Where i is the row number and j is the column number, then for each identified text block B k Calculate the coordinates of the center point of its bounding box (x k ,yk When the center point is located at P i,j Within the geometric range, that is, satisfying the spatial inclusion relation (x k ,y k )∈P i,j When the text subimage is determined to belong to the corresponding cell (i,j), the text content of the text subimage is written into the text field of the cell. If there are multiple text blocks in cell (i,j), they are spliced ​​according to the horizontal and vertical order of the center point coordinates to maintain the logical consistency of reading. After completing the initial mapping, for areas with low confidence levels or overlapping or ambiguous attribution, a dual correction strategy based on contextual semantic consistency and rule verification is adopted to correct and complete abnormal mappings, ensuring the accuracy and completeness of the correspondence between structure and content. Finally, based on the restored row and column hierarchy and merging relationship information, a standardized Excel workbook is built using the openpyxl library. The format information such as cell position, merged area, border style, font attributes, and alignment is restored item by item. The cell position, merged area, border style, font attributes, and alignment are highly consistent with the original PDF table in terms of visual presentation, semantic structure, and editability, and can be directly used for subsequent manual review, data analysis, or automated processing.

[0025] Compared with existing technologies, the intelligent recognition and structure restoration method for large and highly complex tables described in this invention has the following advantages: (1) This invention proposes a high-precision cell detection algorithm, TableVerseNet. TableVerseNet achieves high-precision cell detection and structural representation in complex table scenarios through a four-stage collaborative process of "multi-scale feature fusion, multi-type boundary detection, structural mask segmentation, and Transformer modeling", laying the foundation for subsequent table hierarchy reconstruction and semantic restoration; (2) Content-aware automatic segmentation strategy and position tracking mechanism. For ultra-large table images, this invention provides a content-aware automatic segmentation strategy. This strategy adaptively determines the segmentation boundary based on the inherent layout features of the image, avoiding structural breaks caused by fixed windows or manual segmentation. The segmentation process is accompanied by precise position tracking: each segment records its absolute coordinates and relative position information in the original image during segmentation, forming a positioning index that can be reconstructed, thereby maintaining the consistency of the overall table space while ensuring local processing efficiency; (3) A graph analysis-driven method for constructing cell structure graphs and restoring hierarchy is adopted. Based on cell detection output, this invention treats cells as graph nodes and spatial adjacency as graph edges to construct a table structure graph. Graph theory algorithms are used to restore multi-layered nested headers, cross-tabs, and complex merging relationships. In addition, to address the structural splitting problem caused by cutting, a splicing strategy based on spatial alignment and adaptive elimination of edge redundancy is designed, which can accurately merge cell instances across cut blocks and restore the hierarchy and semantic relationships of the original table during the splicing stage; (4) Achieve a multilingual and orientation-aware OCR recognition system. To address the issues of mixed language text and multi-directional text (horizontal and vertical), this invention proposes a priori text orientation detection step to determine the writing orientation and possible language category of the text in each cell; then, based on the orientation and language information, the corresponding multilingual OCR sub-model is called for recognition, significantly improving the overall recognition reliability in multilingual and multi-directional mixed text tables; (5) Intelligent cell detection algorithm for large and complex tables -- TableVerseNet: It achieves accurate table structure detection and cell location in large and complex table images, so that complex organizational forms including multi-layer nested table headers, irregularly merged cells and cross tables can be stably identified and the original hierarchical relationship can be maintained. Attached Figure Description

[0026] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an undue limitation of the invention. In the drawings: Figure 1 This is a flowchart illustrating the overall processing flow of the intelligent recognition and structure restoration method for large and highly complex tables described in this invention. Detailed Implementation

[0027] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other.

[0028] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0029] like Figure 1As shown, this invention proposes an intelligent recognition and structure restoration method for large and highly complex tables. This method is specifically designed for large, multilingual, and highly complex table images, primarily applied to non-editable electronic documents such as PDFs. It focuses on automatic detection of table content, text recognition, structure restoration, and accurate reconstruction of editable formats such as Excel. This method achieves accurate table structure detection in large and structurally complex table images, enabling the accurate identification and preservation of original hierarchical relationships for complex organizational forms, including multi-layered nested headers, irregularly merged cells, and crosstabs. The method not only designs an adaptive cutting strategy based on layout features and structural clues but also proposes a cell intelligent detection algorithm—TableVerseNet—for large and highly complex tables, achieving high-precision detection of cell boundaries and preliminary reconstruction of structural hierarchy. Furthermore, the method designs a redundancy elimination strategy based on spatial alignment and edge overlap determination, and combines adjacency graph analysis to restore the global hierarchical relationships of multi-layered nested headers, crosstabs, and irregularly merged cells.

[0030] The following is a lengthy, multilingual, and highly complex explanation of this invention: Large table size: refers to a large table size, usually containing more than 100 rows and 100 columns, with a huge amount of data and a rich structure, which is a typical large table structure.

[0031] Multilingual: This refers to tables containing text information in multiple languages ​​simultaneously, such as Chinese, English, and numbers / symbols. Furthermore, multilingual tables often involve mixed text layouts with different orientations, such as horizontal and vertical text, making text recognition and structure reconstruction more complex.

[0032] High complexity: refers to a table structure that is highly complex, containing multiple levels of row headers and table headers, and often involving various complex organizational forms such as merged cells, nested headers, and crosstabs.

[0033] The intelligent recognition and structure restoration method for large and highly complex tables proposed in this invention includes several key steps such as PDF rendering and image processing, content-aware image preprocessing and segmentation, local cell detection and candidate structure construction, cross-block splicing and redundancy elimination, text direction awareness and multilingual OCR recognition, structure content mapping and standardized export.

[0034] Specifically, this invention provides an intelligent recognition and structure restoration method for large and highly complex tables, comprising the following steps: Step 1: PDF Rendering and Image Processing: Receive the PDF document to be processed, and use the PDF rendering module to convert each page into a bitmap image; ensure the preservation of table details (including thin lines, dashed lines, and small annotations) through adaptive resolution and anti-aliasing strategies, laying the foundation for subsequent visual processing; Step 2: Content-aware image preprocessing and segmentation: Preprocessing is performed on the rendered full-page bitmap image (preprocessing includes denoising, grayscale normalization, contrast enhancement, etc.), and adaptive segmentation is performed based on page features and structural clues; each generated sub-block records its absolute position and relative offset information in the original page coordinate system; in order to form an indexable position mapping table, ensuring that local processing does not destroy the global spatial semantics.

[0035] Step 3: Local cell detection and candidate structure construction: For each sub-plot, the TableVerseNet algorithm is called to detect cells, and the corner coordinates, confidence score and other attributes of each cell are output. Based on the attributes output by each cell, a candidate table structure of cells of each sub-plot is constructed at the local scale to describe the relative row and column relationships and preliminary topological connections of the cells, providing a candidate basis for subsequent cross-block merging and hierarchy determination. Step 4: Cross-block stitching and redundancy elimination to restore the global structure: Project the candidate cell images of each sub-block back to the original coordinate system through the position mapping table, and merge the cell instances of cross-blocks using spatial position alignment, boundary overlap judgment and edge redundancy adaptive elimination strategies. Then, use graph analysis methods (graph matching and partitioning algorithms based on adjacency and semantic similarity) to restore the global hierarchical relationships such as multi-level nested headers, cross tables and irregular merged cells, and generate a consistent and unambiguous full-page table structure diagram; Step 5: Text Direction Awareness and Multilingual OCR Recognition: For each defined cell region, text direction detection and language category prediction are performed first to determine whether the text is in horizontal or vertical format and to identify the possible language. Based on the direction and language information, the corresponding multilingual OCR sub-model is dispatched for accurate recognition, and the accurate bounding box and confidence of the recognized text are recorded at the same time. Step 6: Structure, Content Mapping, and Standardized Export: The full-page table structure diagram generated in Step 4 is matched cell-by-cell with the OCR recognition results from Step 5 to establish a one-to-one correspondence between cells and text content. For ambiguous or low-confidence mappings, rule-based validation and contextual consistency strategies are used for correction. Finally, based on the restored row and column hierarchy and merging relationships, cell merging, border styles, and formatting attributes (such as font and alignment) are set according to standard Excel file specifications, and exported as an editable Excel document. This ensures that the output file is highly consistent with the original table in visual presentation and semantic structure and can be directly used for subsequent manual or automated processing. Here, confidence refers to the confidence level of the text recognition result. A text recognition result has a probability corresponding to a given recognition result, which is the confidence level. When the probability of a text recognition result is less than 70%, it is considered to have low confidence.

[0036] In step 1, the PDF document to be processed is received, and a high-precision (500 dots per inch) PDF rendering module is used to convert each page of the PDF document into a bitmap image with a resolution of 9925×7017. The goal is to preserve the minute structure and font details of the table with minimal information loss, including details such as thin lines, dashed lines, and tiny annotations. During the rendering process, adaptive resolution and anti-aliasing strategies are enabled to maintain line continuity and to avoid multiple interpolations and unnecessary resampling as much as possible. The output adopts a lossless or near-lossless format (PNG is preferred, and TIFF is selected for multi-page scenes or when hierarchical information needs to be preserved), laying the foundation for subsequent visual processing.

[0037] Step 2, after completing the PDF image conversion, performs content-aware preprocessing and segmentation on the entire high-resolution image. This aims to divide the table page into several structurally sound and well-defined sub-plots while preserving global spatial semantics. Each sub-plot synchronously records its absolute and relative position in the original coordinate system to support subsequent structure recognition, cell restoration, and full image reconstruction. This step mainly includes four sub-processes: (1) Image preprocessing: Before image segmentation, a series of preprocessing steps are performed on the input image to improve the clarity and separation of table lines and content areas; First, the original image is subjected to grayscale conversion and adaptive normalization to balance the overall brightness distribution and alleviate uneven lighting in scanned or photographed documents; the formula is as follows: In the formula, and Thresholds The pixel ratio of the background to the foreground; for The variance is dimensionless; for The variance is dimensionless; Indicates that T is the minimum value. Subsequently, using The original image is binarized using a threshold to enhance the contrast between the table lines and the background, making the line structure more prominent in grayscale projection. To address noise and fracture lines, a median filter combined with morphological closing operations is used. This effectively removes isolated noise points and repairs virtual fracture edges, ensuring the stability of subsequent projection analysis. The processing formula is as follows: in, The image to be processed is dimensionless. It is a rectangular core, dimensionless; This is an expansion operation, dimensionless; This is a corrosion operation, dimensionless. (2) Content-aware layout projection and outline analysis: Traditional fixed sliding window cutting methods often disrupt the structural continuity of tables. This step introduces an adaptive projection and contour fusion strategy based on content distribution. By calculating the average gray-level projection curves of the image in the row and column directions, the content density distribution characteristics are obtained: low-density areas usually correspond to the row spacing or column spacing of the table. The average gray-level projections of the image in the row and column directions are as follows: in, and These are the image width and height, respectively; Represents the pixel values ​​of an image; The average pixel value along the row direction is dimensionless. The pixel mean along the column direction is dimensionless; the low-density areas correspond to the row spacing or column spacing of the table, calculated from the average projection of rows and columns. Combining the above contour detection to extract the bounding rectangle region of the valid content, the extracted bounding rectangle region is the contour positioning information. Non-table edge noise or title area is excluded to ensure that the cutting line is consistent with the actual table boundary. (3) Adaptive block segmentation strategy design: Based on the projection curve results and contour positioning information in the row and column directions, a dynamic cutting grid is constructed. This step automatically identifies horizontal and vertical low-density bands as candidate segmentation boundaries, generates a preliminary cutting scheme, and fine-tunes it through boundary expansion and alignment mechanisms to prevent characters, borders, or table lines from being truncated. To this end, the edge of each cutting block is extended outward by several pixels to align with the adjacent table edge or page edge, and uniformly corrected into an approximately rectangular area. The adaptive cutting strategy can avoid semantic loss caused by over-segmentation while maintaining the integrity of the internal structure of the cutting block. (4) Mapping of location and index information: To ensure the traceability of subsequent table reconstruction and spatial positioning, each sub-tile is generated with complete spatial metadata, including: Absolute coordinate information: Records the coordinates (x, y) of the top left corner of the sub-plot in the original image and its width and height (w, h) to ensure that any sub-plot can be restored to the full image coordinate system; Relative index information: Based on the cutting grid structure, assign a relative position index (i,j) to each tile and identify its row and column position in the global structure; Adjacency mapping: Constructs the topological relationships between sub-tiles and records the top, bottom, left, and right adjacency information of each tile for spatial stitching during reconstruction.

[0038] Unique Identifier: Generate a globally unique ID (such as tile_0001, tile_0002) for each sub-tile to ensure consistency and traceability during data flow.

[0039] Step 3, the core objective of which is to accurately detect the cell boundaries of the table within each preprocessed and segmented sub-tile through a four-stage collaborative process of "multi-scale feature fusion, multi-type boundary detection, structural mask segmentation, and Transformer modeling," using a deep vision model. This process also establishes a structured candidate topology map within a local area, providing foundational information for subsequent cross-tile structure fusion and hierarchy determination. The entire process comprises five sub-steps, as follows: (1) Cell detection and boundary extraction: Each sub-plot is fed as input into TableVerseNet, a deep learning-based network for recognizing table structures. TableVerseNet consists of four core sub-modules: First, the FusionBackbone feature extractor (multi-scale feature fusion): This feature extractor is based on the ResNet (Residual Network, used to extract deep semantic features) network and the FPN (Feature Pyramid Network, used to fuse features at different scales) network structure to achieve multi-scale feature aggregation, and introduces Deformable Conv2D to enhance the model's adaptability to different shaped table lines. At the same time, it improves the attention distribution of intersection regions (such as the intersection of table header lines) through a lightweight Transformer (a sequence modeling structure based on self-attention mechanism, used to capture global dependencies). Second, the bounding box detection branch (multi-type boundary detection): The bounding box detection branch borrows the differentiable binarization mechanism of DBNet (Differentiable Binarization Network), which not only outputs the overall table box (Normal Box), but also refines and generates inner cell boxes (Inner Box) and weak bounding boxes (WeakBox) to distinguish between valid cell boundaries and noisy edges. Third, the structure mask branch (structure mask segmentation): The structure mask branch uses Mask R-CNN to achieve pixel-level structure segmentation. It generates Row Mask and Column Mask through projection constraints in the row and column directions, which are used to assist in the hierarchical modeling of merged cells and nested headers. Fourth, Cell structural modeler (Transformer modeling): The Cell structural modeler uses a DETR-style Transformer decoder to decode each query vector into a cell, directly outputting the coordinates of the four corner points of its bounding box and the confidence score, providing a geometric basis for subsequent structural construction.

[0040] (2) Cell deduplication and optimization: After the model outputs preliminary detection results, optimization and filtering are performed on duplicate detections, abnormal small boxes, and non-standard rectangles. First, the area A of each candidate cell is calculated, which is the number of pixels within the candidate cell area. When A < ε, it is considered a noise area and is removed. ε (number of pixels) is an empirical threshold. Second, the intersection-union ratio (IOU) is calculated for any two cells. The calculation formula is as follows: In the formula, express and The number of pixels in the intersection of the two regions; express and The number of pixels after merging the two regions; This represents the intersection, union, and ratio of numbers; it is a floating-point number and dimensionless. If IOU > θ (θ is generally set to 0.7), it is judged as a duplicate detection and boundary merging is performed. For the detected non-standard quadrilateral cells Q={x1,y1,x2,y2,x3,y3,x4,y4}, the approximate rectangular boundary is refitted by the minimum bounding rectangle algorithm to ensure the geometric consistency of subsequent row and column clustering. All corner coordinates are uniformly retained to two decimal places in the final output to improve the accuracy of subsequent structure splicing. (3) Logical row and column clustering and index generation After obtaining the optimized set of cells, extract the center coordinates (x, y) of each cell. c y c ), according to y c Vertical clustering is performed from smallest to largest to form logical rows, according to x c Horizontal clustering is performed from smallest to largest to form logical columns. Adjacent center points are considered to be at the same logical level if the distance between them is less than a set threshold δ. After clustering, a unique logical index (row) is assigned to each cell. i ,col i ), and construct a preliminary table matrix structure: In the formula, It is a unique cell number, unique within the block; for cells that span multiple logical rows or columns in space, such as cells with the top left corner at (row1, col1) and the bottom right corner at (row2, col2), their structural range is determined by the row and column indices corresponding to the top left and bottom right corners, forming the description information of merged cells (merged_cells), ensuring the complete expression of complex headers and multi-level structures; (4) Structural candidate graph representation: Based on logical indexing relationships, a candidate graph of local structures is constructed, treating each cell as a node in the graph and establishing directed edges according to spatial adjacency relationships. For example, if cell A has cell B to its right, a "right" edge is established between A and B; if cell A has cell C below it, a "down" edge is established between A and C. Adjacency relationships are stored in an adjacency list as a graph structure. "neighbors": { "0": {"right": 1, "down": 5}, / / 0, 1, and 5 represent the unique cell numbers. Cell 0 is next to cell 1 and next to cell 5. "1": {"left": 0} } This structure can fully describe the local topological relationships, providing a connectivity reference for subsequent cross-block merging and hierarchical reconstruction.

[0041] (5) Metadata output: Finally, standardized structured data is output for cross-block stitching and global structure fusion. The output format is as follows: { "cut_id": "tile_0003", / / Unique ID of the sub-tile "cells": [ {"cell_id": 0, "position": [...], "row": 0, "col": 0}, {"cell_id": 1, "position": [...], "row": 0, "col": 1}, / / "cell_id", the unique cell ID, unique within this sub-block; "position", the BoundingBox value output by TableVerseNet, which is the coordinates of the four corner points of the cell, typically [x1,y1,x2,y2,x3,y3,x4,y4], arranged clockwise; "row", the logical row number (starting from 0) of the cell within this sub-block; "col", the logical column number (starting from 0) of the cell within this sub-block. ... ], "structure": { "rows": 20, "cols": 30, "merged_cells": [{"start": [1,1], "end": [1,2]}], "neighbors": { "0": {"right": 1, "down": 5}, ... } } } Here, "cut_id" is used to uniquely identify the sub-tile. The "cells" array contains information such as the cell number, position coordinates (usually the four corner points arranged clockwise), and logical row and column numbers for each cell. In the "structure" object, rows and cols represent the total number of logical rows and columns detected within the current tile; merged_cells represents the start and end positions of the merged cells, used to restore complex structures; and neighbors records the spatial adjacency relationships of each cell in the form of an adjacency list.

[0043] Step 4 aims to uniformly map the candidate local table structures from multiple sub-plots to the global coordinate space, and achieve seamless splicing and global hierarchical reconstruction of the entire page table structure through spatial alignment, boundary overlap elimination, and graph structure repair. The entire process includes four key sub-steps to ensure that topological consistency and structural integrity are maintained even in complex table scenarios.

[0044] (1) Global coordinate relocation and spatial mapping: First, project the cell coordinates of all sub-tiles from the local coordinate system to the global coordinate system. For any sub-tile... cell Its global corner coordinates Calculate as follows: In the formula, This indicates the local coordinates of the cell within the current slice. and These are the width and height of the slice in column m and row n, respectively. To avoid splicing errors, the offset is dynamically adjusted based on the overlapping boundary record during the image cutting stage. Sub-pixel level alignment correction is performed on the overlapping area. After sub-pixel level alignment correction, all cells are mapped to a unified coordinate system, which has geometric consistency that enables direct global structural fusion. (2) Global logical row and column normalization and continuity correction: After spatial alignment, the logical rows and columns of cells spanning multiple blocks are uniformly clustered to restore the complete row and column sequence, based on the coordinates of the center point of each cell in the global coordinate system (x, y, y). c y c Automatic clustering is performed according to the following rules: Row clustering rule: If If so, then both are considered to belong to the same logical line; Column clustering rules: If If they are in the same logical column, then they are considered to belong to the same logical column. in =0.3, This represents the average height of all cells, i.e. This indicates the average height of the global cells in pixels. This represents the average height of all cells, i.e. This indicates the average width of all cells in pixels. After clustering is completed, the global row and column indexes (global_row, global_col) are reallocated, and a smooth transition is automatically performed across tile boundaries to ensure that the table remains visually and logically continuous and consistent. (3) Boundary overlap detection and redundant cell elimination: Since overlapping areas between segments may lead to duplicate detections, redundant matching and removal are necessary. The intersection-union ratio (IOU) of any two cells C1 and C2 in the full-image coordinate system is calculated as follows: When IOU > θ (θ is 0.7) and cell C1 and cell C2 have the same global row and column index, they are determined to be duplicate cells. The algorithm will make a comprehensive judgment based on area size and confidence level, automatically retaining cells with more complete boundaries or larger areas and deleting redundant items. For cells with a partial overlap ratio of less than 5% but not completely duplicated, the boundary similarity score will be calculated to determine whether edge fusion operation needs to be performed, thereby minimizing boundary cracks and misalignments at cross-block splicing points. (4) Global structural topology repair and hierarchical consistency: After redundancy elimination, the structural splicing stage begins. Based on the global row and column indices, the adjacency relationships (top, bottom, left, right) between cells are reconstructed, and logical breaks caused by block segmentation are repaired. For any cell A, if a cell B exists at an adjacent position in both the global row and column indices and their boundaries are adjacent, an edge in the corresponding direction is added to the adjacency list, such as "right":B_id or "down":C_id. Simultaneously, the algorithm performs graph matching based on the adjacency graph and cell semantic similarity to structurally merge multi-level nested headers, cross-tab structures, and irregularly merged cells, ensuring that the output global table structure is topologically complete, hierarchically clear, and unambiguous. After structural splicing, the final output structure should have the following fields: { "table_id": "page_001_table_1", "rows": 158, "cols": 122, "cells": [ { "cell_id": "cell_00001", / / Unique cell ID, globally unique "position": [x1,y1, x2,y2, x3,y3, x4,y4], / / Global coordinates "row": 1, "col": 2, "text":None / / In the output of step 4, this field has an empty value, which will be filled in step 6. "merged": false, "neighbors": {"left": "cell_00002", "right": null} }, ... ], "merged_cells": [ {"start": [3,2], "end": [3,4]}, / / Spanning 3 columns horizontally {"start": [10,0], "end": [13,0]}, / / Spanning 4 rows vertically ] ... } Step 5, after the global table structure is determined, performs orientation awareness, language prediction, and accurate text recognition on each cell region, outputting a structured result with location, language, and confidence information. This step is implemented in five sub-stages: text region refinement, text orientation detection, language category prediction, OCR model scheduling and recognition, boundary fine-tuning, and confidence estimation, specifically: (1) Text area refinement: After restoring the table structure according to the aforementioned steps, each cell in the table will have the coordinates of its four corners. Using the cell corner coordinates output in the aforementioned steps as the initial input, the cell area is first finely refined: possible table lines, cell fills or background interference within the cell are masked / removed (while retaining text pixels), and local contrast enhancement and local noise reduction are performed to improve the robustness of orientation detection and OCR recognition. (2) Text direction detection: For each refined text sub-image, orientation detection is performed. A lightweight regression network (hereinafter referred to as OCRAngleNet) is used first to directly regress the rotation angle angle of the text paragraph relative to the horizontal axis. The rotation angle angle is a continuous value in degrees, ranging from [-180, 180) or [0, 360). At the same time, a binary classifier is run in parallel to determine the main horizontal / vertical orientation to process vertical text (such as vertical Chinese text, vertical English text). If a vertical label or rotation angle angle is detected that is close to ±90°, it will be processed first according to the vertical recognition process. The angle detection output includes the confidence score_angle. (3) Language category prediction: The most likely set of language categories (Top-K, including confidence) is predicted using a lightweight language recognizer (LangDetectNet) on the normalized text tiles, and it is labeled whether it contains mixed scripts (e.g., mixed Chinese and English, numbers + symbols). The prediction results are used to schedule the corresponding OCR sub-model and guide subsequent regularization. (4) OCR sub-model scheduling and recognition (by direction and language): Dynamically select the appropriate OCR model and inference process based on the rotation angle and language prediction results: If the rotation angle angle≠0 or the rotation angle angle≈90°, first perform an affine rotation on the block to set the text to the desired orientation of the model. During the rotation process, retain the original coordinate mapping for back projection. If the text is determined to be vertical, the dedicated vertical text model is used or the character order of the horizontal text model is adjusted. Based on the Top-K language candidates, try the corresponding multilingual OCR sub-models (such as Chinese / English / Japanese / etc.) in order of confidence. When the first round of recognition is lower than the confidence threshold, it falls back to the suboptimal candidate or hybrid model. The OCR output includes the recognized text string (text), the precise bounding box of each word or character (which can be a list of character-level or word-level bboxes), and the model confidence (confidence_text). It also records the model identifier and version number used for traceability. (5) Boundary fine-tuning and confidence fusion For the bounding boxes returned by OCR, perform boundary refinement: based on text pixel density and connected component analysis, refine the four corner coordinates of each word / character and project them back to the global coordinate system (maintaining a consistent mapping with the cell coordinates in step 4). For the recognition results from different models or different rotation attempts within the same cell, use confidence weighting and character-level consistency checks to fuse the final text and confidence, and record the overall confidence and character-level confidence distribution of each text segment.

[0045] { "ocr_results": [ { "cell_id": "cell_00001", / / Unique cell ID derived from step 4 "text": "Project Start Date", / / Text content recognized by OCR "language": ["en"], / / Predict language "direction": "horizontal", / / Text direction: horizontal / vertical "angle": 0.0, / / Rotation angle (degrees) "confidence": 0.98, / / Overall confidence level of the text "bboxes": [ / / Character or word-level bounding boxes (global coordinates)] {"text": "Project", "bbox": [x1,y1,x2,y2,x3,y3,x4,y4], "confidence": 0.99}, {"text": "Start", "bbox": [x1,y1,x2,y2,x3,y3,x4,y4], "confidence": 0.97}, {"text": "Date", "bbox": [x1,y1,x2,y2,x3,y3,x4,y4], "confidence": 0.96} ], "model": {"name": "XXX", "version":"2025-09"}, "quality": "high" / / Overall quality label }, ... ] } Step 6 aims to accurately integrate the table structure information with the text recognition results and output an editable Excel document in a standard format. First, the table structure diagram generated in Step 4 is compared and mapped cell-by-cell with the OCR recognition results obtained in Step 5. Specifically, the four corner coordinates of each cell in the full-page table structure diagram generated in Step 4 are extracted to form the polygon boundary P. i,j Where i is the row number and j is the column number, then for each identified text block B k Calculate the coordinates of the center point of its bounding box (x k ,y k When the center point is located at P i,j Within the geometric range, that is, satisfying the spatial inclusion relation (x k ,y k )∈P i,j When the text subimage is determined to belong to the corresponding cell (i,j), the text content of the text subimage is written into the text field of the cell. If there are multiple text blocks in cell (i,j), they are spliced ​​according to the horizontal and vertical order of the center point coordinates to maintain the logical consistency of reading. After completing the initial mapping, for areas with low confidence levels or overlapping or ambiguous attribution, a dual correction strategy based on contextual semantic consistency and rule verification is adopted to correct and complete abnormal mappings, ensuring the accuracy and completeness of the correspondence between structure and content. Finally, based on the restored row and column hierarchy and merging relationship information, a standardized Excel workbook is built using the openpyxl library. The format information such as cell position, merged area, border style, font attributes, and alignment is restored item by item. The cell position, merged area, border style, font attributes, and alignment are highly consistent with the original PDF table in terms of visual presentation, semantic structure, and editability, and can be directly used for subsequent manual review, data analysis, or automated processing.

[0046] The method described in this invention can be deployed on Linux or Windows operating systems. Recommended configurations include: CPU: Processor with 8 or more cores; GPU: NVIDIA RTX 3090 (for model acceleration); Memory: 32GB or more.

[0047] The Linux or Windows operating system can run the intelligent recognition and structure restoration method for large and highly complex tables described in this invention.

[0048] It should be noted that there are other alternative solutions for step 1 in this invention, such as using third-party PDF toolkits (such as Poppler, MuPDF, Adobe SDK) for multi-channel rendering.

[0049] For step 3, if there are other alternatives, a table semantic segmentation model (such as TableNet or CascadeTabNet) can be used to output cell masks instead of border lines.

[0050] If there are alternative solutions for step 5, a multi-engine OCR fusion mechanism (such as Tesseract + PaddleOCR) can be used to improve the accuracy of multilingual recognition.

[0051] This invention proposes an intelligent recognition and structure restoration method for large and highly complex tables, providing a systematic solution for cause-effect graph table and content recognition in offshore platform control systems. It achieves high-precision recognition and structure restoration of large, multilingual, and highly complex tables, thus meeting the needs of editable output and industrial applications. It effectively solves the following problems: (1) How to achieve accurate table structure detection and cell positioning in large and complex table images, so that complex organizational forms including multi-layer nested table headers, irregularly merged cells and cross tables can be stably identified and the original hierarchical relationship can be maintained; (2) How to construct a multilingual text recognition mechanism with high generalization ability so that it can recognize Chinese, English and other multilingual characters in a unified and accurate manner, while being compatible with different writing directions (such as horizontal and vertical), diverse fonts and special symbols, so as to minimize the misrecognition rate and character loss in mixed language scenarios. (3) How to accurately restore the semantic structure of the table based on the table detection results, including the hierarchical relationship of multi-level table headers, the coverage of merged cells and the row and column mapping of cross tables, and achieve accurate binding with the corresponding text, so that the restored structure can be output in a standard Excel format without ambiguity, ensuring the editability and visual presentation consistency of the file.

[0052] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for intelligent recognition and structure restoration of large and highly complex tables, characterized in that, Includes the following steps: Step 1: PDF Rendering and Image Processing: Receive the PDF document to be processed, and use the PDF rendering module to convert each page into a bitmap image; ensure the preservation of table details through adaptive resolution and anti-aliasing strategies, laying the foundation for subsequent visual processing; Step 2: Content-aware image preprocessing and segmentation: Preprocess the rendered full-page bitmap image and perform adaptive segmentation based on layout features and structural clues; record the absolute position and relative offset information of each generated sub-block in the original page coordinate system to form an indexable position mapping table to ensure that local processing does not destroy the global spatial semantics; Step 3: Local cell detection and candidate structure construction: For each sub-plot, the TableVerseNet algorithm is called to detect cells, and the corner coordinates and confidence attributes of each cell are output; Based on the attributes output by each cell, a candidate table structure of each sub-plot cell is constructed at a local scale to describe the relative row and column relationships and preliminary topological connections of the cells, providing a candidate basis for subsequent cross-block merging and hierarchy determination. Step 4: Cross-block stitching and redundancy elimination: Project the candidate cell images of each sub-block back to the original coordinate system through the position mapping table, and merge the cell instances of cross-blocks using spatial position alignment, boundary overlap judgment and edge redundancy adaptive elimination strategies. Then, use graph analysis methods to restore the global hierarchical relationships such as multi-level nested headers, cross tables and irregularly merged cells to generate a consistent and unambiguous full-page table structure diagram. Step 5: Text Direction Awareness and Multilingual OCR Recognition: For each defined cell region, text direction detection and language category prediction are performed first to determine whether the text is in horizontal or vertical format and to identify the possible language. Based on the direction and language information, the corresponding multilingual OCR sub-model is dispatched for accurate recognition, and the accurate bounding box and confidence of the recognized text are recorded at the same time. Step 6: Structure, Content Mapping, and Standardized Export: The full-page table structure diagram generated in Step 4 is matched cell by cell with the OCR recognition results in Step 5 to establish a one-to-one correspondence between cells and text content. For mappings with ambiguity or low confidence, rule-based validation and contextual consistency strategies are used for correction. Finally, based on the restored row and column hierarchy and merging relationships, cell merging, border styles, and formatting attributes are set according to standard Excel file specifications, and exported as an editable Excel document. This ensures that the output file is highly consistent with the original table in terms of visual presentation and semantic structure and can be directly used for subsequent manual or automated processing.

2. The intelligent recognition and structure restoration method for large and highly complex tables according to claim 1, characterized in that: In step 1, a PDF rendering module with 500 dots per inch is used to convert each page of the PDF document into a bitmap image with a resolution of 9925×7017. The goal is to preserve the minute structure and font details of the table with minimal information loss, including details such as thin lines, dotted lines, and tiny annotations. During the rendering process, adaptive resolution and anti-aliasing strategies are enabled to maintain line continuity. The output adopts a lossless or near-lossless format, laying the foundation for subsequent visual processing.

3. The intelligent recognition and structure restoration method for large and highly complex tables according to claim 1, characterized in that, Step 2 includes four sub-processes: (1) Image preprocessing: Before image segmentation, a series of preprocessing steps are performed on the input image to improve the clarity and separation of table lines and content areas; First, the original image is subjected to grayscale conversion and adaptive normalization to balance the overall brightness distribution and alleviate uneven lighting in scanned or photographed documents; the formula is as follows: In the formula, and Thresholds The pixel ratio of the background to the foreground; for The variance is dimensionless; for The variance is dimensionless; Indicates that T is the minimum value. Subsequently, using The original image is binarized using a threshold to enhance the contrast between the table lines and the background, making the line structure more prominent in grayscale projection. To address noise and fracture lines, a median filter combined with morphological closing operations is used. This effectively removes isolated noise points and repairs virtual fracture edges, ensuring the stability of subsequent projection analysis. The processing formula is as follows: in, The image to be processed is dimensionless. It is a rectangular core, dimensionless; This is an expansion operation, dimensionless; This is a corrosion operation, dimensionless. (2) Content-aware layout projection and outline analysis: An adaptive projection and contour fusion strategy based on content distribution is introduced. By calculating the gray-level average projection curves of the image in the row and column directions, the content density distribution features are obtained. The gray-level average projections of the image in the row and column directions are as follows: in, and These are the image width and height, respectively; Represents the pixel values ​​of an image; The average pixel value along the row direction is dimensionless. The pixel mean along the column direction is dimensionless; the low-density areas correspond to the row spacing or column spacing of the table, calculated from the average projection of rows and columns. Combining the above contour detection to extract the bounding rectangle region of the valid content, the extracted bounding rectangle region is the contour positioning information. Non-table edge noise or title area is excluded to ensure that the cutting line is consistent with the actual table boundary. (3) Adaptive block segmentation strategy design: Based on the projection curve results and contour positioning information in the row and column directions, a dynamic cutting grid is constructed. This step automatically identifies horizontal and vertical low-density bands as candidate segmentation boundaries, generates a preliminary cutting scheme, and fine-tunes it through boundary expansion and alignment mechanisms to prevent characters, borders, or table lines from being truncated. To this end, the edge of each cutting block is extended outward by several pixels to align with the adjacent table edge or page edge, and uniformly corrected into an approximately rectangular area. The adaptive cutting strategy can avoid semantic loss caused by over-segmentation while maintaining the integrity of the internal structure of the cutting block. (4) Mapping of location and index information: To ensure the traceability of subsequent table reconstruction and spatial positioning, each sub-tile is generated with complete spatial metadata, including: Absolute coordinate information: Records the coordinates (x, y) of the top left corner of the sub-plot in the original image and its width and height (w, h) to ensure that any sub-plot can be restored to the full image coordinate system; Relative index information: Based on the cutting grid structure, assign a relative position index (i,j) to each sub-tile and identify its row and column position in the global structure; Adjacency mapping: Constructing topological relationships between sub-tiles and recording the top, bottom, left, and right adjacency information of each sub-tile for spatial stitching during reconstruction; Unique Identifier: A globally unique ID is generated for each sub-plot to ensure consistency and traceability during data flow.

4. The intelligent recognition and structure restoration method for large and highly complex tables according to claim 1, characterized in that, Step 3 includes: (1) Cell detection and boundary extraction: Each sub-plot is fed as input into TableVerseNet, a deep learning-based network for recognizing table structures. TableVerseNet consists of four core sub-modules: First, the FusionBackbone feature extractor: This feature extractor achieves multi-scale feature aggregation based on the ResNet and FPN network structures, and introduces deformable convolution to enhance the model's adaptability to different shaped table lines. At the same time, it improves the attention distribution in the intersection region through a lightweight Transformer module. Second, the bounding box detection branch: The bounding box detection branch borrows DBNet's differentiable binarization mechanism, which not only outputs the overall table frame, but also refines and generates inner cell frames and weak bounding boxes to distinguish between valid cell boundaries and noisy edges. Third, the structure mask branch: The structure mask branch uses Mask R-CNN to achieve pixel-level structure segmentation. It generates Row Mask and Column Mask through projection constraints in the row and column directions, which are used to assist in the hierarchical modeling of merged cells and nested headers. Fourth, Cell structure modeler: The Cell structure modeler uses a DETR-style Transformer decoder to decode each query vector into a cell, directly outputting the coordinates of the four corner points of its Bounding Box and the confidence score, providing a geometric basis for subsequent structure construction; (2) Cell deduplication and optimization: After the model outputs preliminary detection results, optimization and filtering are performed on duplicate detections, abnormal small boxes, and non-standard rectangles. First, the number of pixels A in each candidate cell is calculated. When A < ε, it is considered a noise region and is removed. ε is an empirical threshold. Second, the intersection-union ratio (IUU) is calculated for any two cells. The calculation formula is as follows: In the formula, express and The number of pixels in the intersection of the two regions; express and The number of pixels after merging the two regions; This represents the intersection, union, and ratio of numbers; it is a floating-point number and dimensionless. If IOU>θ, it is determined to be a duplicate detection and boundary merging is performed. For the detected non-standard quadrilateral cells Q={x1,y1,x2,y2,x3,y3,x4,y4}, the approximate rectangular boundary is refitted by the minimum bounding rectangle algorithm to ensure the geometric consistency of subsequent row and column clustering. All corner coordinates are uniformly retained to two decimal places in the final output to improve the accuracy of subsequent structure splicing. (3) Logical row and column clustering and index generation After obtaining the optimized set of cells, extract the center coordinates (x, y) of each cell. c y c ), according to y c Vertical clustering is performed from smallest to largest to form logical rows, according to x c Horizontal clustering is performed from smallest to largest to form logical columns; adjacent center points are considered to be at the same logical level when the distance between them is less than a set threshold δ; after clustering, a unique logical index (row) is assigned to each cell. i ,col i ), and construct a preliminary table matrix structure: In the formula, It is a unique cell number, unique within the block; for cells that span multiple logical rows or columns in space, their structural range is determined by the row and column indices corresponding to the top left and bottom right corners, forming the description information of the merged cells, ensuring the complete expression of complex headers and multi-layered structures; (4) Structural candidate graph representation: Based on logical index relationships, a local structure candidate graph is constructed, with each cell as a node in the graph, and directed edges are established according to spatial adjacency relationships; (5) Metadata output: The module outputs standardized structured data for cross-block splicing and global structure fusion.

5. The intelligent recognition and structure restoration method for large and highly complex tables according to claim 1, characterized in that, Step 4 includes: (1) Global coordinate relocation and spatial mapping: First, project the cell coordinates of all sub-tiles from the local coordinate system to the global coordinate system. For any sub-tile... cell Its global corner coordinates Calculate as follows: In the formula, This indicates the local coordinates of the cell within the current slice. and These are the width and height of the slice in column m and row n, respectively. To avoid splicing errors, the offset is dynamically adjusted based on the overlapping boundary record during the image cutting stage. Sub-pixel level alignment correction is performed on the overlapping area. After sub-pixel level alignment correction, all cells are mapped to a unified coordinate system, which has geometric consistency that enables direct global structural fusion. (2) Global logical row and column normalization and continuity correction: After spatial alignment, the logical rows and columns of cells spanning multiple blocks are uniformly clustered to restore the complete row and column sequence, based on the coordinates of the center point of each cell in the global coordinate system (x, y, y). c y c Automatic clustering is performed according to the following rules: Row clustering rule: If If so, then both are considered to belong to the same logical line; Column clustering rules: If If so, then they are considered to belong to the same logical column; in =0.3, This indicates the average height of the global cells in pixels. This indicates the average width of all cells in pixels. After clustering is completed, the global row and column indexes are reallocated, and the transition across tile boundaries is automatically smoothed to ensure that the table remains visually and logically continuous and consistent. (3) Boundary overlap detection and redundant cell elimination: Calculate the intersection-union ratio (IUR) of any two cells C1 and C2 in the full-map coordinate system: When IOU > θ and cell C1 and cell C2 have the same global row and column index, they are determined to be duplicate cells. It is necessary to make a comprehensive judgment based on area size and confidence level, automatically retain cells with more complete boundaries or larger areas, and delete redundant items. For cells with a partial overlap ratio of less than 5% but not completely duplicated, calculate the boundary similarity to determine whether edge fusion operation needs to be performed, thereby minimizing boundary cracks and misalignment at cross-block splicing points. (4) Global structure topology restoration and hierarchical consistency: After redundancy elimination, the structural-level splicing stage begins. Based on the global row and column indices, the adjacency relationships between cells are reconstructed, and logical breaks caused by block segmentation are repaired. For any cell A, if there is a cell B adjacent to the global row and column indices and A and B are adjacent, then an edge in the corresponding direction is added to the adjacency list. At the same time, based on the adjacency graph and cell semantic similarity, a graph matching algorithm is executed to structurally merge multi-level nested headers, cross-tab structures, and irregularly merged cells, restoring the row and column hierarchy and merging relationship information, ensuring that the output global table structure is topologically complete, hierarchically clear, and unambiguous.

6. The intelligent recognition and structure restoration method for large and highly complex tables according to claim 1, characterized in that: Step 5 is implemented in five sub-stages: text region refinement, text direction detection, language category prediction, OCR model scheduling and recognition, boundary fine-tuning and confidence estimation, specifically: (1) Text area refinement: Using the cell corner coordinates output from the previous steps as initial input, the cell area is first fine-tuned locally: the table lines, cell fills or background interference within the cell are masked / removed, and local contrast enhancement and local noise reduction are performed to improve the robustness of orientation detection and OCR recognition. (2) Text direction detection: For each refined text sub-image, orientation detection is performed. A lightweight regression network is used first to directly regress the rotation angle (angle) of the text paragraph relative to the horizontal axis. The rotation angle (angle) is a continuous value in degrees, ranging from [-180, 180) or [0, 360). At the same time, a binary classifier is run in parallel to determine the main horizontal / vertical orientation to process vertical text. If a vertical label or rotation angle (angle) is detected that is close to ±90°, it will be processed first according to the vertical recognition process. The angle detection output includes the confidence score (score_angle). (3) Language category prediction: A lightweight language recognizer is used to predict the most likely set of language categories for the normalized text subgraphs, and it is labeled whether mixed scripts are included. The prediction results are used to schedule the corresponding OCR sub-models and guide subsequent regularization. (4) OCR sub-model scheduling and recognition: Dynamically select the appropriate OCR model and inference process based on the rotation angle and language prediction results: If the rotation angle ≠ 0 or the rotation angle ≈ 90°, first perform an affine rotation on the text sub-image to place the text in the desired orientation of the model. During the rotation process, retain the original coordinate mapping for back projection. If the text is determined to be vertical, the dedicated vertical text model is used or the character order of the horizontal text model is adjusted. Based on the Top-K language candidates, the corresponding multilingual OCR sub-models are tried in order of confidence. When the first round of recognition is lower than the confidence threshold, it falls back to the suboptimal candidate or hybrid model. The OCR output includes the recognized text string, the precise bounding box of each word or character, and the model confidence score, while also recording the model identifier and version number used for traceability; (5) Boundary fine-tuning and confidence fusion For the bounding boxes returned by OCR, perform boundary fine-tuning: based on text pixel density and connected component analysis, refine the four corner coordinates of each word / character and project them back to the global coordinate system. For the recognition results from different models or different rotation attempts within the same cell, use confidence weighting and character-level consistency checks to fuse the final text and confidence, and record the overall confidence and character-level confidence distribution of each text segment.

7. The intelligent recognition and structure restoration method for large and highly complex tables according to claim 5, characterized in that, Step 6 specifically involves: extracting the four corner coordinates of each cell region in the full-page table structure diagram generated in step 4 to form the polygon boundary P. i,j Where i is the row number and j is the column number, then for each identified text block B k Calculate the coordinates of the center point of its bounding box (x k ,y k When the center point is located at P i,j Within the geometric range, that is, satisfying the spatial inclusion relation (x k ,y k )∈P i,j When the text subimage is determined to belong to the corresponding cell (i,j), the text content of the text subimage is written into the text field of the cell. If there are multiple text blocks in cell (i,j), they are spliced ​​according to the horizontal and vertical order of the center point coordinates to maintain the logical consistency of reading. After completing the initial mapping, for areas with low confidence or overlapping or ambiguous attribution, a dual correction strategy based on contextual semantic consistency and rule verification is adopted to correct and complete abnormal mappings, ensuring the accuracy and completeness of the correspondence between structure and content. Finally, based on the restored row and column hierarchy and merging relationship information, a standardized Excel workbook is constructed using the openpyxl library. The format information such as cell position, merged area, border style, font attributes, and alignment is restored item by item. The cell position, merged area, border style, font attributes, and alignment are highly consistent with the original PDF table in terms of visual presentation, semantic structure, and editability, and can be directly used for subsequent manual review, data analysis, or automated processing.

Images