A document layout analysis and reconstruction method based on vector topology and conflict arbitration

Abstract

The application discloses a document layout analysis and reconstruction method based on vector topology and conflict arbitration, relates to the technical field of document image processing, and particularly relates to a method for accurately identifying, classifying and structurally reconstructing layout elements of unstructured electronic documents containing complex vector charts. A page vector topology map is constructed by using bottom layer vector instructions and a connected topology structure; on the basis, multi-dimensional geometric features, statistical features and text mode features are introduced to jointly participate in conflict arbitration between tables and complex graphics; then, the arbitration result is used to apply a forced spatial exclusion constraint to text extraction, and semantic classification and rearrangement are performed in combination with style benchmarks and spatial proximity relationships. Therefore, the document layout analysis and reconstruction method improves the distinguishing ability of tables and complex graphics, improves the attribution judgment accuracy of in-graph words and main text, and realizes adaptive identification of title and main text styles under different layout styles.

Description

Technical Field

[0001] This invention relates to the field of document image processing technology, specifically to a document layout parsing and reconstruction method based on vector topology and conflict arbitration. Background Technology

[0002] With the widespread use of PDF and other formatted documents in technical reports, standards, papers, and manuals, the need to convert these documents into streaming formats such as Markdown, HTML, or Word to support retrieval, re-editing, and structured storage is increasing. Existing document layout parsing technologies still face the following technical challenges when processing technical documents containing complex vector graphics:

[0003] 1. Confusing classification of layout elements:

[0004] Many existing parsing tools rely primarily on bounding box detection and grid line extraction to identify table structures. However, when documents contain complex vector graphics such as flowcharts, system architecture diagrams, and UML diagrams, these graphics also contain a large number of bounding boxes and connecting lines. Existing methods are prone to misinterpreting them as "tables," leading to incorrect splitting based on row and column structures and thus disrupting the original semantic structure of the graphics.

[0005] 2. Errors in the order and attribution of text extracted from areas with mixed text and images:

[0006] In areas where text and graphics are mixed, if the boundaries of the graphics cannot be accurately defined, the parser will often mix the explanatory text inside the graphics into the main text paragraphs according to the scanning order from left to right and from top to bottom, thus disrupting the semantic coherence and readability of the document.

[0007] 3. Limitations of static style rules:

[0008] Traditional layout parsing methods typically use fixed font size thresholds or predefined templates to identify headings and body text. These static rules are poorly adapted to document scenarios involving cross-language, cross-typography styles, and mixed fonts, resulting in unstable heading hierarchy recognition and inaccurate body text range identification. Summary of the Invention

[0009] The technical problems to be solved by this invention are chaotic layout elements, easy errors in the order and attribution of text extraction in mixed text and image areas, and limitations of static style rules. The purpose is to provide a document layout parsing and reconstruction method based on vector topology and conflict arbitration. This method uses the document's underlying vector drawing instructions for topological analysis and combines multi-source features for conflict arbitration, thereby improving the ability to distinguish between tables and complex graphics, improving the accuracy of determining the attribution of text within images and body text, and achieving adaptive recognition of title and body text styles under different layout styles.

[0010] This invention is achieved through the following technical solution:

[0011] The first aspect of this invention provides a document layout parsing and reconstruction method based on vector topology and conflict arbitration, comprising the following specific steps:

[0012] Define the base text style and heading styles for the vector document;

[0013] Based on the defined text base style and heading styles at all levels, the underlying drawing instructions of the vector document are parsed, the vector document is pre-classified, and a spatial index tree is built for each pre-classified object.

[0014] Based on the spatial index tree, multi-scale iterative clustering is performed to generate preliminary vector topological descriptors;

[0015] Construct a topological graph based on preliminary vector topological descriptors;

[0016] Based on the topological map, construct graphical and tabular channels, and determine the candidate attribute conflict regions of the graphical and tabular channels;

[0017] Based on each candidate attribute conflict region, geometric topological features, table structure confidence features, and text distribution features are obtained to construct table attribute scoring functions and graphic attribute scoring functions. Combined with arbitration rules, the boundaries of the conflict region are obtained.

[0018] Based on the boundaries of conflict areas, a preliminary screening of non-text areas is performed, a global non-text mask is established to filter text within the image, and the remaining text is divided into structured documents by combining the defined text baseline style and heading styles at all levels.

[0019] Extract the layout units of the structured document and sort the layout units in order;

[0020] Based on sequentially sorted page layout units, it outputs editable documents.

[0021] Furthermore, the definition of the text base style for the vector document specifically includes:

[0022] The text lines of the layout document page are parsed to obtain the font family, font size and weight corresponding to each text line, and the frequency of occurrence of each font family and font size tuple is counted.

[0023] A two-dimensional statistical distribution matrix is ​​established using font family and font size as dimensions, and the presence of multiple statistical peaks is detected.

[0024] When there are multiple statistical peaks, the percentage of the page coverage area is introduced as a criterion: for each font size, the area of ​​the enclosed area of ​​all text lines corresponding to that font size is accumulated to obtain the cumulative area of ​​the current font size;

[0025] For each candidate font family and font size combination, a weighted score is calculated according to the formula Score(F,S)=α·C(F,S)+β·A(S) to obtain the score result, where α and β are preset weight coefficients, α+β=1, A(S) is the cumulative area of ​​the current font size, and C(F,S) is the frequency of occurrence of each font family and font size combination.

[0026] Select the candidate combination with the highest score as the candidate style of the main text, and record its font family, font size and corresponding weight as the main text baseline style vector Base_Style={Base_Family,Base_Size,Base_Weight};

[0027] Calculate the line spacing distribution of candidate body text styles: If the line spacing of a candidate style is significantly greater than that of other candidate styles, it is determined to be a title / section title style and removed from the body text candidate set to obtain the body text baseline.

[0028] Furthermore, the definition of the heading styles at each level in the vector document specifically includes:

[0029] Define multi-level heading thresholds based on the body text baseline style vector:

[0030] First-level heading requirements: font size ≥ Base_Size×α1, and line spacing ≥ Base_Line_Space×β1;

[0031] Second-level heading conditions: font size is in the range of [Base_Size×α2, Base_Size×α1), and matches the level number + text regular expression pattern;

[0032] Where α1>α2>1, β1>1;

[0033] If the font size distribution of the body text does not show a clear main peak and multiple font size intervals show severe overlap, a degradation strategy is adopted:

[0034] Use the font size range with the largest cumulative area contribution as the benchmark for the main text;

[0035] Automatic title recognition based on font size ratio is suspended; only hierarchical numbered title detection triggered by regular expressions is retained.

[0036] Furthermore, the step of performing multi-scale iterative clustering based on the spatial index tree to generate a preliminary vector topological descriptor specifically includes:

[0037] Acquire line segment data, and set the anisotropic expansion kernel normal direction radius and tangential direction radius based on the average direction field of the line segment data;

[0038] A first expansion radius is set along the tangent direction of the line segment, and a second expansion radius is set along the normal direction of the line segment, wherein the second expansion radius is greater than the first expansion radius;

[0039] Set two different spatial scale parameters in the spatial index tree: microscale s1 and macroscale s2, and satisfy s2>s1;

[0040] At the microscale s1, the initial spatial dataset is clustered using the first expansion radius to generate initial connected clusters;

[0041] Based on the initial connected clusters, the clusters are used as basic units, and the boundary boxes of the clusters are merged in a second round at a macroscopic scale s2 using a second expansion radius to form aggregated clusters. The scale of the aggregated clusters is larger than the macroscopic scale s2.

[0042] At the end of each round of clustering, the number of internal objects, object type distribution, average line segment length, and number of closed paths of each cluster at the current level are recorded to form a preliminary vector topology descriptor.

[0043] Furthermore, the construction of the topological graph based on the preliminary vector topological descriptor specifically includes:

[0044] Each connected cluster is represented as a graph structure: based on the vector endpoints in the vector topology descriptor as nodes, vector line segments as edges, and filled closed regions as faces, a graph G=(V,E,F) is constructed, where V is the set of nodes, E is the set of edges, and F is the set of faces;

[0045] Calculating the topological graph features of the graph structure includes:

[0046] Node degree distribution: Calculates the number of nodes and their proportions for multiple statistical degrees;

[0047] Number of closed loops: Identify all closed loops in the diagram and count their total number;

[0048] Closed loop shape approximation: Perform geometric fitting on each closed loop, classify it into rectangle, rhombus, circle or other shapes, and record the number of each shape;

[0049] Number of intersections: Identify all intersections in the graph, and count the total number of intersections and the percentage of non-orthogonal intersections;

[0050] The above topological graph features are integrated into a vector topological graph feature vector.

[0051] Furthermore, the step of constructing graphical and tabular channels based on topological maps and determining candidate attribute conflict regions for the graphical and tabular channels specifically includes:

[0052] Step A, Graphics Topology Channel Processing:

[0053] Based on vector connected clusters and their vector topology graph feature vectors, for each connected cluster, topological indices are calculated: orthogonality rate, non-orthogonal crossover ratio, and closure unit diversity.

[0054] If the connected clusters satisfy any of the following conditions, then their corresponding regions are marked as candidate bounding boxes in the graphics:

[0055] The orthogonality rate is lower than the first preset threshold θ1; or

[0056] The non-orthogonal crossover ratio is higher than the second preset threshold θ2; or

[0057] The diversity of closed units is higher than the third preset threshold θ3 to indicate the existence of non-rectangular closed regions;

[0058] Step B, Table Grid Channel Processing:

[0059] An improved table detection algorithm is used to comprehensively evaluate the alignment of multi-line text baselines in multiple column directions, as well as the uniformity of the row and column directions of the candidate cell region.

[0060] Output the candidate bounding boxes for the table, and append the estimated number of rows and columns for each region, as well as a score representing the regularity of the cell rectangle;

[0061] Step C: Identify candidate attribute conflict areas in the graphics channel and the table channel;

[0062] For each graphic candidate bounding box output by the graphic channel, calculate the intersection-union ratio (IoU) with the table candidate bounding boxes output by the table channel;

[0063] When the IoU is greater than or equal to the set threshold, the corresponding region is marked as a candidate attribute conflict region.

[0064] Furthermore, based on each candidate attribute conflict region, geometric topological features, table structure confidence features, and text distribution features are obtained to construct table attribute scoring functions and graphic attribute scoring functions. Combined with arbitration rules, the boundary of the conflict region is obtained, specifically including:

[0065] Get:

[0066] Geometric topological features, including orthogonality ratio, non-orthogonal cross ratio, and entropy of closed shape;

[0067] Table structure confidence features include grid regularity score and row and column stability;

[0068] Text distribution features, including the average rotation angle and angular variance of text blocks within the region, are used to identify whether there are a large number of vertical or irregularly oriented explanatory texts, as well as the projection overlap pattern of text and line segments.

[0069] Based on geometric topological features, table structure confidence features, and text distribution features, table attribute scoring functions and graphic attribute scoring functions are constructed.

[0070] Define arbitration rules and score table and graphic attributes according to the arbitration rules; perform multi-dimensional conflict arbitration to determine the bounding boxes of table areas, complex graphics / flowchart areas, and other image / bitmap areas.

[0071] Furthermore, based on the boundaries of conflict areas, a preliminary screening of non-text regions is performed, a global non-text mask is established to filter text within the image, and combined with the defined text baseline style and heading styles at all levels, the remaining text is divided to obtain a structured document, specifically including:

[0072] Add the bounding boxes of table areas, complex graphics / flowchart areas, and other image / bitmap areas that have been confirmed by conflict arbitration to the Non_Text_BBoxes collection;

[0073] Detect text blocks located inside complex graphics / flowcharts that satisfy a preset distance constraint relationship with connecting lines or nodes within the graphics, mark local sub-regions of such text blocks as in-graphics explanatory text regions, and add their bounding boxes to the Non_Text_BBoxes set;

[0074] For any text block, the text block is only identified as in-image text and excluded if its center point falls within any non-text region bounding box in the Non_Text_BBoxes set, and the intersection-union ratio (IoU) between the text block bounding box and the non-text region bounding box is greater than or equal to a set threshold; otherwise, it is retained as body text.

[0075] Extract text blocks from the main text and determine their category as main text, first-level heading, second-level heading, or footnote based on the relative relationship between the current text block's font size, font family, font weight, and base style.

[0076] Text blocks that meet the heading regular expression pattern and whose font size and line spacing meet the threshold conditions described in the multi-level heading threshold are further marked as headings of the corresponding level.

[0077] For text blocks whose position is less than a preset distance from the nearest non-text area in the Non_Text_BBoxes set, and whose font size is significantly smaller than the base font size or are italic, mark them as figure captions or table captions;

[0078] Attach figure captions or table captions below the corresponding table or image objects to create a structured document.

[0079] Furthermore, the step of extracting the layout units of the structured document and sequentially sorting the layout units specifically includes:

[0080] Obtain a set of structured units for the page layout, wherein the set includes at least titles, body paragraphs, table objects, complex graphic objects, and chart annotations;

[0081] Two-dimensional sorting is performed using the y-coordinate of the top of each cell as the primary key and the x-coordinate as the secondary key.

[0082] When a multi-column layout is detected, the column segmentation algorithm is first executed to delineate independent column areas, and then the x-coordinate sequence is independently executed within each column area to obtain the page unit sequence.

[0083] Furthermore, the output of an editable document based on the sequentially sorted page layout units specifically includes:

[0084] Receive the sequence of page layout units that have been sorted sequentially;

[0085] Map heading and body text units directly to heading / paragraph nodes in Markdown, HTML, or Word format;

[0086] Convert the table cells into table objects of the corresponding markup language, and fill the cell content according to the detected row and column structure;

[0087] Rasterize flowcharts or complex graphical units to generate image files, and optionally use in-figure explanatory text to generate alternative text or graphical descriptions;

[0088] Based on spatial proximity and semantic tags, the system attaches image and table annotation nodes to the corresponding image or table objects, outputting a complete and editable document.

[0089] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0090] 1. Adaptive style benchmark construction improves cross-format adaptability. By jointly analyzing multi-dimensional statistical features such as font family, font size, page area, and line spacing stability, a benchmark style vector for the main text is dynamically established, and multi-level heading recognition is performed based on relative font size ratios and text patterns. The style recognition of this invention does not require predefined templates or fixed thresholds, and can maintain high stability in the recognition of main text and headings in document scenarios with cross-template, cross-language, and multi-font mixed layouts.

[0091] 2. Layered vector topology graph analysis enhances the ability to distinguish between tables and graphs. It introduces multi-scale, anisotropic vector clustering and topology graph modeling, and integrates topological features such as node degree distribution, types of closed shapes, and proportion of non-orthogonal intersections to effectively distinguish between regular table frames and complex graphs such as irregular flowcharts and architecture diagrams, thereby reducing the situation where complex graphs are misjudged as tables.

[0092] 3. Conflict arbitration of dual-channel detection and multi-dimensional feature fusion: This invention uses a graphic channel based on topological features and a table channel based on grid alignment features for parallel detection. When candidate regions overlap, a comprehensive scoring function composed of geometric topological features, table structure confidence and text distribution features is introduced for arbitration. This can adaptively distinguish between tables and complex graphics, improve the classification results of conflict regions and improve the overall parsing accuracy.

[0093] 4. Spatial Exclusivity Text Extraction and Style / Spatial Joint Semantic Recognition: Using the Non_Text_BBoxes set and a dual condition of center point + IoU, this mechanism strictly distinguishes between captions and main text within figures. Combined with adaptive style benchmarks and spatial proximity, text closer to figures / tables and with smaller font sizes is automatically categorized as figure captions or table captions. This mechanism significantly reduces the rate of mis-extracted text within figures as main text, resulting in an output that more closely resembles the reading experience and semantic structure of the original document.

[0094] 5. A closed-loop control process from vector analysis to structured output: This invention organically combines adaptive style benchmark construction, vector topology graph analysis, dual-channel conflict arbitration, spatially exclusive text extraction, and multimodal serialization reconstruction to form a mutually constraining and mutually correcting closed-loop process. In practical applications, this closed-loop process can simultaneously reduce the misclassification rate of tables / flowcharts, reduce the proportion of text within diagrams being mistakenly incorporated into the main text, and improve the recognition and reconstruction accuracy of structured elements such as titles and captions. Attached Figure Description

[0095] To more clearly illustrate the technical solutions of the exemplary embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly described below. It should be understood that the following drawings only show some embodiments of the present invention and should not be considered as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort. In the drawings:

[0096] Figure 1 This is a flowchart of the document layout parsing and reconstruction method based on vector topology and conflict arbitration in an embodiment of the present invention. Detailed Implementation

[0097] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the embodiments and accompanying drawings. The illustrative embodiments and descriptions of the present invention are only used to explain the present invention and are not intended to limit the present invention.

[0098] As one possible implementation method, such as Figure 1As shown, this embodiment provides a document layout parsing and reconstruction method based on vector topology and conflict arbitration. This embodiment mainly targets the data processing of PDF documents. The data source of PDF documents can be text-based PDFs or image-based PDFs. Text-based PDFs can be directly parsed as character streams, while image / scanned PDFs need to undergo OCR (Optical Character Recognition) for vectorization conversion and layout analysis to restore searchable and editable text and layout information. This embodiment starts from the analysis of underlying vector instructions and sequentially executes a closed-loop processing flow of adaptive style benchmark construction, multi-scale vector topology map analysis, dual-channel detection and multi-dimensional conflict arbitration, and content extraction and serialization reconstruction based on spatial exclusivity and hierarchical semantics. By introducing vector connectivity topology analysis, it solves the problem of classification confusion between complex graphics such as flowcharts and architecture diagrams and tables caused by simply relying on bounding box detection, reducing the situation where complex graphics are mistakenly parsed as tables. By constructing spatial exclusivity constraints and style / space joint semantic recognition rules, it improves the order and attribution judgment of text extraction in mixed text and image areas, avoids the incorrect inclusion of explanatory text inside charts into the main text, and improves the accuracy of document semantic structure restoration. By constructing an adaptive document style benchmark through multidimensional statistics, the robustness of title and body text recognition in documents with cross-typesetting styles, cross-language and multi-font is improved, breaking through the limitations of traditional style recognition methods based on fixed thresholds or static rules.

[0099] The specific implementation steps include:

[0100] 1. Adaptive document style baseline construction;

[0101] This embodiment aims to adaptively determine the basic style of the document body and the styles of headings at all levels without the need for pre-set templates, thus providing a foundation for subsequent text classification.

[0102] 1.1 Two-level statistics on the joint distribution of font family and font size;

[0103] a. For each text object in the document, extract at least the following style attributes from the underlying information: font family, font weight, font size, whether it is bold, whether it is italic, etc.

[0104] b. Construct a two-dimensional statistical distribution with “font family – font size” as the dimension, and record the frequency of each combination.

[0105] c. When multiple statistical peaks exist, the "page coverage area ratio" is introduced as a criterion:

[0106] a) Accumulate the total area of ​​the text enclosed region corresponding to each font size;

[0107] b) Weight the frequency and cumulative area, and select the candidate with the highest score as the candidate style for the main text.

[0108] d. Record the font family, font size, and font weight of the selected candidate text styles as the text base style vector:

[0109] Base_Style={Base_Family,Base_Size,Base_Weight}.

[0110] 1.2 Adaptive correction of line spacing / paragraph pre-paragraph and post-paragraph statistics to the text baseline;

[0111] Calculate the line spacing distribution for candidate body text styles; if the line spacing of a candidate style is significantly greater than that of other candidate styles (e.g., more than 1.5 times the median), it is determined to be a title / section title style and removed from the body text candidate set; only styles that have passed this "line spacing stability screening" are confirmed as the body text benchmark.

[0112] 1.3 Joint determination of the relative ratio of multi-level heading thresholds and text patterns;

[0113] Based on the determined Base_Style, define multi-level heading thresholds:

[0114] First-level heading requirements: font size ≥ Base_Size×α1, and line spacing ≥ Base_Line_Space×β1;

[0115] Second-level heading criteria: font size is in the range [Base_Size×α2, Base_Size×α1), and matches the regular expression pattern "hierarchical number + text";

[0116] Where α1>α2>1, β1>1, is limited by a pre-defined empirical interval (e.g., α1∈[1.6,2.5], α2∈[1.2,1.6]).

[0117] By combining font size ratio with text pattern matching, multi-level headings can be robustly identified under different document styles.

[0118] 1.4 Robustness mechanism for abnormal layouts;

[0119] If the text size distribution does not show a clear main peak and multiple font size intervals are severely overlapping, the present invention can adopt a degradation strategy: use the font size interval with the largest cumulative area contribution as the text benchmark; suspend automatic title recognition based on font size ratio, and only retain the hierarchical numbering title detection triggered by regular expressions, so as to ensure that usable structured results can still be obtained under abnormal layout.

[0120] 2. Graph clustering based on vector connected component analysis;

[0121] 2.1 Instruction-level pre-classification and spatial indexing;

[0122] The underlying drawing instructions of PDF or vector documents are parsed, and vector objects are pre-classified into at least the following types: line segments, rectangles, polylines, Bézier curves, and polygonal filled areas.

[0123] Build a spatial index tree (such as an R-tree or kd-tree) for each type of object to reduce the computational complexity of subsequent clustering.

[0124] 2.2 Orientation-sensitive anisotropic expanding nuclei;

[0125] To improve the distinction between complex graphics and table borders, this invention employs an anisotropic expansion strategy for long, narrow line segments: setting a relatively large expansion radius r along the normal direction of the line segment. n Set a small expansion radius r along the tangent direction of the line segment. t The aforementioned direction-sensitive expansion can prevent excessive merging of long-distance parallel lines in the horizontal / vertical direction, which helps to maintain the geometric difference between table grid lines and flowchart connection lines.

[0126] 2.3 Multi-scale iterative clustering;

[0127] Set two different spatial scale parameters in the spatial index tree: microscale s1 and macroscale s2, and satisfy s2>s1;

[0128] At the microscale s1, the initial spatial dataset is clustered using the first expansion radius to generate initial connected clusters;

[0129] Based on the initial connected clusters, the clusters are used as basic units, and the boundary boxes of the clusters are merged in a second round at the macroscopic scale s2 using a second expansion radius to form aggregated clusters. The scale of the aggregated clusters is larger than the macroscopic scale s2, and the second expansion radius is larger than the first expansion radius.

[0130] At the end of each round of clustering, the number of internal objects, object type distribution, average line segment length, and number of closed paths of each cluster at the current level are recorded to form a preliminary vector topology descriptor.

[0131] 2.4 Construction of topological graph;

[0132] Each connected cluster is represented as a graph structure: based on the vector endpoints in the vector topological descriptor as nodes, vector line segments as edges, and filled closed regions as faces, a graph G=(V,E,F) is constructed, where V is the set of nodes, E is the set of edges, and F is the set of faces;

[0133] Calculate the node degree distribution (proportion of nodes with degrees of 1 / 2 / ≥3); the number of closed loops and their approximate shapes (rectangular, rhomboid, circular); and the number of intersections (proportion of non-orthogonal intersections). This information forms the vector topological feature vector T_Feature of the cluster, which is used for subsequent conflict arbitration.

[0134] 3. An adaptive conflict arbitration mechanism for multi-source feature fusion;

[0135] To avoid complex graphics being misidentified as tables, this invention employs a dual-channel detection strategy in which the graphics detection channel and the table detection channel operate in parallel, and performs multi-dimensional conflict arbitration on the detection results.

[0136] 3.1 Dual-channel candidate region generation;

[0137] 3.1.1 Graphics Channel A (based on topological map);

[0138] Input: All vector connected clusters and their vector topological graph feature vectors T_Feature obtained in step 2;

[0139] For each connected cluster, the computation includes at least:

[0140] Orthorth ratio Ortho_R = (horizontal line length + vertical line length) / total line length;

[0141] Cross-NonOrtho Ratio = Number of nonorthogonal crossover points / Total number of crossover points;

[0142] Closed unit diversity α_closed (number of different shapes of closed loops / total number of closed loops);

[0143] If any of the following conditions are met, the region corresponding to the connected cluster is marked as a candidate region for complex graphs:

[0144] Ortho_R is lower than the preset threshold θ1;

[0145] Cross_NonOrtho_R is higher than the preset threshold θ2;

[0146] If α_closed is higher than the preset threshold θ3, it indicates that there are many non-rectangular closed regions.

[0147] 3.1.2 Table Channel B (based on grid and alignment features);

[0148] This improved table detection algorithm, in addition to traditional grid line detection, integrates: multi-column alignment of multi-line text baselines and row-column uniformity of candidate cell regions. Specifically, the improved algorithm builds upon traditional table detection based on horizontal / vertical lines / rectangles by introducing text row-column alignment features and grid regularity evaluation to jointly determine candidate table regions. The algorithm outputs candidate table boxes and confidence information by estimating row-column structure and calculating cell rectangle regularity (such as row-column spacing uniformity and cell shape stability).

[0149] Output a set of candidate regions in a table, with each region accompanied by: an estimate of the number of rows and columns; and a cell rectangle regularity score (Grid_Regularity).

[0150] 3.2 Identification of conflict zones;

[0151] For each candidate bounding box output by graphics channel A, the intersection-over-union ratio (IoU) is calculated with the candidate bounding boxes output by table channel B. When the IoU is greater than or equal to a set threshold τ, the corresponding region is marked as a candidate attribute conflict region and enters the subsequent arbitration process.

[0152] 3.3 Multi-dimensional feature fusion arbitration;

[0153] 3.3.1 For each conflict region, this invention uses a combination of geometric topological features, table structure confidence features, and text distribution features for comprehensive discrimination:

[0154] e. The orthogonality ratio ΔOrtho = 1 - Ortho_R;

[0155] f. Cross-NonOrtho ratio (R);

[0156] g. Closed shape entropy H_closed: Calculates information entropy based on the frequency distribution of various closed loop shapes, used to characterize the diversity of closed shapes.

[0157] 3.3.2 Table structure confidence characteristics:

[0158] a. Grid Regularity score;

[0159] b. RowCol_Stability can be obtained by uniformly dividing the region and statistically inferring it;

[0160] 3.3.3 Text distribution characteristics:

[0161] The average rotation angle and angular variance of the text blocks within the region are used to identify whether there is a large amount of vertical or irregularly oriented explanatory text.

[0162] Text and line segment projection overlap patterns: In typical tables, the center of text blocks is mostly located inside the cells and is highly aligned with the grid; while in flowcharts or complex graphics, explanatory text is usually close to the center of the graphic elements, but its position distribution relative to line segments and nodes is more irregular.

[0163] 3.3.4 Comprehensive Scoring and Arbitration Rules:

[0164] Constructing table attribute rating functions and graph attribute rating functions:

[0165] Score_Table = w1 * Ortho_R + w2 * Grid_Regularity + w3 * (1 - H_closed);

[0166] Score_Graph = v1 * ΔOrtho + v2 * Cross_NonOrtho_R + v3 * H_closed + v4 * Text_Irregularity;

[0167] Among them, Text_Irregularity can be comprehensively calculated from the variance of the text rotation angle and the misalignment of the text-segment projection; w1, w2, w3, v1, v2, v3, v4 are weight parameters that can be set according to experience or training data.

[0168] The arbitration rules can include:

[0169] If Score_Graph - Score_Table ≥ δ, then veto the table attribute and determine that the candidate region is a complex graphic region;

[0170] If Score_Table - Score_Graph ≥ δ, confirm that the candidate region is a table region;

[0171] If the two are close (|difference| < δ), then trigger the conservative strategy:

[0172] If the number of rows and columns < 2 or Grid_Regularity < G_min, then process it as a complex graphic preferentially; otherwise process it as a table preferentially.

[0173] 3.4 Content extraction and reconstruction based on spatial exclusivity and hierarchical semantics;

[0174] Based on determining the boundary coordinates of all non-text objects (such as pictures, tables confirmed by arbitration, complex graphics confirmed by arbitration), this module performs text filtering, semantic classification, and structured reconstruction through a spatial exclusivity mechanism and style / spatial joint rules.

[0175] 4.1 Spatial exclusivity text filtering;

[0176] 4.1.1 Establishment of non-text region set;

[0177] Construct the Non_Text_BBoxes set, where the Non_Text_BBoxes set is a set of non-text bounding boxes, used to store the bounding box information of all non-text elements in the document page, and it includes at least:

[0178] The bounding box of the table region confirmed by conflict arbitration;

[0179] Boundary boxes of complex graphical / flowchart areas confirmed through conflict arbitration;

[0180] The bounding boxes of other image / bitmap regions.

[0181] Text blocks located inside complex graphics and having specific distance constraints with connecting lines or nodes of the graphics are detected. Local sub-regions of these text blocks are marked as "text regions within the graphics" and their bounding boxes are added to the Non_Text_BBoxes set, thus excluding them in subsequent text extraction.

[0182] 4.1.2 The dual condition of overlapping center point and boundary;

[0183] The condition for a text block to be considered "text within the image" is that its center point falls into a certain Non_Text_BBox, and the IoU between the text block bbox and the Non_Text_BBox is ≥ρ (e.g., 0.3), in order to avoid the text at the boundary being mistakenly excluded.

[0184] 4.2 Text category correction aided by style benchmarks;

[0185] 4.2.1 Establishment of a set of non-text regions;

[0186] Construct a collection of Non_Text_BBoxes that includes at least the following:

[0187] The bounding box of the table area confirmed by conflict arbitration;

[0188] Boundary boxes of complex graphical / flowchart areas confirmed through conflict arbitration;

[0189] The bounding boxes of other image / bitmap regions.

[0190] Text blocks located inside complex graphics and having specific distance constraints with connecting lines or nodes of the graphics are detected. Local sub-regions of these text blocks are marked as "text regions within the graphics" and their bounding boxes are added to the Non_Text_BBoxes set, thus excluding them in subsequent text extraction.

[0191] 4.2.2 The dual condition of overlapping center point and boundary;

[0192] The condition for a text block to be classified as "text within the image" is that its center point falls into a certain Non_Text_BBox set, and the IoU between the text block bbox and the Non_Text_BBox set is ≥ρ (e.g., 0.3), in order to avoid the text at the boundary being mistakenly excluded.

[0193] 4.2.2.1 Classification based on style attributes;

[0194] For each text block, its category (body text, first / second level heading, or footnote) is determined based on the relative relationship between font size, font family, font weight, etc. and Base_Style.

[0195] Text blocks that meet the title regular expression pattern and whose font size and line spacing meet the threshold conditions described in 1.3 are further marked as titles of the corresponding level.

[0196] 4.2.2.2 Caption / Table Label Recognition Based on Spatial Proximity;

[0197] For text blocks whose nearest vertical distance to a non-text area (table / graphic / image) is less than the preset distance d_min, and whose font size is significantly smaller than Base_Size or are italicized, they can be marked as figure captions or table captions.

[0198] In subsequent structured output, this type of text will be attached below the corresponding table or image object.

[0199] 4.3 Multimodal serialization reconstruction;

[0200] 4.3.1 Sequence control of page layout units;

[0201] Sort all structured units (including headings, body paragraphs, table objects, complex graphic objects, chart annotations, etc.) according to their top y-coordinate;

[0202] When the top y-coordinates are close to or the same as the layers, arrange them according to the x-coordinate order;

[0203] For documents with multi-column layouts, each column region can be identified in advance using a column segmentation algorithm, and the sorting process can be performed independently within each column to restore an output sequence that is closer to the original reading order.

[0204] 4.3.2 Structured output format;

[0205] The title and body text are output as heading / paragraph nodes in Markdown, HTML, or Word format;

[0206] The table area is output as a Markdown table, HTML table, or Word table object, and the cell content is filled according to the detected row and column structure;

[0207] Flowcharts or complex graphical areas can be rasterized to generate image files, and alternative text (alttext) or graphical descriptions can be extracted from the in-figure explanatory text.

[0208] Captions and table annotations are attached to the corresponding image or table object nodes based on spatial proximity and semantic tags.

[0209] As one possible implementation, this embodiment selects a technical PDF document containing multiple pages of content, which includes:

[0210] Complex tables with multiple headers and merged cells;

[0211] Multiple flowcharts and system architecture diagrams, which consist of rectangular nodes, diamond decision boxes, and multiple connecting lines;

[0212] Multi-column main text, as well as figure captions and table notes.

[0213] 1. Parse the underlying vector and text objects page by page of the PDF document, and use the method described in 5.1 to count the font family and font size distribution to obtain the base style Base_Style of the main text, where Base_Size=10pt and Base_Family is a certain monospace font.

[0214] 2. After filtering based on line spacing stability, text with a font size of 14pt and significantly increased line spacing is excluded from the title style.

[0215] 3. In the vector analysis phase, connected component aggregation is performed on line segments and rectangular objects using a two-level dilation scale of s1=2pt and s2=10pt. For each cluster, features such as Ortho_R, Cross_NonOrtho_R, α_closed, and H_closed are calculated.

[0216] 4. The table channel uses grid line detection and multi-column alignment analysis to obtain several candidate table regions, and calculates Grid_Regularity and RowCol_Stability. Typically, the Grid_Regularity of a table is greater than 0.9, while some flowchart regions, although containing rectangles, have a Grid_Regularity lower than 0.6.

[0217] 5. For candidate table regions and candidate complex graph regions with an IoU greater than 0.6, conflict arbitration is performed based on the scoring function described in 5.3. For flowchart regions containing a large number of non-orthogonal intersections and diverse closed shapes, Score_Graph is significantly higher than Score_Table, and therefore is ultimately judged as a complex graph rather than a table.

[0218] 6. During the spatial exclusion phase, add all table and complex graphic boundaries to Non_Text_BBoxes. Text blocks whose center point falls within the boundaries and whose IoU ≥ 0.3 are marked as in-figure text. Text blocks that are not in Non_Text_BBoxes and whose font is Base_Style are marked as body text.

[0219] 7. Text blocks located below figures, less than 20pt from the top of the figure area, and with a font size smaller than Base_Size are marked as figure captions; text blocks with similar positional relationships below tables are marked as table captions.

[0220] 8. The final output is a Markdown document: headings are represented by "# / ##", body text is represented by paragraphs, tables are represented by Markdown table syntax, complex graphics are represented by image links, and there is an alternative text extracted from the text describing the image. At the same time, there is a caption text below the image.

[0221] Tests show that in this embodiment, the number of cases where flowcharts were incorrectly identified as tables has been significantly reduced, the explanatory text within the diagrams has no longer been incorrectly incorporated into the main text, and the structure of titles and captions has been correctly restored.

[0222] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A document layout parsing and reconstruction method based on vector topology and conflict arbitration, characterized in that, The specific steps include the following: Define the base text style and heading styles for the vector document. Specifically, the definition of the base text style for the vector document includes: The text lines of the layout document page are parsed to obtain the font family, font size and weight corresponding to each text line, and the frequency of occurrence of each font family and font size tuple is counted. A two-dimensional statistical distribution matrix is ​​established using font family and font size as dimensions, and the presence of multiple statistical peaks is detected. When there are multiple statistical peaks, the percentage of the page coverage area is introduced as a criterion: for each font size, the area of ​​the enclosed area of ​​all text lines corresponding to that font size is accumulated to obtain the cumulative area of ​​the current font size; For each candidate font family and font size combination, a weighted score is calculated according to the formula Score(F,S)=α·C(F,S)+β·A(S) to obtain the score result, where α and β are preset weight coefficients, α+β=1, A(S) is the cumulative area of ​​the current font size, and C(F,S) is the frequency of occurrence of each font family and font size combination. Select the candidate combination with the highest score as the candidate style of the main text, and record its font family, font size and corresponding weight as the main text baseline style vector Base_Style={Base_Family,Base_Size,Base_Weight}; Calculate the line spacing distribution of candidate body text styles: If the line spacing of a certain candidate style is significantly greater than that of other candidate styles, it is determined to be a title / section title style and removed from the body text candidate set to obtain the body text baseline; Based on the defined text base style and heading styles at all levels, the underlying drawing instructions of the vector document are parsed, the vector document is pre-classified, and a spatial index tree is built for each pre-classified object. Based on the spatial index tree, multi-scale iterative clustering is performed to generate preliminary vector topological descriptors; Construct a topological graph based on preliminary vector topological descriptors; Based on the topological map, construct graphical and tabular channels, and determine the candidate attribute conflict regions of the graphical and tabular channels; Based on each candidate attribute conflict region, geometric topological features, table structure confidence features, and text distribution features are obtained to construct table attribute scoring functions and graphic attribute scoring functions. Combined with arbitration rules, the boundaries of the conflict region are obtained. Based on the boundaries of conflict areas, a preliminary screening of non-text areas is performed, a global non-text mask is established to filter text within the image, and the remaining text is divided into structured documents by combining the defined text baseline style and heading styles at all levels. Extract the layout units of the structured document and sort the layout units in order; Based on sequentially sorted page layout units, it outputs editable documents.

2. The document layout parsing and reconstruction method based on vector topology and conflict arbitration according to claim 1, characterized in that, The definition of heading styles at each level in the vector document specifically includes: Define multi-level heading thresholds based on the body text baseline style vector: First-level heading requirements: font size ≥ Base_Size×α1, and line spacing ≥ Base_Line_Space×β1; Second-level heading conditions: font size is in the range of [Base_Size×α2, Base_Size×α1), and matches the level number + text regular expression pattern; Where α1>α2>1, β1>1; If the font size distribution of the body text does not show a clear main peak and multiple font size intervals show severe overlap, a degradation strategy is adopted: Use the font size range with the largest cumulative area contribution as the benchmark for the main text; Automatic title recognition based on font size ratio is suspended; only hierarchical numbered title detection triggered by regular expressions is retained.

3. The document layout parsing and reconstruction method based on vector topology and conflict arbitration according to claim 1, characterized in that, The process of multi-scale iterative clustering based on spatial index trees to generate preliminary vector topological descriptors specifically includes: Acquire line segment data, and set the anisotropic expansion kernel normal direction radius and tangential direction radius based on the average direction field of the line segment data; A first expansion radius is set along the tangent direction of the line segment, and a second expansion radius is set along the normal direction of the line segment, wherein the second expansion radius is greater than the first expansion radius; Set two different spatial scale parameters in the spatial index tree: microscale s1 and macroscale s2, and satisfy s2>s1; At the microscale s1, the initial spatial dataset is clustered using the first expansion radius to generate initial connected clusters; Based on the initial connected clusters, the clusters are used as basic units, and the boundary boxes of the clusters are merged in a second round at a macroscopic scale s2 using a second expansion radius to form aggregated clusters. The scale of the aggregated clusters is larger than the macroscopic scale s2. At the end of each round of clustering, the number of internal objects, object type distribution, average line segment length, and number of closed paths of each cluster at the current level are recorded to form a preliminary vector topology descriptor.

4. The document layout parsing and reconstruction method based on vector topology and conflict arbitration according to claim 3, characterized in that, The construction of the topology graph based on the preliminary vector topology descriptor specifically includes: Each connected cluster is represented as a graph structure: based on the vector endpoints in the vector topology descriptor as nodes, vector line segments as edges, and filled closed regions as faces, a graph G=(V,E,F) is constructed, where V is the set of nodes, E is the set of edges, and F is the set of faces; Calculating the topological graph features of the graph structure includes: Node degree distribution: Calculates the number of nodes and their proportions for multiple statistical degrees; Number of closed loops: Identify all closed loops in the diagram and count their total number; Closed loop shape approximation: Perform geometric fitting on each closed loop, classify it into rectangle, rhombus, circle or other shapes, and record the number of each shape; Number of intersections: Identify all intersections in the graph, and count the total number of intersections and the percentage of non-orthogonal intersections; The above topological graph features are integrated into a vector topological graph feature vector.

5. The document layout parsing and reconstruction method based on vector topology and conflict arbitration according to claim 1, characterized in that, The process of constructing graphical and tabular channels based on topological maps and determining candidate attribute conflict regions for the graphical and tabular channels specifically includes: Step A, Graphics Topology Channel Processing: Based on vector connected clusters and their vector topology graph feature vectors, for each connected cluster, topological indices are calculated: orthogonality rate, non-orthogonal crossover ratio, and closure unit diversity. If the connected clusters satisfy any of the following conditions, then their corresponding regions are marked as candidate bounding boxes in the graphics: The orthogonality rate is lower than the first preset threshold θ1; or The non-orthogonal crossover ratio is higher than the second preset threshold θ2; or The diversity of closed units is higher than the third preset threshold θ3 to indicate the existence of non-rectangular closed regions; Step B, Table Grid Channel Processing: An improved table detection algorithm is used to comprehensively evaluate the alignment of multi-line text baselines in multiple column directions, as well as the uniformity of the row and column directions of the candidate cell region. Output the candidate bounding boxes for the table, and append the estimated number of rows and columns for each region, as well as a score representing the regularity of the cell rectangle; Step C: Identify candidate attribute conflict areas in the graphics channel and the table channel; For each graphic candidate bounding box output by the graphic channel, calculate the intersection-union ratio (IoU) with the table candidate bounding boxes output by the table channel; When the IoU is greater than or equal to the set threshold, the corresponding region is marked as a candidate attribute conflict region.

6. The document layout parsing and reconstruction method based on vector topology and conflict arbitration according to claim 1, characterized in that, The process involves acquiring geometric topological features, table structure confidence features, and text distribution features for each candidate attribute conflict region to construct table attribute scoring functions and graphic attribute scoring functions. These functions are then combined with arbitration rules to obtain the conflict region boundaries. Specifically, this includes: Get: Geometric topological features, including orthogonality ratio, non-orthogonal cross ratio, and entropy of closed shape; Table structure confidence features include grid regularity score and row and column stability; Text distribution features, including the average rotation angle and angular variance of text blocks within the region, are used to identify whether there are a large number of vertical or irregularly oriented explanatory texts, as well as the projection overlap pattern of text and line segments. Based on geometric topological features, table structure confidence features, and text distribution features, table attribute scoring functions and graphic attribute scoring functions are constructed. Define arbitration rules, and apply the arbitration rules to the scoring functions for table attributes and graphic attributes; Perform multi-dimensional conflict arbitration to determine the bounding boxes of table areas, complex graphics / flowchart areas, and other image / bitmap areas.

7. The document layout parsing and reconstruction method based on vector topology and conflict arbitration according to claim 6, characterized in that, The process involves initial screening of non-text regions based on conflict area boundaries, establishing a global non-text mask to filter text within the image, and combining defined text baseline styles and heading styles at all levels to segment the remaining text and obtain a structured document. Specifically, this includes: The bounding boxes of table areas, complex graphics / flowchart areas, and other image / bitmap areas confirmed by conflict arbitration are added to the Non_Text_BBoxes collection. The Non_Text_BBoxes collection is a collection of non-text bounding boxes used to store the bounding box information of all non-text elements in the document page. Detect text blocks located inside complex graphics / flowcharts that satisfy a preset distance constraint relationship with connecting lines or nodes within the graphics, mark local sub-regions of the text blocks as in-graphics explanatory text regions, and add their bounding boxes to the Non_Text_BBoxes set; For any text block, the text block is only identified as in-image text and excluded if its center point falls within any non-text region bounding box in the Non_Text_BBoxes set, and the intersection-union ratio (IoU) between the text block bounding box and the non-text region bounding box is greater than or equal to a set threshold; otherwise, it is retained as body text. Extract text blocks from the main text and determine their category as main text, first-level heading, second-level heading, or footnote based on the relative relationship between the current text block's font size, font family, font weight, and base style. Text blocks that meet the heading regular expression pattern and whose font size and line spacing meet the threshold conditions described in the multi-level heading threshold are further marked as headings of the corresponding level. For text blocks whose position is less than a preset distance from the nearest non-text area in the Non_Text_BBoxes set, and whose font size is significantly smaller than the base font size or are italic, mark them as figure captions or table captions; Attach figure captions or table captions below the corresponding table or image objects to create a structured document.

8. The document layout parsing and reconstruction method based on vector topology and conflict arbitration according to claim 1, characterized in that, The extraction of layout units from the structured document and the sequential sorting of these layout units specifically include: Obtain a set of structured units for the page layout, wherein the set includes at least titles, body paragraphs, table objects, complex graphic objects, and chart annotations; Two-dimensional sorting is performed using the y-coordinate of the top of each cell as the primary key and the x-coordinate as the secondary key. When a multi-column layout is detected, the column segmentation algorithm is first executed to delineate independent column areas, and then the x-coordinate sequence is independently executed within each column area to obtain the page unit sequence.

9. The document layout parsing and reconstruction method based on vector topology and conflict arbitration according to claim 8, characterized in that, The sequentially ordered layout units output an editable document, specifically including: Receive the sequence of page layout units that have been sorted sequentially; Map heading and body text units directly to heading / paragraph nodes in Markdown, HTML, or Word format; Convert the table cells into table objects of the corresponding markup language, and fill the cell content according to the detected row and column structure; Rasterize flowcharts or complex graphical units to generate image files, and optionally use in-figure explanatory text to generate alternative text or graphical descriptions; Based on spatial proximity and semantic tags, the system attaches image and table annotation nodes to the corresponding image or table objects, outputting a complete and editable document.

Images