A parsing method for transaction detail type frameless line grouping table

Abstract

The application provides a kind of transaction detail class frameless line grouping type table analysis method, it is related to document intelligent processing technical field.The transaction detail class frameless line grouping type table analysis method includes: estimating the size of document main word as a reference unit by connected domain analysis;Based on the reference unit, construct a morphological structure element, connect the same row of text into a text block using morphological dilation operation, form a logical line and label using vertical clustering;Perform vertical projection analysis, analyze the column space region, and draw vertical separation lines;Fuse the line label and vertical separation lines with the original image to generate an auxiliary image containing explicit structure hints, and then perform model recognition.The application establishes a complete grid framework through double labeling of rows and columns, enabling model recognition to unify the structure as a whole rather than fragmented text, significantly improving the accuracy of analysis.

Description

Technical Field

[0001] This invention relates to the field of intelligent document processing technology, and in particular to a method for parsing borderless grouped tables for transaction details. Background Technology

[0002] Visual language models (VLMs) have made significant progress in document understanding. While VLMs have strong text recognition capabilities, they are relatively weak in understanding the spatial structure of documents, especially when dealing with specific documents such as invoices and statements. These documents are characterized by irregular table structures, dense text, lack of borders, and a lack of explicit structural anchors, causing the model to fail to accurately determine the boundaries of closely packed rows and columns, frequently resulting in content confusion.

[0003] The core challenge of borderless tables for transaction details lies in their implicit structure: row and column relationships are implied only through whitespace and alignment, rather than explicit line demarcation. This manifests in three main issues: row boundaries are difficult to identify, making it easy to merge or split adjacent rows; column boundaries are ambiguous, making it easy for multiple columns to be linked together when arranged closely; and the overall structure is lacking, causing the table to be recognized as fragmented text blocks rather than a unified structure.

[0004] Existing methods mainly solve this problem by training specialized table recognition models or improving the VLM architecture, but both have limitations: the former requires a large amount of labeled data, is costly, and has limited generalization ability; the latter requires redesigning the model and consumes huge computational resources.

[0005] Therefore, the existing visual language models are not accurate enough when parsing borderless tables of transaction details; at the same time, the use of a fixed threshold for table parsing causes the parsing method to fail when the font changes. Summary of the Invention

[0006] The technical problem to be solved by this invention is: how to construct a lightweight auxiliary method that can significantly improve the parsing accuracy of borderless tables of transaction details without modifying the model architecture or retraining.

[0007] This invention uses font size as the atomic dimension of the table structure. By leveraging the proportional relationship between row height, row spacing, column spacing, and font size, an adaptive spatial scale system is established. Through methods such as row clustering, column detection, and morphological operations, the implicit structure is transformed into explicit visual cues, accurately identifying row and column boundaries. Higher image resolution further enhances the analytical performance of this invention.

[0008] In a first aspect, the present invention provides a method for parsing borderless grouped tables for transaction details, comprising the following steps:

[0009] S1. Baseline Unit Determination: The input image is segmented using an adaptive threshold to obtain a binary image. Then, the document main character size is estimated through connected component analysis and used as the baseline unit.

[0010] S2. Generate line annotations: Construct morphological structural elements based on the baseline unit, use morphological dilation operation to connect the text in the same row into text blocks, use vertical clustering to form logical lines and annotate them.

[0011] S3. Detect column boundaries: Determine the column gap threshold based on the reference unit, perform vertical projection analysis on the binary image, detect blank areas between columns, and draw vertical dividing lines;

[0012] S4. Image Fusion: Merges row labels and vertical separators with the original image to generate an auxiliary image containing explicit structural cues;

[0013] S5. Model Recognition: Input the auxiliary image into the visual language model for OCR and table parsing.

[0014] In a second aspect, the present invention provides a computer device including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method described in the first aspect.

[0015] Thirdly, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in the first aspect.

[0016] Fourthly, the present invention provides a computer program product that, when executed by a processor, implements the steps of the method described in the first aspect.

[0017] This invention clearly identifies the range of each logical row through striped annotation layers to avoid vertical confusion; vertical separators clearly indicate column boundaries to avoid horizontal concatenation; and double annotation of rows and columns establishes a complete grid framework, enabling the model to recognize a unified structure rather than fragmented text, significantly improving parsing accuracy. Attached Figure Description

[0018] Figure 1 This is a flowchart of a parsing method for a borderless grouped table for transaction details, as disclosed in this invention.

[0019] Figure 2 This is a schematic diagram of a stripe annotation layer generated based on morphological operations;

[0020] Figure 3 A schematic diagram for detecting and drawing vertical dividing lines. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Steps S1, S2… in the described embodiments of the present invention do not limit the scope of execution of the present invention; the various models, simulation environments, and software described in the present invention are not considered as the only limiting methods of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0022] In this invention, computer device / equipment / system refers to a related entity applied to a computer, such as hardware, a combination of hardware and software, software, or software in execution. More specifically, for example, software includes, but is not limited to, a process running on a processor, a processor, an object, executable software, an execution thread, a program, and / or a computer. Furthermore, an application program or script running on a server, and the server itself, can also be software. One or more software programs may be in an execution process and / or thread, and the software may be localized on one computer and / or distributed across two or more computers, and may be run on various computer-readable media.

[0023] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other.

[0024] Firstly, the present invention provides an embodiment of a parsing method for borderless grouped tables of transaction details, such as... Figure 1 As shown, the specific process can be as follows:

[0025] S1. Baseline Unit Determination: The input image is preprocessed, and the document main character size is estimated through adaptive threshold segmentation and connected component analysis, which is used as the baseline unit.

[0026] The method is as follows: the input image is grayscaled and adaptively binarized, text components are extracted through connected component analysis, and the mode of the effective component height is calculated as the reference unit.

[0027] The specific steps can be as follows:

[0028] The input color image is converted to a grayscale image, and binarization is performed using the Otsu's method to generate a binary image with separated background and text. Connectivity analysis is then performed on the binary image using a connected component labeling algorithm to obtain statistical information such as height, width, and area of ​​each connected component (i.e., connected unit, representing the connected unit of character pixels). The median height of all components is calculated, and valid text components with heights between 0.8 and 1.2 times the median and areas greater than the noise threshold are selected. The mode of the heights of the valid text components is used as the estimated font size of the document, which serves as the benchmark unit for all subsequent morphological kernel sizes and line / column spacing determinations.

[0029] S2. Generate line labels: Based on the estimated font size, use morphological operations and vertical clustering to generate a stripe label layer that identifies logical lines.

[0030] The method is as follows: construct morphological structural elements based on font size and perform expansion operations to connect text in the same row into text blocks; cluster them according to vertical position to form logical rows, and draw stripes with alternating colors.

[0031] Specifically, based on the reference unit size obtained from S1, morphological structural elements are constructed, namely rectangular structural cores, whose morphological characteristics are horizontal elongated strips.

[0032] The aspect ratio of the morphological structuring element is calculated based on the paper size to ensure it connects text within the same line and avoids connections across lines. This structuring element is then used to dilate the binary image, merging discrete, valid text components within the same line into connected text blocks. The dilation process stops when a text block contains only text from the same line and excludes text from different lines. The outlines of the dilated text blocks are extracted, and the bounding boxes of each block are obtained and sorted vertically from top to bottom.

[0033] Perform logical line clustering: Traverse the sorted text blocks and determine whether there is vertical overlap between the bottom of the block and the bottom of the previous logical line, or whether the spacing is less than the vertical tolerance set based on the font size. If the condition is met, the block is added to the current logical line; otherwise, a new logical line is started.

[0034] The tolerance design of this invention solves the problem of inconsistent row spacing in borderless grouped tables of transaction details: in dense areas with small row spacing, the tolerance based on font size can accurately determine whether the bounding boxes belong to the same row; in sparse areas with large row spacing, the tolerance can also correctly separate different rows, avoiding the problem of the fixed threshold method failing in different density areas.

[0035] For the logical rows obtained from clustering, rectangular stripes are drawn in an alternating manner (even-numbered rows are colored, and odd-numbered rows are left blank) to form a stripe label layer, such as... Figure 2As shown, extend the rectangular strip horizontally to the edge of the document to completely cover the text area.

[0036] This step uses a reference unit as a basis to determine the line break area or line spacing through horizontal pixel density detection. The line break situation is marked with an alternating gray and white background to help the large visual model identify line relationships and avoid line overlap.

[0037] S3. Detect column boundaries: Based on the estimated font size, detect blank areas between columns through vertical projection analysis and draw vertical dividing lines.

[0038] The method is as follows: perform vertical projection analysis on the binary image, determine the column gap threshold based on the font size, detect the effective blank column segments, and draw a vertical dividing line in the center of them.

[0039] Specifically, vertical projection analysis is performed on the binary image. For the naturally formed column distribution in the binary image, the sum of pixel values ​​in each column is calculated. A pixel density threshold is set based on the entire file page. When the sum of pixels in a column is less than this threshold, the column is determined to be a blank column. Consecutive blank column regions are detected; if their width is greater than the minimum column gap threshold set based on font size, they are considered valid column separation regions. Vertical dividing lines are drawn at the center of each valid column separation region, using high-contrast colors to clearly define the column boundaries, such as... Figure 3 As shown.

[0040] This step adaptively determines the column gap threshold based on a baseline unit. Based on this threshold, it filters table column segments without borders according to the sparsity of column pixels. Using the column gap threshold filtering method can identify the true column boundaries while filtering out normal intervals between characters, avoiding misinterpreting spaces between words as column separators.

[0041] S4. Image Fusion: Fusion of the stripe annotation layer and separator layer with the original image to generate an auxiliary image containing explicit structural cues.

[0042] Specifically, image fusion techniques such as alpha blending are used to overlay the stripe annotation layer onto the original image with appropriate transparency, ensuring that the stripes provide clear row structure cues without obscuring the original text content. Vertical separator layers are overlaid with opaque or high-opacity layers to clearly define column boundaries. The resulting auxiliary image contains explicit row and column structure information, establishing a complete gridded visual framework.

[0043] S5. Model Recognition: Input the auxiliary image into the visual language model for OCR and table parsing.

[0044] Specifically, auxiliary images are input into the visual language model, which can use the explicit visual cues to accurately identify the row and column structure of the frameless table and output structured parsing results.

[0045] In a second aspect, the present invention provides an embodiment of a computer device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method described in the first aspect.

[0046] Thirdly, the present invention provides an embodiment of a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in the first aspect.

[0047] Fourthly, the present invention provides a computer program product embodiment, which, when executed by a processor, implements the steps of the method described in the first aspect.

[0048] The embodiments described above are merely examples of several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention.

Claims

1. A parsing method for transaction-oriented detailed class frameless line grouping table, characterized in that, Includes the following steps: S1. Baseline Unit Determination: The input image is segmented using an adaptive threshold to obtain a binary image. Then, the document main character size is estimated through connected component analysis and used as the baseline unit. S2. Generate line annotations: Construct morphological structural elements based on the baseline unit, use morphological dilation operation to connect the text in the same row into text blocks, use vertical clustering to form logical lines and annotate them. S3. Detect column boundaries: Determine the column gap threshold based on the reference unit, perform vertical projection analysis on the binary image, detect blank areas between columns, and draw vertical dividing lines; S4. Image Fusion: Merges row labels and vertical separators with the original image to generate an auxiliary image containing explicit structural cues; S5. Model Recognition: Input the auxiliary image into the visual language model for OCR and table parsing.

2. The method of claim 1, wherein, The specific steps of step S1 include: S11. Perform connected component analysis on all closed pixel sets in the input image using the connected component labeling algorithm to obtain connected components; S12. Calculate the height and area of ​​the connecting components and filter out the valid text components; S13. The mode of the effective text component height is used as the reference unit.

3. The method of claim 2, wherein, The specific steps of step S12 include: identifying connected components whose height is in the range of 0.8 to 1.2 times the median and whose area is greater than the threshold as valid text components.

4. The method of claim 2, wherein, The specific steps of step S2 include: S21. Construct morphological structural elements and determine their aspect ratios according to the paper size; S22. Use morphological structural elements to perform dilation operation on the binary image, merge the discrete effective text components in the same row into connected text blocks, extract the text block outlines, and sort them from top to bottom according to the vertical coordinates. S23. Traverse each text block outline as a logical line. Text blocks that have vertical overlap with the bottom of the previous logical line or whose spacing is less than the vertical tolerance are included in the current logical line; otherwise, start a new logical line. S24. Use alternately drawn rectangular strips to label logical rows.

5. The method according to claim 4, characterized in that, Vertical tolerance is set based on reference units; The rectangular strips expand and fill the horizontal direction.

6. The method of claim 1, wherein, The specific steps of step S3 include: S31. Perform vertical projection analysis on the binary image and determine the column with the sum of the column pixel values ​​greater than the column gap threshold as the column blank area. S32. Define consecutive blank column regions with a width greater than the minimum column gap threshold as column separation regions; where the column gap threshold is set based on the font size. S33. Draw vertical dividing lines at the center of each column's dividing area.

7. The method of claim 1, wherein, Step S4 includes: Using an alpha blending method, row labels are overlaid on the original image in a semi-transparent manner, and vertical dividing lines are overlaid on the original image in a high-contrast color to construct a gridded visual framework.

8. A computer apparatus comprising a memory, a processor, and a computer program stored on the memory, wherein the computer program, when executed by the processor, causes the processor to perform the method of any one of claims 1 to 7. The processor executes the computer program to implement the steps of the method according to any one of claims 1-7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the steps of the method described in any one of claims 1-7.

10. A computer program product, characterised in that, When executed by a processor, the computer program implements the steps of the method described in any one of claims 1-7.

Images