Table generation method and generation system

By matching and adjusting the table configuration to generate target styles, the problem of incomplete labeling of table data sets in the prior art is solved. The generated tables conform to real scenarios, and the quality and diversity of deep learning training sets are improved.

CN119067085APending Publication Date: 2024-12-03NANJING WUDAO ZHIXIN INFORMATION TECH CO LTD
View PDF 0 Cites 2 Cited by

Patent Information

Application Number
CN202411107225.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-13
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

The labeling information of existing large-scale table data sets is incomplete and error-free, and deep learning methods rely on a large amount of labeling data, making it difficult to guarantee the quality and diversity of tables and to build high-quality training sets.

Method used

By obtaining the input table, matching its category, selecting the table configuration, and randomly adjusting the attributes within a certain range to generate target styles to generate a table that matches the real scene, which is used to build a deep learning training set.

Benefits of technology

The generated table conforms to the table type of real scenes, has similar semantic features and layout features, avoiding the lack of flexibility and authenticity problems caused by random generation, and improving the quality and diversity of the training set.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119067085A_ABST
    Figure CN119067085A_ABST
Patent Text Reader

Abstract

The invention discloses a table generation method and system, and belongs to the technical field of table generation, and the method comprises the following steps: obtaining a first table; matching the table category of the first table; selecting a first configuration from one or more table configurations of the table category, the table configuration comprising a table attribute; according to the first configuration, randomly adjusting numerical value attributes in a first threshold range to obtain a plurality of target styles; and generating a second table according to the target style. Table configuration is matched and selected according to the first table, tables of the same category have similar semantic features and layout features, the table configuration is made to be close to a real scene table of a table type to which the first table belongs, and on the basis, a target style is generated in a mode of randomly adjusting attributes within a certain range; as the first table is a real bezel table, the generated second table conforms to the real scene of the table category of the first table.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of table generation, and in particular to a table generation method and generation system. Background Art

[0002] As an important carrier of data, tables are widely used in various rich-format documents, which can store and display data compactly and clearly. Automatically extracting table information from massive documents and storing it as structured data can greatly improve the efficiency of information acquisition and utilization.

[0003] With the development of deep learning in recent years, table recognition has made significant progress. Deep learning methods can better handle complex table structures and diverse styles, but they rely on large-scale data sets. Different deep learning methods or businesses require different table annotation information. The annotation of data sets requires a lot of work and is prone to inconsistent annotation standards.

[0004] Most existing large-scale datasets have incomplete annotation information or even erroneous information, and rely on data sources that provide file structures. The quality and diversity of the tables in these large-scale datasets cannot be guaranteed. Constructing tabular datasets by synthesizing or generating tables has become an important development direction. Summary of the invention

[0005] In view of the above technical problems existing in the prior art, the present invention provides a table generation method and a generation system, which generates a second table by matching a target style for an input first table.

[0006] The present invention discloses a table generation method, comprising the following steps: obtaining a first table; matching a table category of the first table; selecting a first configuration from one or more table configurations of the table category, wherein the table configuration includes table attributes; according to the first configuration, randomly adjusting numerical attributes within a first domain value range to obtain multiple target styles; and generating a second table according to the target style.

[0007] Preferably, the method for obtaining the table configuration includes:

[0008] Get the table set;

[0009] Clustering the table set to obtain table categories;

[0010] Get table properties of the table class;

[0011] One or more table configurations are established according to the table attributes.

[0012] Preferably, the method for obtaining the table category and the table configuration includes:

[0013] Clustering the third tables in the table set according to the text information of the third table in the target scene, and summarizing and checking to obtain table categories;

[0014] Statistics are collected on table attributes of the third table of the table category, and at least one table configuration is established.

[0015] Preferably, the table attributes include text block attributes, cell attributes and border attributes.

[0016] The text block attributes include text line attributes and text line relationships; the text line information includes font type, font style, font size and text color, and the text line relationship includes line spacing and alignment of text lines in the text block;

[0017] The cell attributes include padding, background color, and alignment of the text block in the cell;

[0018] The border attributes include border display, border style and border color.

[0019] Preferably, the method for generating the target style includes:

[0020] According to the text line, a text block is obtained, and the relative coordinates of the text line in the text block are obtained;

[0021] According to the text block and the text block alignment setting, obtaining the relative coordinates of the text block boundary box and the text block in the text block boundary box;

[0022] According to the padding of the cell, obtain the relative coordinates of the text block bounding box in the cell;

[0023] Get the size of the table based on the cell coordinates;

[0024] Based on the cell coordinates and the size of the table, calculate the absolute coordinates of the text line in the table and the cell border line coordinates.

[0025] Preferably, the cells include: non-merged cells, row-merged cells only, column-merged cells only, row-merged cells and column-merged cells.

[0026] Non-merged cells are cells without merges;

[0027] The row-only merged cells are cells that are merged only in the vertical direction;

[0028] The column-only merged cells are cells that are merged only in the horizontal direction;

[0029] The row-merged cells are cells merged in the vertical direction, including cells merged in rows only and cells merged in rows and columns;

[0030] The column-merged cells are cells merged in the horizontal direction, including cells merged in columns only and cells merged in both rows and columns;

[0031] The text block includes at least one text line, and the method for obtaining the relative coordinates of the text line in the text block includes:

[0032] Calculate the width and height of the text line according to the text line attributes;

[0033] According to the height of the text line and the line spacing of the text line, obtain the height of the text block and the first relative vertical coordinate y1 of the text line in the text block;

[0034] Calculate the width of the text block according to the width and alignment of the text line, and obtain the first relative horizontal coordinate x1 of the text line in the text block;

[0035] Specifically, the first cell includes a non-merged cell or a column-merged cell only, and the method for calculating the relative vertical coordinate of the text block in the first cell in the text block boundary box includes:

[0036] Get the rows of a table in order;

[0037] The maximum height of the text block in the first cell of the current row is used as the height of the text block bounding box of the first cell of the current row;

[0038] Calculate the second relative vertical coordinate y2 of the text block in the first cell of the current row in the text block bounding box according to the height of the text block, the height of the text block bounding box, and the top and bottom alignment settings of the text block;

[0039] Calculate the relative vertical coordinate of the text block bounding box in the first cell: calculate the height of the first cell and the third relative vertical coordinate y3 of the text block bounding box in the first cell according to the height of the text block bounding box and the upper and lower margin settings of the cell padding;

[0040] Specifically, the second cell includes a non-merged cell or a row-merged cell only, and the method for calculating the relative horizontal coordinate of the text block in the second cell in the text block boundary box includes:

[0041] Get the columns of a table in order;

[0042] The maximum width of the text block in the second cell of the current column is used as the width of the text block bounding box of the second cell of the current column;

[0043] Calculate the second relative horizontal coordinate x2 of the text block in the second cell of the current column in the text block bounding box according to the width of the text block, the width of the text block bounding box, and the left-right alignment setting of the text block;

[0044] Calculate the relative horizontal coordinate of the text block bounding box in the second cell: calculate the width of the second cell and the third relative horizontal coordinate x3 of the text block bounding box in the second cell according to the width of the text block bounding box and the left and right margin settings of the cell padding;

[0045] Specifically, the method for calculating the relative horizontal coordinates of the text block boundary box in the cell of the merged column includes:

[0046] Get column merged cells in order;

[0047] Get the width of the merged cell in the column according to the width of the second cell in the column to which the merged cell in the current column belongs;

[0048] According to the width of the column merged cell and the left and right margin settings of the inner margin, the width of the text block bounding box and the fourth relative horizontal coordinate x4 of the text block bounding box in the column merged cell in the cell are calculated;

[0049] Specifically, the method for calculating the relative vertical coordinate of the text block boundary box in the row merged cell includes:

[0050] Get row merged cells in order;

[0051] Get the height of the merged cell in the row according to the height of the first cell in the row to which the merged cell in the current row belongs;

[0052] According to the height of the row merged cell and the upper and lower margin settings of the inner margin, the height of the text block bounding box and the fourth relative vertical coordinate y4 of the text block bounding box in the row merged cell in the cell are calculated;

[0053] Specifically, the method for calculating the relative horizontal coordinate of the text block in the text block boundary box in the column merged cell includes:

[0054] According to the width of the text block, the width of the text block bounding box, and the left and right alignment settings of the text block, calculate the fifth relative horizontal coordinate x5 of the text block in the text block bounding box in the merged cell of the column;

[0055] Specifically, the method for calculating the relative vertical coordinate of the text block in the row merged cell in the text block boundary box includes:

[0056] The fifth relative vertical coordinate y5 of the text block in the text block bounding box in the row merged cell is calculated according to the height of the text block, the height of the text block bounding box, and the top and bottom alignment settings of the text block.

[0057] Preferably, the method for generating the second table includes:

[0058] Stack each cell to obtain the size of the table and the relative coordinates of the cells in the table;

[0059] Connect the border lines of cells in the same row and column to obtain the absolute coordinates of the row and column dividing lines;

[0060] Calculate the absolute coordinates of the text line in the table according to the relative coordinates of the text line in the text block, the relative coordinates of the text block in the text block bounding box, the relative coordinates of the text block bounding box in the cell, and the relative coordinates of the cell in the table;

[0061] A second table is generated according to the absolute coordinates of the row and column dividing lines and the absolute coordinates of the text lines in the table.

[0062] Preferably, the second table is used to construct a training set for deep learning.

[0063] The method for drawing the second table includes: establishing a blank canvas; establishing a background color; drawing the text line according to the coordinates of the text line in the table, and the font and color of the text line;

[0064] Draw a border line according to the border coordinates, style, line type and color.

[0065] Preferably, the method for obtaining the first table parameter includes:

[0066] Detecting line segments in the PDF file or image of the first form;

[0067] Filter and splice the valid border lines in the first table page, and calculate the table border and cell border;

[0068] According to the table border, capture the three-channel high-resolution original image of the table;

[0069] Calculate the logical structure of the table based on the cell borders;

[0070] Identify lines of text within table borders.

[0071] The present invention also provides a generation system for implementing the above table generation method, comprising an input module, a matching module, a configuration module, a style generation module and a drawing module.

[0072] The input module is used to obtain a first table;

[0073] The matching module is used to match the table category of the first table;

[0074] The configuration module is used to select a first configuration from one or more table configurations of the table category;

[0075] The style generation module is used to randomly adjust the attributes within a first domain value range according to the first configuration to obtain multiple target styles;

[0076] The drawing module is used to generate a second table according to the target style.

[0077] Compared with the prior art, the beneficial effects of the present invention are: according to the first table matching and selecting table configuration, tables of the same category have similar semantic features and layout features, so that the table configuration is close to the real scene table of the table type to which the first table belongs. On this basis, the target style is generated by randomly adjusting the attributes within a certain range. Since the first table is a real bordered table, the generated second table conforms to the real scene of the table category of the first table and is close to the table style of the target application field. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] Figure 1 is a flow chart of the table generation method of the present invention;

[0079] Figure 2 is an identification diagram of the first table;

[0080] Figure 3 is an example of the first table category;

[0081] Figure 4 is another example of the first table category;

[0082] Figure 5 is an example of the second table category;

[0083] Figure 6 is an example of the third table category;

[0084] Figure 7 This is the style of the first and second tables;

[0085] Figure 8 This is the style of the second second table;

[0086] Fig. 9 This is the style of the third second table;

[0087] Fig.10 It is the style of the fourth second table;

[0088] Fig.11 is a logic block diagram of a generation system of the present invention;

[0089] Fig.12 It is a schematic diagram of cell classification. DETAILED DESCRIPTION

[0090] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0091] The present invention is further described in detail below in conjunction with the accompanying drawings:

[0092] The first aspect of the present invention provides a table generation method, such as Figure 1 As shown, the following steps are included:

[0093] Step 101: Obtain a first table.

[0094] The first table can come from a PDF file or an image.

[0095] Step 102: Match the table type of the first table.

[0096] The table category reflects the application scenario of the table. The table category is obtained by clustering the third table, and the first table is classified to match the category.

[0097] Step 103: Select a first configuration from one or more table configurations of the table category, wherein the table configuration includes table attributes.

[0098] The table configuration is a preset or pre-generated standard table attribute.

[0099] Step 104: According to the first configuration, randomly adjust the numerical attributes within the first domain value range, and calculate the table attributes to obtain multiple target styles.

[0100] For example, the attribute may be randomly adjusted within a range of 10%.

[0101] Step 105: Generate a second table according to the target style.

[0102] According to the first table, the table configuration is matched and selected to make the table configuration conform to the real scene of the table category of the first table. On this basis, the target style is generated by randomly adjusting the attributes within a certain range. The generated second table is closer to the real scene of the category to which the first table belongs, the style is close to the third table, and the second table is more realistic. The defects of the lack of flexibility and authenticity of the training set caused by the random generation of random tables are avoided.

[0103] Step 101 also includes a method for identifying parameters of the first table:

[0104] Step 201: Detect line segments in a PDF file or image of a first table.

[0105] Among them, PDF files can use pdfplumber, pdfminer.six and other pdf parsing tools to read the line segments or continuous points and related elements in the x-objects in the PDF page, and calculate the set of all horizontal and vertical lines in the page. Line segment detection algorithms can be used to detect horizontal and vertical lines in images, such as HoughLines, HoughLinesP, LSD, Wireframe, LCNN, PPGNet, TP-LSD, M-LSD, LETR, and line segment detection algorithms based on traditional computer vision technology and deep learning technology such as unet.

[0106] Step 202: Filter and splice the valid border lines in the first table page, and calculate the table border (bounding box) and cell border.

[0107] Step 203: According to the table border, capture the three-channel high-resolution original image of the table.

[0108] Step 204: Calculate the logical structure of the table based on the cell borders, such as logical structures in the styles of HTML, OTSL matrix, etc., but not limited thereto.

[0109] Step 205: Identify text lines within the table border.

[0110] PDF parsing tools such as pdfplumber and pdfminer.six can be used to read the character-related elements in the x-objects of the boundingbox range of the table in the PDF page, and align the text line bounding box and the corresponding text content in each cell. OCR technology can be used to detect the text line bounding box in each cell and recognize the corresponding text content, such as: text detection algorithms: DBNet, DBNet++, EAST, PSENet, etc.; text recognition algorithms: CRNN, SVTR, etc.

[0111] Figure 2The recognition result of the first table in an audit report is shown. The annotation information that needs to be saved for the table includes: the original electronic document of the page to which the table belongs (if any), high-resolution table image, such as PNG, 300DPI; logical structure: HTML, OTSL matrix, etc.; cell information includes (needs to correspond to the cells in the logical structure): cell bounding box: gray border in the figure (only exists in tables with complete border lines); text line bounding box: background color area in the figure; text line text content: text content corresponding to each text line bounding box.

[0112] In step 103, the method for obtaining the table category and its table configuration includes:

[0113] Step 301: Obtain a table set. Real tables in multiple scenarios may be collected, and the table set includes multiple third tables.

[0114] The layout of the PDF document or image in the target scene can be analyzed to obtain the location of the table; at the same time, it can be determined whether the table has a complete border or no border; then the tables are clustered and checked and summarized to obtain the classification of the tables. Tables of the same type have similar semantic features and layout features.

[0115] Step 302: Detect table parameters of the third table and extract text.

[0116] Step 303: clustering the table set to obtain table categories.

[0117] More specifically, according to the text information of the third table in the target scene, the third table in the table set is clustered, and the table category is obtained by inductive inspection.

[0118] Step 304: Obtain table attributes of the table category.

[0119] Step 305: Establish one or more table configurations according to the table attributes.

[0120] More specifically, statistics are collected on the table attributes of the third table of the table category, and at least one table configuration is established, that is, a configuration list is established.

[0121] Observe the various styles of each type of table in the target scene and count the actual distribution, collect tables of the same type but with different styles by type, and then count the attribute parameters of different styles and combine them into table parameter configurations. Each type can have multiple configurations as the candidate configuration list of each type.

[0122] The first input table is classified to match the corresponding table category, and a table configuration is selected from the corresponding candidate configuration list according to the matching category. For the attributes of the numerical type in the table configuration, random adjustments are made within a certain range, such as randomly selecting a numerical value as the attribute value within a range of 10% above and below the original attribute value, so as to obtain a configuration of the target style.

[0123] Figure 3 and Figure 4 A first table category is shown; Figure 5 and Figure 6 The second table category and the third table category are shown respectively, but are not limited to this.

[0124] The elements of a table include text, cells, borders, etc. The table attributes include text block attributes, cell attributes, and border attributes.

[0125] The text block attributes include text row information, text row relationship and text block relationship, which can be set and configured in units of table / row / column / cell. The text row information includes font type, font style, font size and text color.

[0126] The text row relationship includes line spacing and alignment of text rows in text blocks, such as horizontal alignment. The cell attributes include padding, background color and alignment of text blocks in cells, such as horizontal alignment and vertical alignment. Horizontal alignment includes left alignment, right alignment, center alignment, and fixed distances of left and right indents, and vertical alignment includes top alignment, bottom alignment, center alignment, and fixed distances of top and bottom indents. Padding is set in units of table / row / column. The top padding (padding-top) and bottom padding (padding-bottom) of non-merged cells are set by row, and the padding-top of row merged cells is the padding-top of the starting row, and the padding-bottom is the padding-bottom of the ending row. The padding-left and padding-right of non-merged cells are set by column, and the padding-left of column merged cells is the padding-left of the starting column, and the padding-right is the padding-right of the ending column.

[0127] The border attributes include border display, border style and border color. Border display includes outer border display and inner border display. Common ways of outer border display (set in units of tables) are: complete display, missing left and right, missing top and bottom, and missing all. Inner border display (set in units of tables / rows / columns / cells) is: display by rows and columns: complete display, missing horizontal lines, missing vertical lines, missing all, partially missing horizontal lines, partially missing vertical lines, split rows (headers, totals) to display the bottom border line, etc.; display by cell: merge cells to display the bottom border line, display randomly in proportion, display lines of different styles, etc.; and combined display: indent rows to display the edge border lines after aggregation, etc.

[0128] Border styles include outer border styles and inner border styles. Outer border styles (set in table units) include: single solid line, overlapping solid lines, line thickness, etc. Inner border styles (set in table / row / column / cell units) include: solid line, dotted line, dashed line, line thickness, dashed line gap, etc. Border colors also include outer border colors (set in table units) and inner border colors (set in table / row / column / cell units).

[0129] In step 104, the method of generating the target style includes:

[0130] Step 401: Obtain a text block according to a text line, and obtain the relative coordinates of the text line in the text block.

[0131] A text block consists of multiple texts with preset line spacing and alignment.

[0132] Step 402: According to the text block and the text block alignment setting, obtain a text block bounding box (aligned textblock bounding box) and the relative coordinates of the text block in the text block bounding box.

[0133] The relative vertical coordinate of the text block in the text block bounding box is calculated according to the height of the text block, the height of the text block bounding box, and the top and bottom alignment settings of the text block; the relative horizontal coordinate of the text block in the text block bounding box is calculated according to the width of the text block, the width of the text block bounding box, and the left and right alignment settings of the text block.

[0134] Step 403: Obtain the relative coordinates of the cell and the text block boundary box in the cell according to the inner margin of the cell. For example, obtain the relative vertical coordinate of the text block boundary box in the cell according to the upper and lower margins, and obtain the relative horizontal coordinate of the text block boundary box in the cell according to the left and right margins.

[0135] Step 404: Obtain the relative coordinates of the table and the cells in the table.

[0136] Step 405: Calculate the absolute coordinates of the text line in the table and the absolute coordinates of the cell border line based on the relative coordinates of the text line in the text block, the relative coordinates of the text block in the text block boundary box, the relative coordinates of the text block boundary box in the cell, and the relative coordinates of the cell in the table.

[0137] Implement bottom-up table recognition or synthesis.

[0138] More specifically, the cells include: non-merged cells, row-only merged cells, column-only merged cells, row-merged cells, and column-merged cells.

[0139] Non-merged cells are cells without merges;

[0140] The row-only merged cells are cells that are merged only in the vertical direction;

[0141] The column-only merged cells are cells that are merged only in the horizontal direction;

[0142] The row-merged cells are cells merged in the vertical direction, including cells merged in rows only and cells merged in rows and columns;

[0143] The column-merged cells are cells merged in the horizontal direction, including cells merged in columns only and cells merged in both rows and columns.

[0144] The text block includes at least one text line. The method for obtaining the relative coordinates of the text line in the text block in step 401 includes the following steps:

[0145] Step 411: Calculate the width and height of the text line according to the text line information or attributes.

[0146] If you use image drawing to generate text lines: use the Pillow tool's ImageFont to load the ttf / ttc / otf font, and call the getbbox() function to calculate the size of the specified text in the specified font size (that is, the width and height of the text line's boundingbox). If you use the screenshot method of the text line in the original image: use the Pillow tool's crop function to make a screenshot of the text line on the original image, and scale it proportionally to the specified synthetic font size.

[0147] Step 412: Obtain the height of the text block and the first relative vertical coordinate y1 of the text line in the text block according to the height of the text line and the line spacing of the text line.

[0148] If there are multiple text lines in a cell: the total height of the text block and the y coordinates of each text line in the text block (the upper left corner is the coordinate origin) can be calculated based on the defined text line spacing; the total width of the text block and the x coordinates of each text line in the text block (the upper left corner is the coordinate origin) can be calculated based on the defined text line alignment. If the cell is a single-line text: the text line height is the text block height, and the text line width is the text block width.

[0149] Step 413: Calculate the width of the text block according to the width and alignment of the text line, and obtain the first relative horizontal coordinate x1 of the text line in the text block.

[0150] The first relative abscissa and the first relative ordinate are combined into the relative coordinate of the text line in the text block.

[0151] Fig.12 Non-merged cells and different types of merged cells are shown. The first cell includes non-merged cells or only column-merged cells, and the first cell has only one row in the horizontal direction, so the relative vertical coordinates can be calculated using the same method. The method for calculating the relative vertical coordinates of the text block in the first cell in the text block boundary box includes the following steps:

[0152] Step 420: Obtain the rows of the table in order. The rows of the table may be obtained by scanning row by row.

[0153] Step 421: The maximum height of the text block in the first cell of the current row is used as the height of the text block boundary box of the first cell of the current row.

[0154] Step 422: Calculate the second relative vertical coordinate y2 of the text block in the first cell of the current row in the text block boundary box according to the height of the text block, the height of the text block boundary box, and the top and bottom alignment settings of the text block.

[0155] Step 423: Calculate the relative vertical coordinate of the text block boundary box in the first cell:

[0156] According to the height of the text block bounding box and the upper and lower margin settings of the cell padding, the height of the first cell and the third relative vertical coordinate y3 of the text block bounding box in the first cell are calculated.

[0157] The second cell includes a non-merged cell or a row-merged cell, which has only one column in the vertical direction, so the horizontal relative coordinate can be calculated in the same way. The method for calculating the relative horizontal coordinate of the text block in the second cell in the text block boundary box includes the following steps:

[0158] Step 430: Obtain the columns of the table in order. The columns of the table may be obtained by scanning column by column.

[0159] Step 431: The maximum width of the text block in the second cell of the current column is used as the width of the text block boundary box of the second cell of the current column.

[0160] Step 432: Calculate the second relative horizontal coordinate x2 of the text block in the second cell of the current column in the text block boundary box according to the width of the text block, the width of the text block boundary box, and the left-right alignment setting of the text block.

[0161] Step 433: Calculate the relative horizontal coordinate of the text block bounding box in the second cell:

[0162] According to the width of the text block bounding box and the left and right margin settings of the cell padding, the width of the second cell and the third relative horizontal coordinate x3 of the text block bounding box in the second cell are calculated.

[0163] For non-merged cells, when the first cell and the second cell point to the same cell, the second relative vertical coordinate y2 and the second relative horizontal coordinate x2 are combined as the relative coordinates of the text block in the text block bounding box, and the third relative vertical coordinate y3 and the third relative horizontal coordinate x3 are combined as the relative coordinates of the text block bounding box in the cell.

[0164] For column merged cells, methods for calculating the relative horizontal coordinates of the text block bounding box in the column merged cells include:

[0165] Step 440: Get column merged cells in order.

[0166] Step 441: Obtain the width of the merged cell in the column according to the width of the second cell in the column to which the merged cell in the current column belongs.

[0167] Step 442: Calculate the width of the text block boundary box and the fourth relative horizontal coordinate x4 of the text block boundary box in the column merged cell according to the width of the column merged cell and the left and right margin settings of the inner margin.

[0168] Accordingly, for a row-merged cell, a method for calculating the relative vertical coordinate of the text block bounding box in the row-merged cell in the cell includes:

[0169] Step 450: Get row merged cells in order.

[0170] Step 451: Obtain the height of the row merged cell according to the height of the first cell in the row to which the current row merged cell belongs.

[0171] Step 452: Calculate the height of the text block boundary box and the fourth relative vertical coordinate y4 of the text block boundary box in the row-merged cell according to the height of the row-merged cell and the top and bottom margin settings of the inner margin.

[0172] Specifically, the method for calculating the relative horizontal coordinate of the text block in the text block boundary box in the column merged cell includes:

[0173] The fifth relative horizontal coordinate x5 of the text block in the text block bounding box in the merged cell of the column is calculated according to the width of the text block, the width of the text block bounding box, and the left and right alignment settings of the text block.

[0174] Specifically, the method for calculating the relative vertical coordinate of the text block in the row merged cell in the text block boundary box includes:

[0175] The fifth relative vertical coordinate y5 of the text block in the text block bounding box in the row merged cell is calculated according to the height of the text block, the height of the text block bounding box, and the top and bottom alignment settings of the text block.

[0176] x1 and y1 constitute the relative coordinates of the text lines in all cells in the text block; x2, x5 and y2, y5 constitute the relative coordinates of the text block in the text block bounding box; x3, x4 and y3, y4 constitute the relative coordinates of the text block bounding box in the cell.

[0177] In step 105, the method for generating the second table includes:

[0178] Step 501: stack the cells to obtain the size of the table and the relative coordinates of the cells in the table.

[0179] Step 502: Connect the border lines of cells in the same row and column to obtain the absolute coordinates of the row and column dividing lines.

[0180] Step 503: Calculate the absolute coordinates of the text line in the table based on the relative coordinates of the text line in the text block, the relative coordinates of the text block in the text block boundary box, the relative coordinates of the text block boundary box in the cell, and the relative coordinates of the cell in the table.

[0181] Step 504: Generate a second table according to the absolute coordinates of the row and column dividing lines and the absolute coordinates of the text lines in the table.

[0182] The method of drawing the second table includes:

[0183] Step 505: Create a blank canvas.

[0184] Step 506: Establish background color.

[0185] Step 507: Draw the text line according to the coordinates of the text line in the table, and the font and color of the text line.

[0186] You can use the ImageDraw tool in Pillow to draw text lines. You can also take a screenshot of the text lines in the original image and paste it, then set thresholds based on pixel values ​​to modify the text color and background color in the screenshot to match the table template.

[0187] Step 508: Draw a border line according to the border coordinates, style (mode), line type (line type) and color (color).

[0188] You can use Pillow or OpenCV tools to draw the border line; you can also take a screenshot of the border line elements in the original image and paste them.

[0189] In a specific drawing, Figure 2 The original image is the first table, and multiple second tables are drawn. The styles of some second tables are as follows Figure 7 , Figure 8 , Fig. 9 and Fig.10 shown.

[0190] The second aspect of the present invention also provides a generation system for implementing the above method, such as Fig.11 As shown, it includes an input module 1, a matching module 2, a configuration module 3, a style generation module 4 and a drawing module 5.

[0191] The input module 1 is used to obtain a first table.

[0192] The matching module 2 is used to match the table category of the first table. The table category can be matched according to the logical structure and / or text of the first table.

[0193] The configuration module 3 is used to select a first configuration from one or more table configurations of the table category.

[0194] The style generation module 4 is used to randomly adjust attributes within a first domain value range according to the first configuration to obtain multiple target styles.

[0195] The drawing module 5 is used to generate a second table according to the target style.

[0196] The system of the present invention further comprises a table detection module 7 , a table classification module 8 and a configuration generation module 9 .

[0197] The table detection module 7 is used to detect table parameters of the second table and extract text.

[0198] The table classification module 8 is used to classify the table set to obtain multiple table categories.

[0199] The configuration generation module 9 is used to establish one or more table configurations according to table attributes of the table category.

[0200] In a specific embodiment, nearly 1.5 million tables were detected and extracted from 5049 annual reports of Chinese listed companies. Most of these tables are bordered, and a few are borderless. From these nearly 1.5 million tables, approximately the same number of tables as the English financial announcement dataset FinTabNet were extracted, a total of 105,600 bordered tables, as a table set. In addition, 50% of the tables in the table set were converted to borderless tables. The borderless first table of the audit report was used as the source table to generate the second table and its annotated data as a training set, of which 50% of the training set was the unconverted first table. See Table 1 for the distribution of merged cells in the table set.

[0201] Table 1

[0202] Merged Cell Distribution quantity Proportion Table without merged cells 52208 49.44% Table with 1 merged cell 4272 4.04% Table with 2 merged cells 13973 13.23% Table with 3 merged cells 15582 14.76% Table with more than 4 merged cells 19565 18.53%

[0203] TableMaster, an end-to-end model based on deep learning, is trained on the training set and the FinTabNet training set respectively to obtain the first table recognition model and the second table recognition model. 2290 actual tables were extracted from Chinese financial announcements as a test set to ensure that there is no overlap with the 105,600 tables sampled previously, as a source of comprehensive table structure and content. Among them, 1000 are bordered tables and 1290 are borderless tables, aiming to increase the focus on borderless tables. The selection of tables also prioritizes more challenging tables, including multiple merges or cross-cells.

[0204] The test set was identified and annotated using the table recognition model, respectively, and then manually verified.

[0205] Table 2 shows the validation results of the first table recognition model trained with the table set.

[0206] Table 2

[0207]

[0208] Among them, TEDS (Tree-Edit-Distance-based Similarity Metric) is a similarity evaluation standard based on tree edit distance. TEDS-Struct is a modified version of TEDS, which only calculates the edit distance of the structure, and does not participate in the calculation of text content. AP-50 (AP50, MS COCO AP at IoU = .50) is a commonly used evaluation standard for target detection. The calculation of Average Precision uses 0.5 as the IoU (Intersection over Union) threshold.

[0209] The second table recognition model trained on the FinTabNet training set is validated using the revised FinTabNet test set (9,681 tables). The validation results are shown in Table 3.

[0210] Table 3

[0211]

[0212] Comparing Table 2 with Table 3 (i.e., the comparison of models trained with the Chinese training set and the FinTabNet training set), the performance is slightly reduced. The reason is that the tables in Chinese financial announcements are more complex, such as containing more cells with multi-line text blocks, cells with very long text, or cells with a higher density.

[0213] Compared with Table 3, Table 5 shows that the original FinTabNet training set contains relatively few tables and has multiple generated cells. While expanding the data set while keeping the total number of tables roughly the same, a portion of the enhanced data set is directly obtained by sampling from the original FinTabNet training set, and the partial tables are synthesized using the method of the present invention to form a second training set. The distribution of the tables in the second training set is shown in Table 4; the third table recognition model is obtained after training the second training set, and the test results of the third table recognition model are shown in Table 5.

[0214] Table 4

[0215]

[0216] Table 5

[0217]

[0218] It can be seen that the data enhancement of the original training set by the method of the present invention can comprehensively improve the recognition performance of the table recognition model, especially when recognizing more complex tables with multiple span cells. This proves the effectiveness of the method of the present invention in practical application scenarios.

[0219] The present invention selects a real table with a complete border in an electronic document or a picture as the input data source, and selects the setting of the conversion style according to the actual distribution of tables of various styles in the target document, so that the table set is close to the real scene, the logical structure and text information of the input table are retained, and the conversion of the input table to the target synthetic table of the set style is convenient and flexible. By counting the element composition and style attribute range of the real table, the generated table meets the visual characteristics of the real scene table, making it look natural and realistic, avoiding the flexibility and authenticity lacking in the style of the synthetic table generated by the previous method. Without the limitation of HTML rendering, any possible visual effect of the table can be achieved in theory (the distortion caused by taking pictures or scanning needs to be further enhanced), so that the style of the table is more realistic and rich.

[0220] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for generating a table, characterized in that: The following steps are involved: Get the first table; a table category matching the first table; selecting a first configuration from one or more table configurations of the table category, the table configuration comprising table attributes; According to the first configuration, the numerical attribute is randomly adjusted within the first domain value range to obtain multiple target styles; A second table is generated according to the target style.

2. The table generation method according to claim 1, characterized in that: Methods for obtaining table configuration include: Get the table set; Clustering the table set to obtain table categories; Get table properties of the table class; One or more table configurations are established according to the table attributes.

3. The table generation method according to claim 2, characterized in that: Methods for obtaining table categories and table configurations include: Clustering the third tables in the table set according to the text information of the third table in the target scene, and obtaining the table categories by summarizing and checking; Statistics are collected on table attributes of the third table of the table category, and at least one table configuration is established.

4. The table generation method according to claim 2, characterized in that: The table attributes include text block attributes, cell attributes and border attributes. The text block attributes include text line attributes and text line relationships; the text line information includes font type, font style, font size and text color, and the text line relationship includes line spacing and alignment of text lines in the text block; The cell attributes include padding, background color, and alignment of text blocks in the cell; The border attributes include border display, border style and border color.

5. The table generation method according to claim 4, characterized in that: Methods for generating target styles include: According to the text line, a text block is obtained, and the relative coordinates of the text line in the text block are obtained; According to the text block and the text block alignment setting, obtaining the relative coordinates of the text block boundary box and the text block in the text block boundary box; According to the padding of the cell, obtain the relative coordinates of the text block bounding box in the cell; Get the relative coordinates of the table and cells in the table; The absolute coordinates of the text line in the table and the absolute coordinates of the cell border line are calculated based on the relative coordinates of the text line in the text block, the relative coordinates of the text block in the text block bounding box, the relative coordinates of the text block bounding box in the cell, and the relative coordinates of the cell in the table.

6. The table generation method according to claim 5, characterized in that: The cells include: non-merged cells, row-merged cells only, column-merged cells only, row-merged cells and column-merged cells. Non-merged cells are cells without merges; The row-only merged cells are cells that are merged only in the vertical direction; The column-only merged cells are cells that are merged only in the horizontal direction; The row-merged cells are cells merged in the vertical direction, including cells merged in rows only and cells merged in rows and columns; The column-merged cells are cells merged in the horizontal direction, including cells merged in columns only and cells merged in both rows and columns; The text block includes at least one text line, and the method for obtaining the relative coordinates of the text line in the text block includes: Calculate the width and height of the text line according to the text line attributes; According to the height of the text line and the line spacing of the text line, obtain the height of the text block and the first relative vertical coordinate y1 of the text line in the text block; Calculate the width of the text block according to the width and alignment of the text line, and obtain the first relative horizontal coordinate x1 of the text line in the text block; Specifically, the first cell includes a non-merged cell or a column-merged cell only, and the method for calculating the relative vertical coordinate of the text block in the first cell in the text block boundary box includes: Get the rows of a table in order; The maximum height of the text block in the first cell of the current row is used as the height of the text block bounding box of the first cell of the current row; Calculate the second relative vertical coordinate y2 of the text block in the first cell of the current row in the text block bounding box according to the height of the text block, the height of the text block bounding box, and the top and bottom alignment settings of the text block; Calculate the relative vertical coordinate of the text block bounding box in the first cell: calculate the height of the first cell and the third relative vertical coordinate y3 of the text block bounding box in the first cell in the cell according to the height of the text block bounding box and the upper and lower margin settings of the cell padding; Specifically, the second cell includes a non-merged cell or a row-merged cell only, and the method for calculating the relative horizontal coordinate of the text block in the second cell in the text block boundary box includes: Get the columns of a table in order; The maximum width of the text block in the second cell of the current column is used as the width of the text block bounding box of the second cell of the current column; Calculate the second relative horizontal coordinate x2 of the text block in the second cell of the current column in the text block bounding box according to the width of the text block, the width of the text block bounding box, and the left-right alignment setting of the text block; Calculate the relative horizontal coordinate of the text block bounding box in the second cell: calculate the width of the second cell and the third relative horizontal coordinate x3 of the text block bounding box in the second cell according to the width of the text block bounding box and the left and right margin settings of the cell padding; Specifically, the method for calculating the relative horizontal coordinates of the text block boundary box in the cell of the merged column includes: Get column merged cells in order; Get the width of the merged cell in the column according to the width of the second cell in the column to which the merged cell in the current column belongs; According to the width of the column merged cell and the left and right margin settings of the inner margin, the width of the text block bounding box and the fourth relative horizontal coordinate x4 of the text block bounding box in the column merged cell in the cell are calculated; Specifically, the method for calculating the relative vertical coordinate of the text block boundary box in the row merged cell includes: Get row merged cells in order; Get the height of the merged cell in the row according to the height of the first cell in the row to which the merged cell in the current row belongs; According to the height of the row merged cell and the upper and lower margin settings of the inner margin, the height of the text block bounding box and the fourth relative vertical coordinate y4 of the text block bounding box in the row merged cell in the cell are calculated; Specifically, the method for calculating the relative horizontal coordinate of the text block in the text block boundary box in the column merged cell includes: According to the width of the text block, the width of the text block bounding box, and the left and right alignment settings of the text block, calculate the fifth relative horizontal coordinate x5 of the text block in the text block bounding box in the merged cell of the column; Specifically, the method for calculating the relative vertical coordinate of the text block in the row merged cell in the text block boundary box includes: The fifth relative vertical coordinate y5 of the text block in the text block bounding box in the row merged cell is calculated according to the height of the text block, the height of the text block bounding box, and the top and bottom alignment settings of the text block.

7. The table generation method according to claim 5, characterized in that: The method of generating the second table includes: Stack each cell to obtain the size of the table and the relative coordinates of the cells in the table; Connect the border lines of cells in the same row and column to obtain the absolute coordinates of the row and column dividing lines; Calculate the absolute coordinates of the text line in the table according to the relative coordinates of the text line in the text block, the relative coordinates of the text block in the text block bounding box, the relative coordinates of the text block bounding box in the cell, and the relative coordinates of the cell in the table; A second table is generated according to the absolute coordinates of the row and column dividing lines and the absolute coordinates of the text lines in the table.

8. The table generation method according to claim 7, characterized in that: The second table is used to construct a training set for deep learning. The method of drawing the second table includes: Create a blank canvas; Establish background color; Draw the text line according to the coordinates of the text line in the table, and the font and color of the text line; Draw a border line according to the border coordinates, style, line type and color.

9. The table generation method according to claim 1, characterized in that: Methods for obtaining the first table parameter include: Detecting line segments in the PDF file or image of the first form; Filter and splice the valid border lines in the first table page, and calculate the table border and cell border; According to the table border, capture the three-channel high-resolution original image of the table; Calculate the logical structure of the table based on the cell borders; Identify lines of text within table borders.

10. A generation system, characterized in that: Used to implement the table generation method according to any one of claims 1 to 8, comprising an input module, a matching module, a configuration module, a style generation module and a drawing module, The input module is used to obtain a first table; The matching module is used to match the table category of the first table; The configuration module is used to select a first configuration from one or more table configurations of the table category; The style generation module is used to randomly adjust the attributes within a first domain value range according to the first configuration to obtain multiple target styles; The drawing module is used to generate a second table according to the target style.

Citation Information

Cited By

  • Table data processing method and device, equipment and storage medium

    CN119785373A

  • Table recognition method and device based on thinking chain, electronic equipment and storage medium

    CN121259854A