Table identification method and apparatus based on cell information, and electronic device

Abstract

The invention provides a table recognition method and device based on cell information and electronic equipment, and relates to the field of table recognition. The method comprises the following steps: acquiring position information of a plurality of cells included in a table; determining four elements of each cell according to the position information of the plurality of cells, the four elements including a row index value, a column index value, a row span value and a column span value; and identifying the structure of the table based on the four elements of each cell. The technical problem that the recognition effect cannot be guaranteed when a deep learning model is used for recognizing table pictures with high complexity in related technologies is solved.

Description

Technical Field

[0001] This disclosure relates to the field of table recognition technology, and more specifically, to a table recognition method, apparatus, and electronic device based on cell information. Background Technology

[0002] Table structure recognition in Optical Character Recognition (OCR) is an important task in the fields of computer vision and document analysis. It involves detecting and parsing the structure of tables from images or documents, including table boundaries, cell divisions, and text content within the table. As part of table understanding, table recognition has wide applications in daily life.

[0003] Currently, the most common method in mainstream table recognition technology is to recognize tables based on the powerful capabilities of various deep learning models. Even so, the recognition effect is often not guaranteed for highly complex table images. Summary of the Invention

[0004] This disclosure provides a table recognition method, apparatus, and electronic device based on cell information, which solves the technical problem in related technologies where the recognition effect cannot be guaranteed when using deep learning models to recognize highly complex table images.

[0005] According to one aspect of the present disclosure, a table recognition method based on cell information is provided. The method includes: obtaining position information of multiple cells included in the table; determining four elements of each cell based on the position information of the multiple cells, wherein the four elements include: row index value, column index value, row span value, and column span value; and recognizing the structure of the table based on the four elements of each cell.

[0006] Optionally, the row span value of each cell in the four elements is determined based on the position information of multiple cells, including: Step S1, determining whether there is a relationship between any two cells in the multiple cells; Step S2, constructing a list of left neighboring cells and a list of right neighboring cells for each cell based on the determination result; Step S3, determining a filter list of left neighboring cells for each cell based on the list of left neighboring cells for each cell, and determining a filter list of right neighboring cells for each cell based on the list of right neighboring cells for each cell; Step S4, determining a list of merged cells based on the filter lists of left neighboring cells and right neighboring cells for each cell, wherein the merged cells in the list of merged cells are cells in the table excluding atomic cells, and atomic cells are cells in the table whose row span value and column span value are both 1; Step S5, determining the merged cell value of each merged cell in the list of merged cells, and using the merged cell value as the row span value of each merged cell.

[0007] Optionally, determining whether there is a relationship between any two cells in a plurality of cells includes at least the following conditions: if the horizontal distance between any two cells is less than a preset distance threshold, it is determined that there is a relationship between any two cells; if the vertical overlap rate between any two cells is greater than a first preset overlap rate threshold, it is determined that there is a relationship between any two cells.

[0008] Optionally, based on the judgment result, a list of left neighboring cells and a list of right neighboring cells are constructed for each cell, including: taking the cell located to the left of each cell and having a relationship with each cell as the left neighboring cell of each cell; and taking the cell located to the right of each cell and having a relationship with each cell as the right neighboring cell of each cell.

[0009] Optionally, the left-neighbor cell filter list for each cell is determined based on the list of left-neighbor cells for each cell, including: determining the rightmost column cell in the list of left-neighbor cells for each cell; filtering out cells without any relationship from the rightmost column cell to form the left-neighbor cell filter list; the right-neighbor cell filter list for each cell is determined based on the list of right-neighbor cells for each cell, including: determining the leftmost column cell in the list of right-neighbor cells for each cell; filtering out cells without any relationship from the leftmost column cell to form the right-neighbor cell filter list.

[0010] Optionally, based on the left-neighbor cell filter list and the right-neighbor cell filter list of each cell, a merged cell list is determined, including: for any target cell among multiple cells, determining a first set consisting of the left-neighbor cell filter list of any cell in the target cell's left-neighbor cell filter list and the target cell's left-neighbor cell filter list; if the length of the left-neighbor cell filter list of each cell in the first set is not greater than 1, the target cell is determined as an atomic cell; otherwise, the target cell is determined as a merged cell; or For any target cell among multiple cells, determine a second set consisting of the right neighbor filter list of any cell in the target cell's right neighbor filter list and the target cell's right neighbor filter list; if the length of the right neighbor filter list of each cell in the second set is not greater than 1, the target cell is determined as an atomic cell; otherwise, the target cell is determined as a merged cell.

[0011] Optionally, determining the merged cell value of each merged cell in the merged cell list includes: determining the right merged cell value and the left merged cell value of each merged cell in the merged cell list respectively; and determining the maximum value between the right merged cell value and the left merged cell value of each merged cell as the merged cell value of each merged cell.

[0012] Optionally, determining the value of the right-hand or left-hand merged cell for each merged cell in the merged cell list includes: step S11, determining the right-hand or left-hand neighboring cell filter list for each merged cell; step S12, if the overlap rate between each merged cell and any cell in its right-hand or left-hand neighboring cell filter list is higher than a second preset overlap rate threshold, proceeding to step S13; otherwise, proceeding to step S14; cumulatively calculating the value of the right-hand or left-hand merged cell for each merged cell; step S13, determining that the value of the right-hand or left-hand merged cell for each merged cell contains the value of the right-hand or left-hand merged cell for any cell; step S14, setting the value of the right-hand or left-hand merged cell for each merged cell to 1; step S15, repeating steps S11 to S14 until the value of the right-hand or left-hand merged cell for each merged cell remains unchanged.

[0013] Optionally, the column span value of each cell in the four elements is determined based on the position information of multiple cells, including: rotating the table 90° counterclockwise and then flipping it horizontally to obtain the inverted cells of the table; performing the above steps S1 to S5 on each cell in the inverted cells to obtain the row span value of each cell in the inverted cells; and determining the row span value of each cell in the inverted cells as the column span value of each cell in the table.

[0014] Optionally, the column index value of each cell in the four elements is determined based on the position information of multiple cells, including: step S111, determining the leftmost column cell of the table, wherein the column index value of each cell in the leftmost column cell is 1; step S112, updating the column index value of each cell in the right neighbor cell filter list of each cell in the leftmost column cell in order from top to bottom, wherein the column index value of each cell in the right neighbor cell filter list of each cell in the leftmost column cell is the sum of the column index value of the leftmost column cell and the column span value; step S113, for the cell whose column index value has been updated, updating the column index value of each cell in its right neighbor cell filter list in order, until the column index value of each cell in the table has been updated.

[0015] Optionally, determining the row index value of each cell's four elements based on the position information of multiple cells includes: rotating the table 90° counterclockwise and then flipping it horizontally to obtain the inverted cells of the table; performing steps S111 to S113 above for each cell in the inverted cells to obtain the column index value of each cell in the inverted cells; and determining the column index value of each cell in the inverted cells as the row index value of each cell in the table.

[0016] According to another aspect of the present disclosure, a table recognition device based on cell information is provided. The device includes: an acquisition module for acquiring position information of a plurality of cells included in a table; a determination module for determining four elements of each cell based on the position information of the plurality of cells, wherein the four elements include: row index value, column index value, row span value, and column span value; and a recognition module for recognizing the structure of the table based on the four elements of each cell.

[0017] According to another aspect of the present disclosure, an electronic device is provided, comprising: a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the above-described method.

[0018] According to another aspect of the present disclosure, a computer-readable storage medium is provided that stores a computer program thereon, which, when executed by a processor, implements the above-described method.

[0019] According to one aspect of the present disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the above-described method.

[0020] The beneficial effects of the technical solution provided in this disclosure are as follows: Based on the positional information of multiple cells in a table, four elements of each cell are determined, including row index, column index, row span, and column span. The table structure is then identified based on these four elements. Without requiring a structure recognition model, table reconstruction can be achieved solely based on cell positional information, thereby enabling the identification of the table structure. Furthermore, the recognition algorithm is computationally simple, highly robust, and adaptable to the recognition of table structures of various complex natures. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of this disclosure, the accompanying drawings used in the description of the embodiments of this disclosure will be briefly introduced below.

[0022] Figure 1 A flowchart illustrating a table recognition method based on cell information provided in this disclosure embodiment; Figure 2 This is a schematic diagram of a table provided in an embodiment of this disclosure; Figure 3 It is a display Figure 2 A diagram illustrating the numbering of each cell in the table shown; Figure 4 This is a flowchart illustrating a process for determining the row span value among the four elements of a cell based on its location information, according to an embodiment of this disclosure. Figure 5 This is a schematic diagram illustrating how to determine the interval between cells according to an embodiment of this disclosure; Figure 6 This is a schematic diagram illustrating the determination of the overlap rate between cells according to an embodiment of this disclosure; Figure 7a This is a schematic diagram illustrating the containment relationship between cells provided in an embodiment of this disclosure; Figure 7b This is a schematic diagram illustrating the overlapping relationship between cells provided in an embodiment of this disclosure; Figure 8 This is a schematic diagram of an inverted cell provided in an embodiment of this disclosure; Figure 9 This is a schematic diagram of the structure of a table recognition device based on cell information provided in an embodiment of the present disclosure; Figure 10 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. Detailed Implementation

[0023] The embodiments of this disclosure are described below with reference to the accompanying drawings. It should be understood that the embodiments described below with reference to the accompanying drawings are exemplary descriptions for explaining the technical solutions of the embodiments of this disclosure, and do not constitute a limitation on the technical solutions of the embodiments of this disclosure.

[0024] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the terms “comprising” and “including” as used in embodiments of this disclosure mean that the corresponding feature can be implemented as the presented feature, information, data, step, operation, element, and / or component, but do not exclude implementation as other features, information, data, step, operation, element, component, and / or combinations thereof supported by the art. It should be understood that when we say that an element is “connected” or “coupled” to another element, the one element can be directly connected or coupled to the other element, or it can mean that the one element and the other element are connected through an intermediate element. Furthermore, “connected” or “coupled” as used herein can include wireless connection or wireless coupling. The term “and / or” as used herein indicates at least one of the items defined by the term, for example, “A and / or B” or “A, B” indicates implementation as “A,” or implementation as “B,” or implementation as “A and B.”

[0025] To make the objectives, technical solutions, and advantages of this disclosure clearer, the embodiments of this disclosure will be described in further detail below with reference to the accompanying drawings.

[0026] First, the technical terms used in this disclosure will be introduced and explained: OCR refers to the process by which electronic devices (such as scanners or digital cameras) examine printed characters on paper, determine their shapes by detecting dark and light patterns, and then translate the shapes into computer text using character recognition methods.

[0027] As part of table understanding, table recognition has wide applications in daily life. In recent years, Transformer architecture models have also been increasingly used in table structure recognition and reconstruction scenarios.

[0028] For example, the core idea of ​​the LayoutLMv3 model is to understand the semantics and structure of a table by jointly modeling text, images, and layout information through a multimodal Transformer architecture. This model predicts cell information (e.g., commas), row-column relationships (e.g., spanning rows and columns), and key-value pairs (e.g., name → Zhang San) based on textual information (text content, etc.), layout information (table, cell information, etc.), and visual information (table images).

[0029] There are also some recent research models based on the Transformer architecture that predict HTML tag strings (tags containing text boxes) and the coordinate information of the text boxes, and match them with third-party text detection and recognition results to obtain the final table recognition result.

[0030] A further approach involves applying ROIAlign (a feature alignment method for object detection and instance segmentation) to the features of the third-party text detection result image, and using this as input to the Transformer architecture model for training and predicting HTML tag strings. Simultaneously, the model also outputs a matching matrix between the text box tags of the HTML tag strings and the third-party text boxes to improve matching accuracy.

[0031] Other technical solutions introduce Optimized Table Structure Language (OTSL) for table structure recognition. By simplifying the markup format, the number of tags for table structure recognition is reduced from more than 28 in HTML to 5 (FELUN), and the average sequence length is shortened to half that of HTML, thus improving the efficiency and accuracy of table structure recognition.

[0032] Currently, among the mainstream table recognition technologies, various deep learning models can effectively recognize tables. However, for highly complex table images, the recognition performance often cannot be guaranteed.

[0033] The methods, apparatuses, electronic devices, and computer-readable storage media for identifying forms provided in this disclosure are intended to solve at least one of the above-mentioned technical problems of the prior art.

[0034] The following description of several exemplary embodiments illustrates the technical solutions of this disclosure and the technical effects produced by these solutions. It should be noted that the following embodiments can be referenced, learned from, or combined with each other. Identical terms, similar features, and similar implementation steps in different embodiments will not be repeated.

[0035] This disclosure provides a table recognition method based on cell information, such as... Figure 1As shown, the method includes the following steps: Step S101: Obtain the position information of multiple cells included in the table.

[0036] In this step, the cell location information is usually output by a third party, such as a cell detection model.

[0037] Figure 2 This is a schematic diagram of a table provided in an embodiment of this disclosure, such as... Figure 2 As shown, this table includes merged cells both horizontally and vertically. Merged cells refer to cells that span multiple rows or columns, forming a larger cell area, in addition to the standard grid layout. This is typically used to represent headings, category names, or other information that needs to be displayed across multiple rows or columns.

[0038] Figure 3 It is a display Figure 2 The table shown is a schematic diagram illustrating the numbering of each cell. The cell location information mentioned in this embodiment includes, but is not limited to, the coordinates of the vertices of each cell, and such as... Figure 3 The relative position information of the numbered cells is shown. For example, cell 4 is located on the far left of the table, cell 5 is located to the right of cell 4 and adjacent to cell 4, and cell 5 is located above cell 2.

[0039] Step S102: Determine the four elements of each cell based on the location information of multiple cells. The four elements include: row index value, column index value, row span value, and column span value.

[0040] The four elements of a cell are key attributes that describe the cell's location and the area it spans.

[0041] The row index (rowindex) is the starting row number of the cell, the column index (colindex) is the starting column number of the cell, the row span (rowspan) is the number of columns the cell spans horizontally, and the column span (colspan) is the number of rows the cell spans vertically.

[0042] Step S103: Identify the structure of the table based on the four elements of each cell.

[0043] In the embodiments of this disclosure, identifying the table structure refers to identifying the row and column structure and layout of the table.

[0044] When performing step S103, the reconstruction process in table structure recognition is realized based on the row and column index values ​​and the row and column span values, and the table structure is converted into the target table format, such as HTML.

[0045] The technical solution provided in this disclosure can reconstruct a table based solely on cell location information without requiring a structure recognition model, thereby enabling the recognition of the table's structure. Furthermore, the recognition algorithm is computationally simple, highly robust, and adaptable to the recognition of table structures of various complex forms.

[0046] Figure 4 This is a flowchart illustrating a process for determining the row span value among the four elements of a cell based on its location information, as provided in this embodiment of the disclosure. Figure 4 As shown, it includes the following steps: Step S1: Determine whether there is a relationship between any two cells in the multiple cells.

[0047] According to some optional embodiments of this disclosure, the execution step S1 determines whether there is a relationship between any two cells in a plurality of cells, including at least the following conditions: if the horizontal distance between any two cells is less than a preset distance threshold, it is determined that there is a relationship between any two cells; if the vertical overlap rate between any two cells is greater than a first preset overlap rate threshold, it is determined that there is a relationship between any two cells.

[0048] Determining whether two cells are related mainly involves two conditions, such as... Figure 5 and Figure 6 As shown.

[0049] Figure 5 This is a schematic diagram of determining the interval between cells according to an embodiment of the present disclosure. If the horizontal interval between two cells is ( Figure 5 If the gap (indicated by the middle arrow) is lower than the preset threshold gap, then a relationship between the two cells is determined.

[0050] The calculation method is as follows: Let: use and Represents two vertical lines in a cell So, arrive The interval is represented as:

[0051]

[0052] One of the criteria for determining adjacent cells:

[0053] in, : arrive distance : arrive distance Absolute value function Vector cross product function Vector length function The vertex coordinates of a cell in a two-dimensional plane.

[0054] Maximum gap value configured by the user Figure 6 This is a schematic diagram illustrating the determination of the overlap rate between cells according to an embodiment of this disclosure. If the overlap rate between two cells in the vertical direction is... (See Figure 6 The BC / BD and BC / AC ratios exceed the preset thresholds. If so, then a relationship is determined between the two cells.

[0055] The calculation method is as follows:

[0056] One of the criteria for determining adjacent cells:

[0057] in, Cell 1 overlap rate Cell 2 overlap rate User-configured overlap threshold Calculation method: Refer to the above text. Calculation method Step S2: Based on the judgment result, construct the list of left neighboring cells and the list of right neighboring cells for each cell.

[0058] According to some optional embodiments of this disclosure, step S2 constructs a list of left neighboring cells and a list of right neighboring cells for each cell based on the judgment result, including the following steps: taking the cell located to the left of each cell and having a relationship with each cell as the left neighboring cell of each cell; taking the cell located to the right of each cell and having a relationship with each cell as the right neighboring cell of each cell.

[0059] The left-neighbor list and the right-neighbor list each contain multiple left-neighbor cells and multiple right-neighbor cells, respectively. For any cell in the table, its left-neighbor cell is the cell to its left that is related to it, and its right-neighbor cell is the cell to its right that is related to it. For example... Figure 3 In the example, cell 5's right neighbor list includes cells 3 and 6, and its left neighbor list includes cell 4; cell 4's right neighbor list includes cells 0, 1, 2, 3, 5, and 6, and its left neighbor list is empty.

[0060] Step S3: Determine the left neighbor filter list for each cell based on the left neighbor cell list for each cell, and determine the right neighbor filter list for each cell based on the right neighbor cell list for each cell.

[0061] According to some optional embodiments of this disclosure, step S3 determines the left-neighbor cell filter list for each cell based on the left-neighbor cell list of each cell, which is achieved by the following method: determining the rightmost column cell in the left-neighbor cell list of each cell; filtering out cells without any relationship from the rightmost column cell to form the left-neighbor cell filter list; determining the right-neighbor cell filter list for each cell based on the right-neighbor cell list of each cell includes: determining the leftmost column cell in the right-neighbor cell list of each cell; filtering out cells without any relationship from the leftmost column cell to form the right-neighbor cell filter list.

[0062] In this step, a subset of cells is selected from the left and right neighbor lists of each cell, and a left and right neighbor filter list is constructed for each cell.

[0063] The criteria for selecting cells from the left neighbor list of each cell are: the rightmost column of that list must exhibit an identity matrix characteristic, meaning no two cells are related to each other. Similarly, the criteria for selecting cells from the right neighbor list of each cell are: the leftmost column of that list must exhibit an identity matrix characteristic, meaning no two cells are related to each other.

[0064] like Figure 3 In the example, the right neighbor filter list of cell 5 includes cells 3 and 6, and the left neighbor filter list includes cell 4; the right neighbor filter list of cell 4 includes cells 5 and 2, and the left neighbor filter list is empty.

[0065] Step S4: Based on the left neighbor cell filter list and the right neighbor cell filter list of each cell, determine the merged cell list. The merged cells in the merged cell list are the cells in the table except for the atomic cells. The atomic cells are the cells in the table whose row span value and column span value are both 1.

[0066] First, let's introduce two concepts: atomic cells and merged cells.

[0067] An atomic cell is a cell that meets the condition (rowspan=1, colspan=1), for example... Figure 3 Cells 0, 1, and 6 in the text.

[0068] Merging cells refers to all cells in a table except for the original cells, such as... Figure 3 Cells 2, 3, 4, 5, 7, and 8 in the example. As mentioned above, merging cells refers to a situation where, in a standard grid layout, some cells span multiple rows or columns, forming a larger cell area. Figure 3 Cells 2, 3, 4, 5, 7, and 8 in the text all span multiple rows or columns.

[0069] According to some other optional embodiments of this disclosure, step S4 determines the list of merged cells based on the left neighbor cell filter list and the right neighbor cell filter list of each cell, and is achieved by the following method: For any target cell among multiple cells, determine a first set consisting of the left-neighbor filter list of any cell in the target cell's left-neighbor filter list and the target cell's left-neighbor filter list; if the length of the left-neighbor filter list of each cell in the first set is not greater than 1, the target cell is determined as an atomic cell; otherwise, the target cell is determined as a merged cell; or For any target cell among multiple cells, determine a second set consisting of the right neighbor filter list of any cell in the target cell's right neighbor filter list and the target cell's right neighbor filter list; if the length of the right neighbor filter list of each cell in the second set is not greater than 1, the target cell is determined as an atomic cell; otherwise, the target cell is determined as a merged cell.

[0070] In the embodiments of this disclosure, if the set consisting of the left / right neighbor cell filter list and the left / right neighbor cell filter lists of its descendants of a certain cell (i.e. the target cell mentioned above) satisfies the condition that the length of the left / right neighbor cell filter list of each cell in the set does not exceed 1, then the cell is determined to be an atomic cell; otherwise, it is considered a merged cell.

[0071] by Figure 3 For example, the right neighbor cell filter list of cell 4 includes: cell 5 and cell 2, and the right neighbor cell filter list of cell 5 includes: cell 3 and cell 6. Then cell 6 is the right neighbor descendant cell filter list of cell 4.

[0072] Step S5: Determine the merged cell value for each merged cell in the merged cell list, and use the merged cell value as the row span value for each merged cell.

[0073] In this step, the list of merged cells is processed in a loop. First, the value of the right merged cell of each merged cell is updated, and then the value of the right merged cell of each merged cell is used as the row span value of each merged cell.

[0074] According to some optional embodiments of this disclosure, step S5 is performed to determine the merged cell value of each merged cell in the merged cell list, which is achieved by the following method: determining the right merged cell value and the left merged cell value of each merged cell in the merged cell list respectively; and determining the maximum value between the right merged cell value and the left merged cell value of each merged cell as the merged cell value of each merged cell.

[0075] When updating the row span value, for each merged cell in the merged cell list, the corresponding right merged cell value is generated and summed based on the relationship between each merged cell and each cell in the right neighboring cell filter list. Then, the left merged cell value of each merged cell is calculated, and finally, the maximum value between the right merged cell value and the left merged cell value is taken as the row span value of each merged cell.

[0076] As an optional embodiment of this disclosure, determining the value of the right-hand or left-hand merged cell for each merged cell in the merged cell list includes the following steps: Step S11: Determine the right neighbor cell filter list or the left neighbor cell filter list for each merged cell; Step S12: If the overlap rate between each merged cell and any cell in the filter list of its right neighbor or the filter list of its left neighbor is higher than the second preset overlap rate threshold, proceed to step S13; otherwise, proceed to step S14; cumulatively calculate the value of the right merged cell or the value of the left merged cell for each merged cell. Step S13: Determine that the value of the right merged cell or the value of the left merged cell of each merged cell contains the value of the right merged cell or the value of the left merged cell of any other cell. Figure 7a This is a schematic diagram illustrating the containment relationship between cells provided in an embodiment of this disclosure, such as... Figure 7a As shown, if cell1 and cell2 contain each other (for example, the overlap between cell1 and cell2 is greater than 0.95), then the value of the merged cell to the right of cell2 is also counted as part of the value of the merged cell to the right of cell1.

[0077] Figure 7b This is a schematic diagram illustrating the overlapping relationship between cells provided in an embodiment of this disclosure, such as... Figure 7b As shown, if cells 1 and 2 overlap (for example, the overlap rate between cells 1 and 2 is greater than 0.9), then the value of the merged cell to the right of cell 2 is also counted as part of the value of the merged cell to the right of cell 1.

[0078] Figure 7a and Figure 7b This only shows how the values ​​in the merged cells on the right are calculated; the calculation method for the values ​​in the merged cells on the left is the same, so it will not be repeated here.

[0079] Step S14: Set the value of the right-hand merged cell or the left-hand merged cell of each merged cell to 1; Other situations (not satisfied) Figure 7a and Figure 7b (As shown in the two cases), the value of the merged cell to the right of cell2 is counted as 1.

[0080] Step S15: Repeat steps S11 to S14 until the value of the right-hand or left-hand merged cell of each merged cell remains unchanged.

[0081] It is worth noting that step S15 may be executed multiple times to ensure that the row span value of the cell no longer changes. Only then is the process complete for either the merged cell value on the right or the cell value on the left.

[0082] This disclosure provides an optional implementation method. Step S102 determines the column span value of each cell's four elements based on the position information of multiple cells. This is achieved by the following method: rotating the table 90° counterclockwise and then flipping it horizontally to obtain inverted cells; performing steps S1 to S5 above for each cell in the inverted cells to obtain the row span value of each cell in the inverted cells; and determining the row span value of each cell in the inverted cells as the column span value of each cell in the table.

[0083] Will Figure 3The table shown is rotated 90° counterclockwise and then flipped horizontally to obtain the inverted cells of the table, as shown below. Figure 8 As shown. Figure 8 Repeat the process for each cell in the inverted cells shown. Figure 4 Steps S1 to S5 shown in the diagram obtain the row span value of each cell in the inverted cell. This row span value is the column span value of the cell before the rotation and flipping operation.

[0084] This disclosure provides an optional implementation method. Step S102, which determines the column index value of each cell's four elements based on the position information of multiple cells, includes the following steps: Step S111: Determine the leftmost column cell of the table, wherein the column index value of each cell in the leftmost column is 1; Step S112: In order from top to bottom, update the column index value of each cell in the right neighbor cell filter list of each cell in the leftmost column in turn. The column index value of each cell in the right neighbor cell filter list of each cell in the leftmost column is the sum of the column index value of the leftmost column cell and the column span value. In embodiments of this disclosure, the column index (colindex) of all cells is updated based on the cell column span value (colspan). The leftmost column cell (i.e., the cell whose left neighbor list is empty) is found among all cells. Then, following a top-to-bottom order, the leftmost column cell (obviously, its column index (colindex) is 1) is traversed and processed. The column index (colindex) of each cell in its right neighbor filter list is updated to the sum of the column index (colindex) of the leftmost column cell and its column span value (colspan).

[0085] Step S113: For cells whose column index values ​​have been updated, update the column index values ​​of each cell in the filter list of their right neighbors in turn, until the column index values ​​of each cell in the table have been updated.

[0086] For cells that have already had their column index values ​​(colindex) generated, use the same method to update the column index values ​​(colindex) of each cell in the filter list to their right, until the column index values ​​of all cells have been updated.

[0087] This disclosure provides an optional implementation method. Step S102 determines the row index value of each cell's four elements based on the position information of multiple cells, including the following steps: rotating the table 90° counterclockwise and then flipping it horizontally to obtain inverted cells; performing steps S111 to S113 above for each cell in the inverted cells to obtain the column index value of each cell in the inverted cells; and determining the column index value of each cell in the inverted cells as the row index value of each cell in the table.

[0088] In this embodiment, Figure 3 The table shown is rotated 90° counterclockwise and then flipped horizontally to obtain the following result: Figure 8 The inverted cells are shown. The original top / bottom neighbor relationships between cells have been changed to left / right neighbor relationships. Then, steps S111 to S113 are executed again to obtain the column index (colindex) of each cell in the inverted cells. This column index is the row index (rowindex) of the cell before the rotation and flipping operation. This indirectly updates the row index values ​​of all cells.

[0089] Finally, based on the rowindex, colindex, rowspan, and colspan of each cell, a table recognition result is generated, such as an HTML file.

[0090] The technical solution proposed in this disclosure only requires cell location information to reconstruct the table structure. This cell location information comes from a third party, such as a cell detection model. Therefore, the technical solution proposed in this disclosure can be fused with cell location information generated by different algorithm models to form end-to-end functionality. It can decouple the cell detection or segmentation model from the table text recognition model, allowing users to arbitrarily combine the two models to achieve end-to-end table recognition functionality. Furthermore, the technical solution proposed in this disclosure is computationally simple, has high algorithm robustness, and is suitable for the structural recognition of various complex tables.

[0091] This disclosure provides a table recognition device based on cell information, such as... Figure 9 As shown, the table recognition device 90 may include: an acquisition module 901, a determination module 902, and a generation module 903, wherein, Module 901 is used to obtain the position information of multiple cells in a table.

[0092] The determination module 902 is used to determine the four elements of each cell based on the position information of multiple cells. The four elements include: row index value, column index value, row span value, and column span value.

[0093] The recognition module 903 is used to recognize the structure of the table based on the four elements of each cell.

[0094] According to some optional embodiments of this disclosure, the determining module 902 is further configured to determine the row span value among the four elements of each cell through the following steps: Step S1, determine whether there is a relationship between any two cells in the multiple cells; Step S2, construct a left neighbor cell list and a right neighbor cell list for each cell based on the determination result; Step S3, determine a left neighbor cell filter list for each cell based on the left neighbor cell list for each cell, and determine a right neighbor cell filter list for each cell based on the right neighbor cell list for each cell; Step S4, determine a merged cell list based on the left neighbor cell filter list and the right neighbor cell filter list for each cell, wherein the merged cells in the merged cell list are cells in the table excluding atomic cells, and atomic cells are cells in the table whose row span value and column span value are both 1; Step S5, determine the merged cell value of each merged cell in the merged cell list, and use the merged cell value as the row span value of each merged cell.

[0095] According to some optional embodiments of this disclosure, the determining module 902 is further configured to determine whether there is a relationship between any two cells in a plurality of cells by the following methods: if the horizontal distance between any two cells is lower than a preset distance threshold, it is determined that there is a relationship between any two cells; if the vertical overlap rate between any two cells is higher than a first preset overlap rate threshold, it is determined that there is a relationship between any two cells.

[0096] According to some optional embodiments of this disclosure, the determining module 902 is further configured to construct a list of left neighboring cells and a list of right neighboring cells for each cell respectively by the following methods: taking the cell located to the left of each cell and having an association relationship with each cell as the left neighboring cell of each cell; taking the cell located to the right of each cell and having an association relationship with each cell as the right neighboring cell of each cell.

[0097] According to some optional embodiments of this disclosure, the determining module 902 is further configured to determine the left neighbor cell filter list of each cell by the following methods: determining the rightmost column cell in the left neighbor cell list of each cell; filtering out cells without a relationship from the rightmost column cell to form the left neighbor cell filter list; determining the right neighbor cell filter list of each cell based on the right neighbor cell list of each cell, including: determining the leftmost column cell in the right neighbor cell list of each cell; filtering out cells without a relationship from the leftmost column cell to form the right neighbor cell filter list.

[0098] According to some optional embodiments of this disclosure, the determining module 902 is further configured to determine the merged cell list by the following method: for any target cell among multiple cells, determine a first set consisting of the left-neighbor cell filter list of any cell in the left-neighbor cell filter list of the target cell and the left-neighbor cell filter list of the target cell; if the length of the left-neighbor cell filter list of each cell in the first set is not greater than 1, determine the target cell as an atomic cell; otherwise, determine the target cell as a merged cell; or For any target cell among multiple cells, determine a second set consisting of the right neighbor filter list of any cell in the target cell's right neighbor filter list and the target cell's right neighbor filter list; if the length of the right neighbor filter list of each cell in the second set is not greater than 1, the target cell is determined as an atomic cell; otherwise, the target cell is determined as a merged cell.

[0099] According to some optional embodiments of this disclosure, the determining module 902 is further configured to determine the merged cell value of each merged cell in the merged cell list by the following method: determining the right merged cell value and the left merged cell value of each merged cell in the merged cell list respectively; and determining the maximum value of the right merged cell value and the left merged cell value of each merged cell as the merged cell value of each merged cell.

[0100] According to some optional embodiments of this disclosure, the determining module 902 is further configured to determine the value of the right-hand merged cell or the value of the left-hand merged cell of each merged cell in the merged cell list by the following method: Step S11, determine the right-hand neighbor filter list or the left-hand neighbor filter list of each merged cell; Step S12, if the overlap rate between each merged cell and any cell in the right-hand neighbor filter list or the left-hand neighbor filter list is higher than a second preset overlap rate threshold, execute step S13; otherwise, execute step S14; Step S13, determine that the value of the right-hand merged cell or the value of the left-hand merged cell of each merged cell is the value of the right-hand merged cell of any cell plus 1 or the value of the left-hand merged cell plus 1; Step S14, determine that the value of the right-hand merged cell or the value of the left-hand merged cell of each merged cell is 1; Step S15, repeat steps S11 to S14 until the value of the right-hand merged cell or the value of the left-hand merged cell of each merged cell remains unchanged.

[0101] According to some optional embodiments of this disclosure, the determining module 902 is further configured to determine the column span value among the four elements of each cell by the following method: rotating the table 90° counterclockwise and then flipping it horizontally to obtain the inverted cells of the table; performing the above steps S1 to S5 for each cell in the inverted cells to obtain the row span value of each cell in the inverted cells; and determining the row span value of each cell in the inverted cells as the column span value of each cell in the table.

[0102] According to some optional embodiments of this disclosure, the determining module 902 is further configured to determine the column index value of the four elements of each cell by the following method: Step S111, determining the leftmost column cell of the table, wherein the column index value of each cell in the leftmost column cell is 1; Step S112, updating the column index value of each cell in the right neighbor cell filter list of each cell in the leftmost column cell in order from top to bottom, wherein the column index value of each cell in the right neighbor cell filter list of each cell in the leftmost column cell is the sum of the column index value of each cell in the leftmost column cell and the column span value; Step S113, for the cell whose column index value has been updated, updating the column index value of each cell in its right neighbor cell filter list in order until the column index value of each cell in the table has been updated.

[0103] According to some optional embodiments of this disclosure, the determining module 902 is further configured to determine the row index value of the four elements of each cell by the following method: rotating the table 90° counterclockwise and then flipping it horizontally to obtain the inverted cells of the table; performing the above steps S111 to S113 for each cell in the inverted cells to obtain the column index value of each cell in the inverted cells; and determining the column index value of each cell in the inverted cells as the row index value of each cell in the table.

[0104] The apparatus of this disclosure embodiment can execute the method provided in this disclosure embodiment, and its implementation principle is similar, and it has corresponding technical effects. The actions performed by each module in the apparatus of each embodiment of this disclosure correspond to the steps in the method of each embodiment of this disclosure. For a detailed functional description of each module of the apparatus, please refer to the description in the corresponding method shown above, and it will not be repeated here.

[0105] This disclosure provides an electronic device including a memory, a processor, and a computer program stored in the memory. The processor executes the computer program to implement the steps of the method provided in any optional embodiment of this disclosure. Compared with the prior art, it can achieve table reconstruction based solely on cell location information without requiring a structural recognition model, thereby enabling the recognition of the table structure. Furthermore, the recognition algorithm is computationally simple, highly robust, and capable of adapting to the recognition of table structures of various complex forms. In one alternative embodiment, an electronic device is provided, such as Figure 10 As shown, Figure 10 The illustrated electronic device 10000 includes a processor 10001 and a memory 10003. The processor 10001 and the memory 10003 are connected, for example, via a bus 10002. Optionally, the electronic device 10000 may further include a transceiver 10004, which can be used for data interaction between the electronic device and other electronic devices, such as sending and / or receiving data. It should be noted that in practical applications, the transceiver 10004 is not limited to one type, and the structure of the electronic device 10000 does not constitute a limitation on the embodiments of this disclosure.

[0106] Processor 10001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with this disclosure. Processor 10001 may also be a combination that implements computing functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0107] Bus 10002 may include a pathway for transmitting information between the aforementioned components. Bus 10002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. Bus 10002 can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 10The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0108] The memory 10003 may be ROM (Read Only Memory) or other types of static storage devices capable of storing static information and instructions, RAM (Random Access Memory) or other types of dynamic storage devices capable of storing information and instructions, or EEPROM (Electrically Erasable Programmable Read Only Memory), CD-ROM (Compact Disc Read Only Memory) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media, other magnetic storage devices, or any other medium capable of carrying or storing computer programs and capable of being read by a computer, without limitation herein.

[0109] The memory 10003 is used to store computer programs that execute embodiments of the present disclosure, and is controlled by the processor 10001 to execute them. The processor 10001 is used to execute the computer programs stored in the memory 10003 to implement the steps shown in the foregoing method embodiments.

[0110] Electronic devices include, but are not limited to: computers, mobile phones, tablets and other mobile terminal devices.

[0111] This disclosure provides a computer-readable storage medium storing a computer program, which, when executed by a processor, can implement the steps and corresponding content of the aforementioned method embodiments.

[0112] This disclosure also provides a computer program product, including a computer program that, when executed by a processor, can implement the steps and corresponding content of the aforementioned method embodiments.

[0113] It should be understood that although arrows indicate various operation steps in the flowcharts of the embodiments of this disclosure, the order in which these steps are implemented is not limited to the order indicated by the arrows. Unless explicitly stated herein, in some implementation scenarios of the embodiments of this disclosure, the implementation steps in each flowchart can be executed in other orders as required. Furthermore, some or all of the steps in each flowchart may include multiple sub-steps or multiple stages based on the actual implementation scenario. Some or all of these sub-steps or stages can be executed at the same time, and each sub-step or stage can also be executed at different times. In scenarios where execution times differ, the execution order of these sub-steps or stages can be flexibly configured as required, and the embodiments of this disclosure do not limit this.

[0114] The above description is only an optional implementation method for some implementation scenarios of this disclosure. It should be noted that for those skilled in the art, other similar implementation methods based on the technical concept of this disclosure without departing from the technical concept of this disclosure also fall within the protection scope of the embodiments of this disclosure.

Claims

1. A table recognition method based on cell information, characterized in that, include: Get the position information of multiple cells in a table; The four elements of each cell are determined based on the position information of multiple cells, wherein the four elements include: row index value, column index value, row span value, and column span value; The structure of the table is identified based on the four elements of each cell.

2. The method according to claim 1, characterized in that, The row span value of each cell in the four elements is determined based on the position information of multiple cells, including: Step S1: Determine whether there is a relationship between any two cells among the plurality of cells; Step S2: Based on the judgment result, construct the left neighbor list and right neighbor list for each cell respectively; Step S3: Determine the left neighbor cell filter list for each cell based on the left neighbor cell list for each cell, and determine the right neighbor cell filter list for each cell based on the right neighbor cell list for each cell; Step S4: Based on the left neighbor cell filter list and the right neighbor cell filter list of each cell, determine the merged cell list, wherein the merged cells in the merged cell list are the cells in the table except for the atomic cells, and the atomic cells are the cells in the table whose row span value and column span value are both 1; Step S5: Determine the merged cell value of each merged cell in the merged cell list, and use the merged cell value as the row span value of each merged cell.

3. The method according to claim 2, characterized in that, To determine whether there is a relationship between any two cells in the plurality of cells, at least the following conditions must be met: If the horizontal distance between any two cells is less than a preset distance threshold, it is determined that there is a correlation between the two cells. If the vertical overlap rate of any two cells is higher than the first preset overlap rate threshold, it is determined that there is a relationship between the two cells.

4. The method according to claim 2, characterized in that, Based on the judgment results, construct the left neighbor list and right neighbor list for each cell, including: The cell located to the left of each cell, and having the aforementioned relationship with each cell, is designated as the left neighbor cell of each cell; The cell located to the right of each cell, and having the aforementioned relationship with each cell, is designated as the right neighbor cell of each cell.

5. The method according to claim 2, characterized in that, Determining a left-neighbor cell filter list for each cell based on the left-neighbor cell list for each cell includes: determining the rightmost column cell in the left-neighbor cell list for each cell; filtering out cells from the rightmost column cell that do not have the association relationship to form the left-neighbor cell filter list; Determining a right neighbor filter list for each cell based on the right neighbor cell list of each cell includes: determining the leftmost column cell in the right neighbor cell list of each cell; and filtering out cells from the leftmost column cell that do not have the association relationship to form the right neighbor filter list.

6. The method according to claim 2, characterized in that, Based on the left and right neighbor cell filter lists for each cell, a list of merged cells is determined, including: For any target cell among the plurality of cells, a first set is determined, consisting of the left-neighbor cell filter list of any cell in the left-neighbor cell filter list of the target cell and the left-neighbor cell filter list of the target cell; if the length of the left-neighbor cell filter list of each cell in the first set is not greater than 1, the target cell is determined as an atomic cell; otherwise, the target cell is determined as a merged cell; or For any target cell among the plurality of cells, a second set is determined, consisting of the right neighbor filter list of any cell in the right neighbor filter list of the target cell and the right neighbor filter list of the target cell; if the length of the right neighbor filter list of each cell in the second set is not greater than 1, the target cell is determined as an atomic cell; otherwise, the target cell is determined as a merged cell.

7. The method according to claim 2, characterized in that, Determining the merged cell value for each merged cell in the merged cell list includes: Determine the value of the right merged cell and the value of the left merged cell for each merged cell in the merged cell list; The maximum value between the right and left merged cell values ​​of each merged cell is determined as the merged cell value of each merged cell.

8. The method according to claim 7, characterized in that, Determining the value of the right-hand or left-hand merged cell for each merged cell in the merged cell list includes: Step S11: Determine the right neighbor cell filter list or the left neighbor cell filter list for each merged cell; Step S12: If the overlap rate between each merged cell and any cell in the filter list of its right neighbor or the filter list of its left neighbor is higher than the second preset overlap rate threshold, proceed to step S13; otherwise, proceed to step S14; cumulatively calculate the value of the right merged cell or the value of the left merged cell for each merged cell. Step S13: Determine that the value of the right merged cell or the value of the left merged cell of each merged cell includes the value of the right merged cell or the value of the left merged cell of any one of the cells. Step S14: Set the value of the right-side merged cell or the left-side merged cell of each merged cell to 1; Step S15: Repeat steps S11 to S14 until the value of the right-hand merged cell or the left-hand merged cell of each merged cell remains unchanged.

9. The method according to claim 2, characterized in that, The column span value in the four elements of each cell is determined based on the location information of multiple cells, including: Rotate the table 90° counterclockwise and then flip it horizontally to obtain the inverted cells of the table. For each cell in the inverted cell, perform steps S1 to S5 of claim 2 to obtain the row span value of each cell in the inverted cell; The row span value of each cell in the inverted cell is determined as the column span value of each cell in the table.

10. The method according to claim 2, characterized in that, Determine the column index value of each cell's four elements based on the location information of multiple cells, including: Step S111: Determine the leftmost column cell of the table, wherein the column index value of each cell in the leftmost column is 1; Step S112: In order from top to bottom, update the column index value of each cell in the right neighbor cell filter list of each cell in the leftmost column of cells sequentially, wherein the column index value of each cell in the right neighbor cell filter list of each cell in the leftmost column of cells is the sum of the column index value of the leftmost column cell and the column span value; Step S113: For cells whose column index values ​​have been updated, update the column index values ​​of each cell in the filter list of their right neighbors in turn, until the column index values ​​of each cell in the table have been updated.

11. The method according to claim 10, characterized in that, Determine the row index value of each cell's four elements based on the location information of multiple cells, including: After rotating the table 90° counterclockwise, it is then flipped horizontally to obtain the inverted cells of the table. For each cell in the inverted cells, perform steps S111 to S113 of claim 10 to obtain the column index value of each cell in the inverted cells; The column index value of each cell in the inverted cell is determined as the row index value of each cell in the table.

12. A table recognition device based on cell information, characterized in that, include: The acquisition module is used to obtain the position information of multiple cells in the table; The determination module is used to determine the four elements of each cell based on the position information of the plurality of cells, wherein the four elements include: row index value, column index value, row span value, and column span value; The recognition module is used to recognize the structure of the table based on the four elements of each cell.

13. An electronic device, comprising: A memory, a processor, and a computer program stored on the memory, characterized in that the processor executes the computer program to implement the method of any one of claims 1 to 11.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 11.

15. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 11.

Images