A method, apparatus, and computer program product for establishing table topology relationships.

Abstract

This invention provides a method, apparatus, and computer program product for establishing table topology relationships, relating to the field of electronic digital data processing technology. By identifying the vertex coordinates of cells, selecting matching points, calculating the coordinate difference, and comparing it with length and width error thresholds, the invention identifies the relationships between cells in the row and column directions. It then sorts the cells within rows or columns using the coordinate values ​​of specified marker points, directly dividing rows and columns and reconstructing spatial relationships based on geometric rules. This reduces performance degradation caused by excessively large image sizes, does not rely on the integrity or clarity of table borders, and can adapt to complex table scenarios involving borderless tables and tables with numerous empty rows and columns, while significantly improving processing speed.

Description

Technical Field

[0001] This invention relates to the field of electronic digital data processing technology, and in particular to a method, apparatus, and computer program product for establishing table topology relationships. Background Technology

[0002] In the field of smart finance, the automatic parsing of financial statements is a crucial step in achieving intelligent processing of corporate financial data. Financial statements are typically presented in tabular form, containing a large amount of structured data, requiring the extraction of topological relationships within the tables. Traditional methods for table structure recognition mainly rely on deep learning techniques, such as models based on object detection or semantic segmentation. These methods analyze the overall features of the table image, identify table regions and segment cells, and then use subsequent algorithms to reconstruct the table's structural information. However, for long or complex tables, such as those with slanted or curved borders, or those spanning multiple pages, deep learning models face numerous challenges in practical applications.

[0003] Currently, the mainstream table structure recognition technologies can be mainly divided into the following categories:

[0004] 1) Object detection-based methods, such as DeepTabStR and TableNet, treat the rows and columns of a table as the objects to be detected and use object detection networks for detection. These methods assume that the table is well aligned and that the bounding boxes of the detected cells are all rectangles, making it difficult to handle tables in natural scenes. 2) Semantic segmentation-based methods, such as TableNet, obtain the positional information of columns in the table through semantic segmentation and then use heuristics to obtain row information, thus achieving table structure recognition. This method is often particularly effective for identifying and extracting tables from complex documents, but the unclear boundaries of table regions and sparse visual cues limit their ability to find accurate table segments. 3) Graph network-based methods, such as TIES and GFTE, treat text blocks as vertices of a graph and the relationships between text blocks as edges, predicting the row and column positional relationships between text blocks using graph neural networks. These methods do not rely on table borders or image features and have stronger generalization capabilities, but due to the limitations of the graph network structure, they often do not fully utilize spatial feature information and are prone to losing local spatial information during graph network inference. 4) Text-box-based graph model methods, such as GEAN-TSR, first use the OCR results as a basis to model the graph structure of the text boxes, and then use graph neural networks for inference to reconstruct the table structure information. This method does not rely on table borders, but it has poor anti-interference ability and is difficult to handle complex structures in tables.

[0005] It is evident that deep learning-based table processing solutions require extensive labeled data for training, exhibiting strong data dependency. They suffer from high segmentation error rates when dealing with complex tables, are prone to losing local spatial information leading to misjudgments, and have a large number of model parameters, resulting in high resource consumption and poor real-time performance when processing long tables. Furthermore, they are poorly adaptable to special scenarios, performing poorly on common scenarios such as borderless tables and tables containing numerous empty rows and columns. Therefore, a new solution specifically addressing table topological relationships is urgently needed. Summary of the Invention

[0006] In view of this, embodiments of the present invention provide a method, apparatus and computer program product for establishing table topology relationships, so as to eliminate or improve one or more defects existing in the prior art, and solve the problems of high computing resources required in the table processing process and poor adaptability to complex tables in the prior art.

[0007] This invention provides a method for establishing table topological relationships, which includes the following steps:

[0008] The identification points and their coordinates of each cell in the table image to be analyzed are identified and extracted based on optical character recognition or a preset table structure recognition model. The coordinates of the identification points include vertex coordinates and / or center point coordinates; the vertex coordinates include the top left vertex, bottom left vertex, top right vertex, and bottom right vertex.

[0009] Calculate the average length of each cell in the row direction and the average width in the column direction based on the vertex coordinates of each cell, and multiply by the tolerance rate to obtain the length error threshold and the width error threshold;

[0010] Traverse all cells, and calculate the first matching coordinate difference between the first matching points of the first cell and the second cell in the row direction, and the second matching coordinate difference between the second matching points of the second cell in the column direction. In the row direction, if there exists a first matching coordinate difference that is less than the length error threshold in the row direction and less than the width error threshold in the column direction, then mark the first cell and the second cell as being associated in the same row. In the column direction, if there exists a second matching coordinate difference that is less than the length error threshold in the row direction and less than the width error threshold in the column direction, then mark the first cell and the second cell as being associated in the same column.

[0011] Multiple cells in the same row are sorted first along the horizontal coordinate and then along the vertical coordinate based on the first specified marker point. Multiple cells in the same column are sorted first along the vertical coordinate and then along the horizontal coordinate based on the second specified marker point. The topology is reconstructed based on the arrangement relationship of each cell in the row and column of the table image to be analyzed.

[0012] The first type of matching point, the second type of matching point, the first designated marker point, and the second designated marker point are selected from the marker points.

[0013] In some embodiments, the tolerance rate is a fixed tolerance rate or a dynamic tolerance rate;

[0014] The formula for calculating the fixed tolerance rate is:

[0015] ;

[0016] in, This represents the fixed tolerance rate, and N represents the total number of cells in the analysis table image. This is the adjustment coefficient;

[0017] The formula for calculating the dynamic tolerance rate is:

[0018] ;

[0019] in, This represents the dynamic tolerance rate. Where Q is the base tolerance, S is the image quality coefficient, M is the table structure complexity coefficient, and M is the real-time matching success rate. , and These are the weighting coefficients.

[0020] In some embodiments, the calculation step of the image quality coefficient includes:

[0021] The image of the table to be analyzed is divided into blocks and the standard deviation of the brightness of each block is calculated as the illumination uniformity.

[0022] The gradient magnitude is calculated by convolution using the Sobel operator on the grayscale image of the table to be analyzed. A gradient magnitude matrix is ​​constructed according to the spatial position of the original image, and the gradient variance is calculated based on the gradient magnitude matrix. The edge sharpness is obtained by assigning values ​​based on the preset gradient variance interval.

[0023] Canny edge detection and Hough line detection are performed on the grayscale image of the table to be analyzed. The angles of the table lines in the image are extracted and the deviations from the reference angles are calculated as the perspective distortion.

[0024] The image quality coefficient is obtained by normalizing the illumination uniformity, edge sharpness, and perspective distortion, and then weighting and summing them.

[0025] In some embodiments, the calculation steps for the table structure complexity coefficient include:

[0026] The difference between the total number of cells in the table image to be analyzed and the image area is used as the cell number density.

[0027] The proportion of nested tables is obtained by calculating the proportion of cells containing sub-tables;

[0028] The table structure complexity coefficient is obtained by normalizing the cell density and the proportion of nested tables and then weighting and summing them.

[0029] In some embodiments, the method further includes: when the real-time matching success rate is less than a set value, increasing the dynamic tolerance rate by a preset ratio to relax the restriction.

[0030] In some embodiments, the first type of matching points includes the upper right vertex of the first cell and the upper left vertex of the second cell, as well as the lower right vertex of the first cell and the lower left vertex of the second cell;

[0031] The second type of matching points includes the lower left vertex of the first cell and the upper left vertex of the second cell, as well as the lower right vertex of the first cell and the upper right vertex of the second cell;

[0032] The first matching coordinate difference includes: the first row and column coordinate difference between the top right vertex of the first cell and the top left vertex of the second cell, and the second row and column coordinate difference between the bottom right vertex of the first cell and the bottom left vertex of the second cell;

[0033] The second matching coordinate difference includes: the third row and column coordinate difference between the bottom left vertex of the first cell and the top left vertex of the second cell, and the fourth row and column coordinate difference between the bottom right vertex of the first cell and the top right vertex of the second cell;

[0034] In the row direction, if the first row and column coordinate difference and the second row and column coordinate difference are both less than the length error threshold in the row direction and both less than the width error threshold in the column direction, then the first cell and the second cell are marked as being associated within the same row; in the column direction, if the third row and column coordinate difference and the fourth row and column coordinate difference are both less than the length error threshold in the row direction and both less than the width error threshold in the column direction, then the first cell and the second cell are marked as being associated within the same column.

[0035] In some embodiments, the first type of matching point and the second type of matching point are the cell center points;

[0036] The first matching coordinate difference and the second matching coordinate difference are both the coordinate differences of the cell center points between the first cell and the second cell; the first error is obtained by subtracting the row coordinate difference from the average length, and the second error is obtained by subtracting the column coordinate difference from the average width.

[0037] In the row direction, if the first error is less than the length error threshold, the first cell and the second cell are marked as being associated within the same row; in the column direction, if the second error is less than the width error threshold, the first cell and the second cell are marked as being associated within the same column.

[0038] In some embodiments, the average length is replaced by the median or mode of the length of each cell; the average width is replaced by the median or mode of the width of each cell; and the table structure recognition model is the Cycle-CenterNet model.

[0039] On the other hand, the present invention also provides a table topology relationship establishment device, including a processor, a memory, and a computer program / instructions stored in the memory, wherein the processor is used to execute the computer program / instructions, and when the computer program / instructions are executed, the device implements the steps of the above method.

[0040] On the other hand, the present invention also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the above-described method.

[0041] The table topology relationship establishment method, apparatus, and computer program product described in this invention identify the vertex coordinates of cells, select matching points, calculate the coordinate difference, and compare it with length and width error thresholds to identify the correlation between cells in the row and column directions. By specifying the coordinate values ​​of marked points, the cells are sorted within the row or column. The method directly divides rows and columns and reconstructs spatial relationships based on geometric rules, reducing the performance degradation caused by excessively large image sizes. It does not depend on the integrity or clarity of the table borders, can adapt to processing borderless tables and complex table scenarios containing a large number of empty rows and columns, and significantly improves processing speed.

[0042] Additional advantages, objects, and features of the invention will be set forth in part in the description which follows, and will also become apparent in part to those skilled in the art upon studying the description, or may be learned by practice of the invention. The objects and other advantages of the invention can be realized and obtained by means of the structures specifically pointed out in the description and drawings.

[0043] Those skilled in the art will understand that the objectives and advantages achievable with the present invention are not limited to those specifically described above, and that the above and other objectives achievable with the present invention will become clearer from the following detailed description. Attached Figure Description

[0044] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, are not intended to limit the scope of the invention. In the drawings:

[0045] Figure 1 This is a flowchart illustrating a method for establishing table topology relationships according to an embodiment of the present invention.

[0046] Figure 2 This is a schematic diagram of the edge sharpness analysis process of the table image to be analyzed in the table topology relationship establishment method according to an embodiment of the present invention.

[0047] Figure 3 This is a schematic diagram of the perspective distortion analysis process of the table image to be analyzed in the table topology relationship establishment method according to an embodiment of the present invention. Detailed Implementation

[0048] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the embodiments and accompanying drawings. Here, the illustrative embodiments and descriptions of this invention are used to explain the invention, but are not intended to limit the invention.

[0049] It should also be noted that, in order to avoid obscuring the invention with unnecessary details, only the structures and / or processing steps closely related to the solution according to the invention are shown in the accompanying drawings, while other details that are not closely related to the invention are omitted.

[0050] It should be emphasized that the term "including / comprises" as used herein refers to the presence of a feature, element, step, or component, but does not exclude the presence or addition of one or more other features, elements, steps, or components.

[0051] Existing technologies using deep neural networks for table processing to identify the topological relationships within table cells require extensive pre-training with labeled data, resulting in weak generalization ability and models that are sensitive to variations in table layout, making them difficult to adapt to different scenarios. Text-box-based graph model methods are prone to losing local spatial information during inference, leading to structural misjudgments. Furthermore, due to the large number of model parameters, processing long tables consumes significant resources and suffers from poor real-time performance. For example, deep learning models such as DCNNs involve enormous computational demands when processing long tables, making them unsuitable for practical applications.

[0052] In view of this, the present invention provides a method for establishing table topological relationships, such as... Figure 1As shown, the method includes the following steps S101~S104:

[0053] Step S101: Based on optical character recognition or a preset table structure recognition model, identify and extract the identifier points and their coordinates of each cell in the table image to be analyzed. The coordinates of the identifier points include vertex coordinates and / or center point coordinates; vertex coordinates include the top left vertex, bottom left vertex, top right vertex, and bottom right vertex.

[0054] Step S102: Calculate the average length of each cell in the row direction and the average width in the column direction based on the vertex coordinates of each cell, and multiply by the tolerance rate to obtain the length error threshold and the width error threshold.

[0055] Step S103: Traverse all cells, and calculate the first matching coordinate difference between the first matching points of the first cell and the second cell in the row direction, and the second matching coordinate difference between the second matching points of the second cell in the column direction; in the row direction, if there is a first matching coordinate difference that is less than the length error threshold in the row direction and less than the width error threshold in the column direction, then mark the first cell and the second cell as being associated in the same row; in the column direction, if there is a second matching coordinate difference that is less than the length error threshold in the row direction and less than the width error threshold in the column direction, then mark the first cell and the second cell as being associated in the same column.

[0056] Step S104: Sort multiple cells in the same row according to the first specified marker point, first along the horizontal coordinate and then along the vertical coordinate; sort multiple cells in the same column according to the second specified marker point, first along the vertical coordinate and then along the horizontal coordinate; reconstruct the topology based on the arrangement relationship of each cell in the row and column of the table image to be analyzed.

[0057] Among them, the first type of matching point, the second type of matching point, the first designated marker point, and the second designated marker point are selected from the marker points.

[0058] In step S101, the vertex coordinates of cells are identified based on Optical Character Recognition (OCR) or the Cycle-CenterNet table structure recognition model. OCR is a technology that converts text information in an image into editable text, widely used in the digitization of documents such as tickets, certificates, forms, and books. Cycle-CenterNet is a wired table structure recognition model that focuses on extracting the physical coordinates (four vertices) and logical structure (row and column numbers) of table cells from an image. Cycle-CenterNet has a relatively small parameter size, making it a lightweight model suitable for deployment in resource-constrained environments. It primarily identifies typical markers that provide cell location capabilities, including the four vertices and the cell center point.

[0059] In step S102, based on the identified top-left, bottom-left, top-right, and bottom-right vertices of the cells, the row length and column width of each cell can be calculated, the average length and average width can be calculated, and the length error threshold and width error threshold can be calculated based on these. In other embodiments, the median or mode of the length or width of each cell can also be used to calculate the length error threshold and width error threshold.

[0060] Furthermore, when calculating the length error threshold and the width error threshold, the tolerance rate can be a fixed tolerance rate or a dynamic tolerance rate.

[0061] The formula for calculating the fixed tolerance rate is:

[0062] ;

[0063] in, This represents a fixed tolerance rate, where N represents the total number of cells in the analysis table image. This is the adjustment factor. For example, a fixed tolerance rate of 0.05 can be set.

[0064] To better adapt to processing needs in different scenarios and achieve higher accuracy, the dynamic tolerance rate can be calculated using the following formula:

[0065] ;

[0066] in, Indicates dynamic tolerance rate. Where Q is the base tolerance, S is the image quality coefficient, M is the table structure complexity coefficient, and M is the real-time matching success rate. , and These are the weighting coefficients.

[0067] In this embodiment, the dynamic tolerance rate is adjusted by combining the base tolerance rate with an image quality coefficient, a table structure complexity coefficient, and a real-time matching success rate. The specific methods for obtaining these coefficients are as follows.

[0068] In some embodiments, the calculation of the image quality coefficient includes steps S1021 to S1024:

[0069] Step S1021: Divide the image of the table to be analyzed into blocks and calculate the standard deviation of the brightness of each block as the illumination uniformity.

[0070] Step S1022: As Figure 2As shown, the gradient magnitude is calculated by convolution using the Sobel operator on the grayscale image to be analyzed. The gradient magnitude matrix is ​​constructed according to the spatial position of the original image, and the gradient variance is calculated based on the gradient magnitude matrix. The edge sharpness is obtained by assigning values ​​based on the preset gradient variance interval.

[0071] Step S1023: As Figure 3 As shown, Canny edge detection and Hough line detection are performed on the grayscale image of the table to be analyzed. The angles of the table lines in the image are extracted and the deviations are calculated with reference angles as perspective distortion.

[0072] Step S1024: Normalize the illumination uniformity, edge sharpness, and perspective distortion, and then sum them by weight to obtain the image quality coefficient.

[0073] In step S1021, the input table image to be analyzed is divided into several sub-regions of equal size, and then the average brightness of all pixels in each sub-region is calculated. Next, the standard deviation of the average brightness of these sub-regions is calculated, and this standard deviation is used as an indicator of the image's illumination uniformity. The larger the standard deviation value, the more significant the brightness difference between different regions of the image, that is, the more uneven the illumination; conversely, the smaller the value, the more uniform the illumination distribution across the entire image.

[0074] In step S1022, the table image to be analyzed is first converted into a grayscale image. Then, the Sobel operator is applied to perform convolution on the grayscale image to calculate the gradient magnitude of each pixel in the image, reflecting the degree of brightness change of the pixel, i.e., edge strength, thus obtaining a gradient magnitude image. Next, the variance of the gradient magnitude of all pixels in this gradient magnitude image is calculated. The magnitude of this variance directly reflects the change in the overall edge sharpness of the image. A larger variance usually means that the image contains more sharp edges and fewer blurry or smooth areas, i.e., the edges are sharper; conversely, a smaller variance indicates blurry edges or low contrast. Finally, according to the preset gradient variance range, the calculated gradient variance value is mapped or assigned to a standardized score, which serves as a quantified indicator of edge sharpness.

[0075] In step S1023, also based on the grayscale processed table image, the Canny edge detection algorithm is first used to identify the main edge contours in the image. Then, based on these detected edges, the Hough line detection algorithm is applied to identify and extract all significant line segments in the image. These line segments typically correspond to the table's border lines. Next, the angular deviation between these detected line segments and a preset reference angle is calculated. The reference angle typically uses an ideal horizontal angle of 0° and a vertical angle of 90°. Finally, a statistical value, such as the average value, is calculated for the angular deviation of all detected table line segments. This statistical value is defined as the perspective distortion of the image. The greater the distortion, the more severe the tilt and distortion of the table image caused by the shooting angle or lens distortion; the smaller the distortion, the closer the table is to being shot from a normal angle, and the smaller the geometric deformation.

[0076] In step S1024, the three indicators with different physical meanings and dimensions obtained in step S1021 (illumination uniformity), step S1022 (edge ​​sharpness), and step S1023 (perspective distortion) are normalized respectively. Typically, they are mapped to the range [0,1] to eliminate the influence of dimensions and facilitate comparison and weighting. Then, preset weight coefficients are assigned to each of these three normalized indicator values. Finally, the three weighted values ​​are summed to obtain the final image quality coefficient. This coefficient is a comprehensive value; a higher value indicates better overall image quality, while a lower value indicates poorer image quality, which may require greater tolerance to cope with interference from image defects in subsequent table structure recognition.

[0077] In some embodiments, the calculation steps for the table structure complexity coefficient include steps S1205~S1027:

[0078] Step S1205: Calculate the difference between the total number of cells in the image of the table to be analyzed and the image area as the cell number density.

[0079] Step S1206: Calculate the proportion of cells containing sub-tables to obtain the proportion of nested tables.

[0080] Step S1207: Normalize the cell density and nested table ratio, then sum them by weight to obtain the table structure complexity coefficient.

[0081] In step S1025, the complexity of the table structure is evaluated by quantifying the cell distribution density. Specifically, the total number of cells identified in the image of the table to be analyzed is counted, and the total area of ​​the image is calculated, which can be expressed in pixels. The total number of cells is divided by the total area of ​​the image to obtain the number of cells per unit area, i.e., the cell density. A higher value indicates a denser table structure, smaller cell spacing, and greater difficulty in establishing topological relationships; conversely, a lower value reflects a sparser table structure and lower parsing difficulty.

[0082] In step S1026, nested table structures are detected to assess hierarchical complexity. All identified cells are traversed, and nested cells are determined by analyzing whether their internal areas contain sub-table features, such as internal lines or secondary cell outlines. The number of cells containing sub-tables is counted, and this number is divided by the total number of cells to obtain the nesting ratio. For example, a 100-cell table containing 5 sub-tables has a nesting ratio of 0.05. A higher ratio indicates a more complex table hierarchy.

[0083] In step S1027, the cell density and the nesting ratio are normalized respectively, mapping the cell density to the [0,1] interval, and the nesting ratio is directly converted to a [0,1] value as a percentage. The two normalization results are weighted and summed according to preset weights, and the final output is a table structure complexity coefficient in the range of 0 to 1. The closer the coefficient is to 1, the more complex the table structure, and the higher the tolerance threshold needs to be used in subsequent topology establishment to cope with matching interference caused by dense cells or nesting relationships.

[0084] Furthermore, the real-time matching success rate is the proportion of cells that detect intra-row and intra-column associations. The initial state is set to 100%, and it is recalculated after each round of matching action in step S103.

[0085] In some embodiments, the method further includes: when the real-time matching success rate is less than a set value, increasing the dynamic tolerance rate by a preset ratio to relax the restriction.

[0086] In step S103, the positional relationship between cells can exist in at least three ways: first, they are connected along the row direction; second, they are connected along the column direction; and third, there is no connection. To identify the positional relationship between cells, this invention determines it by the relative position of matching points between cells. Specifically, matching points are points whose relative positions are determined between adjacent cells, such as overlapping points. For example, for two cells connected along the row direction, the upper right and lower right vertices of the first cell overlap with the upper left and lower left vertices of the second cell, respectively. For two cells connected along the column direction, the lower left and lower right vertices of the first cell overlap with the upper left and upper right vertices of the second cell, respectively.

[0087] Therefore, in some embodiments, the first type of matching points includes the top-right vertex of the first cell and the top-left vertex of the second cell, as well as the bottom-right vertex of the first cell and the bottom-left vertex of the second cell. The second type of matching points includes the bottom-left vertex of the first cell and the top-left vertex of the second cell, as well as the bottom-right vertex of the first cell and the top-right vertex of the second cell.

[0088] The first matching coordinate difference includes: the first row and column coordinate difference between the top right vertex of the first cell and the top left vertex of the second cell, and the second row and column coordinate difference between the bottom right vertex of the first cell and the bottom left vertex of the second cell.

[0089] The second matching coordinate difference includes: the third row and column coordinate difference between the bottom left vertex of the first cell and the top left vertex of the second cell, and the fourth row and column coordinate difference between the bottom right vertex of the first cell and the top right vertex of the second cell.

[0090] It should be emphasized that the differences in the coordinates of the first row and column, the second row and column, the third row and column, and the fourth row and column all include the differences along the row direction and the differences along the column direction.

[0091] In the row direction, if the first row and column coordinate difference and the second row and column coordinate difference are both less than the length error threshold in the row direction and both less than the width error threshold in the column direction, then the first cell and the second cell are marked as being associated within the same row; in the column direction, if the third row and column coordinate difference and the fourth row and column coordinate difference are both less than the length error threshold in the row direction and both less than the width error threshold in the column direction, then the first cell and the second cell are marked as being associated within the same column.

[0092] In some embodiments, the first type of matching point and the second type of matching point are the cell center points; then the first matching coordinate difference and the second matching coordinate difference are both the cell center point coordinate differences between the first cell and the second cell; the first error is obtained by subtracting the row-direction coordinate difference from the average length, and the second error is obtained by subtracting the column-direction coordinate difference from the average width; in the row direction, if the first error is less than the length error threshold, the first cell and the second cell are marked as being associated within the same row; in the column direction, if the second error is less than the width error threshold, the first cell and the second cell are marked as being associated within the same column.

[0093] In step S104, all cells marked as related within the same row are grouped, with each group representing a row; the horizontal coordinate of the top left vertex of each cell in the group is taken as the first designated marker point, and the cells are arranged in ascending order first along the horizontal coordinate and then along the vertical coordinate to form a continuous row sequence from left to right.

[0094] Group all cells marked as related within the same column, with each group representing a column; take the top-left ordinate of each cell in the group as the second specified marker point, and arrange the cells in ascending order first along the ordinate and then along the x-coordinate to form a continuous column sequence from bottom to top.

[0095] Using the row and column sequence as the skeleton, each cell is mapped to its row and column indices to generate a two-dimensional matrix structure. For cells that span multiple rows or columns, the index is dynamically expanded based on the range of rows and columns covered by their vertex coordinates, ultimately outputting a table topology graph with merged relationships.

[0096] On the other hand, the present invention also provides a table topology relationship establishment device, including a processor, a memory, and a computer program / instructions stored in the memory, wherein the processor is used to execute the computer program / instructions, and when the computer program / instructions are executed, the device implements the steps of the above method.

[0097] On the other hand, the present invention also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the above-described method.

[0098] The present invention will now be described with reference to a specific embodiment:

[0099] This embodiment provides a table structure topology reconstruction method based on vertex coordinate relationships, specifically including the following steps S1~S4:

[0100] Step S1: Obtain vertex coordinates. First, obtain the vertex coordinates of each cell using OCR or a table structure recognition model. The format is (top left x, top left y), (top right x, top right y), (bottom right x, bottom right y), and (bottom left x, bottom left y). Vertex coordinates can be replaced with the cell center point coordinates.

[0101] Step S2: Error threshold setting, based on the average cell size avg_cell_size in the table, i.e., the average length of the diagonal. Set the error threshold δ = 0.05 × avg_cell_size. This dynamic tolerance can adapt to the processing needs of tables of different sizes.

[0102] Step S3: Spatial continuity determination. By comparing the spatial relationship between the vertex coordinates of adjacent cells, it is determined whether they belong to the same row or the same column.

[0103] For row determination: compare the coordinates of the two right vertices of the current cell with the coordinates of the two left vertices of the next cell. If they are the same or differ within the error threshold δ, they are determined to be in the same row.

[0104] Specifically, iterate through all cells and perform the following processing on each cell: retrieve the coordinates of the top-right and bottom-right vertices of the current cell; find the next possible cell and retrieve its top-left and bottom-left vertices; calculate the coordinate difference between the right vertex of the current cell and the left vertex of the next cell; if the difference is less than or equal to the error threshold δ, they are determined to be in the same row; record the determination result to establish the row association relationship between cells. In some embodiments, other vertex combinations can also be used. The distance between coordinates can be Manhattan distance or Euclidean distance.

[0105] For column determination: compare the coordinates of the bottom left and bottom right vertices of the current cell with the coordinates of the top left and top right vertices of the next cell. If they are the same or differ within the error threshold δ, they are determined to be in the same column.

[0106] Traverse all cells and process each cell as follows: retrieve the coordinates of the bottom left and bottom right vertices of the current cell; find the next possible cell and retrieve the coordinates of its top left and top right vertices; calculate the difference between the coordinates of the bottom vertex of the current cell and the top vertex of the next cell; if the difference is less than or equal to the error threshold δ, determine that they are in the same column; record the determination result and establish the column association relationship between cells.

[0107] In some embodiments, an adjacency list storage structure can be used instead of vertex coordinate comparison.

[0108] Step S4: Topology Reconstruction. Based on the spatial continuity determination result, reconstruct the row and column topology of the table. Specifically: within rows, sort in ascending order by x, then y, ensuring the correct order of cells within the row. Within columns, sort in ascending order by y, then x, ensuring the correct order of cells within the column. Sort methods can include bubble sort or insertion sort.

[0109] For cells that are determined to be in the same row, sort them according to the following rules: First, sort them in ascending order by the x-coordinate of the top left vertex; if the x-coordinates are the same, sort them in ascending order by the y-coordinate of the top left vertex; this ensures that the cells are in the correct order within the row.

[0110] For cells that are determined to be in the same column, sort them according to the following rules: First, sort them in ascending order by the y-coordinate of the top left vertex; if the y-coordinates are the same, sort them in ascending order by the x-coordinate of the top left vertex; this ensures that the cells are in the correct order within the column.

[0111] Based on the results of row and column association determination, the row and column topology of the table is reconstructed: cells with row association are organized into rows according to the sorting results within the row, and cells with column association are organized into columns according to the sorting results within the column, thus forming the row and column logical structure of the table.

[0112] This embodiment reconstructs spatial relationships using vertex coordinates, directly dividing rows and columns based on geometric rules, thus reducing performance degradation caused by large image sizes. Spatial relationship determination is performed directly using cell vertex coordinates, independent of the integrity or clarity of table borders. This vertex coordinate spatial relationship reconstruction can handle common tilted or curved table layouts in practical applications. By directly reconstructing the table structure using geometric rules, computational load is reduced. Experiments show that the processing speed of this embodiment is 3-5 times faster than traditional methods, making it particularly suitable for applications with high real-time requirements. It can also effectively handle special scenarios such as borderless tables and tables containing many empty rows and columns.

[0113] In summary, the table topology relationship establishment method, apparatus, and computer program product of this invention identify the vertex coordinates of cells, select matching points to calculate the coordinate difference and compare it with length error thresholds and width error thresholds, identify the correlation between cells in the row and column directions, and sort them in the row or column by specifying the coordinate values ​​of the marked points. It directly divides rows and columns and reconstructs spatial relationships based on geometric rules, reduces the performance degradation caused by excessively large image sizes, does not depend on the integrity or clarity of the table borders, can adapt to processing borderless tables and complex table scenarios containing a large number of empty rows and columns, and significantly improves processing speed.

[0114] Corresponding to the above method, the present invention also provides an apparatus / system including a computer device, the computer device including a processor and a memory, the memory storing computer instructions, the processor executing the computer instructions stored in the memory, and when the computer instructions are executed by the processor, the apparatus / system performs the steps of the method as described above.

[0115] This invention also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the aforementioned edge computing server deployment method. The computer-readable storage medium can be a tangible storage medium, such as random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, floppy disks, hard disks, removable storage disks, CD-ROMs, or any other form of storage medium known in the art.

[0116] Those skilled in the art will understand that the exemplary components, systems, and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software, or a combination of both. Whether implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this invention. When implemented in hardware, it can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this invention are programs or code segments used to perform the desired tasks. The programs or code segments can be stored in a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried in a carrier wave.

[0117] It should be clarified that the present invention is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of the present invention.

[0118] In this invention, features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, and / or combined with or in place of features of other embodiments.

[0119] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, various modifications and variations of the embodiments of the present invention are possible. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for establishing table topological relationships, characterized in that, The method includes the following steps: The identification points and their coordinates of each cell in the table image to be analyzed are identified and extracted based on optical character recognition or a preset table structure recognition model. The coordinates of the identification points include vertex coordinates and / or center point coordinates; the vertex coordinates include the top left vertex, bottom left vertex, top right vertex, and bottom right vertex. Calculate the average length of each cell in the row direction and the average width in the column direction based on the vertex coordinates of each cell, and multiply by the tolerance rate to obtain the length error threshold and the width error threshold; Traverse all cells, and calculate the first matching coordinate difference between the first type of matching points corresponding to the positions of the first cell and the second cell in the row direction, and the second matching coordinate difference between the second type of matching points corresponding to the positions in the column direction; in the row direction, if there exists a first matching coordinate difference that is less than the length error threshold in the row direction and less than the width error threshold in the column direction, then mark the first cell and the second cell as being associated in the same row; in the column direction, if there exists a second matching coordinate difference that is less than the length error threshold in the row direction and less than the width error threshold in the column direction, then mark the first cell and the second cell as being associated in the same column. Multiple cells in the same row are sorted first along the horizontal coordinate and then along the vertical coordinate based on the first specified marker point. Multiple cells in the same column are sorted first along the vertical coordinate and then along the horizontal coordinate based on the second specified marker point. The topology is reconstructed based on the arrangement relationship of each cell in the row and column of the table image to be analyzed. Among them, the first type of matching point, the second type of matching point, the first designated marker point, and the second designated marker point are selected from the marker points; Wherein, the first type of matching points includes the top right vertex of the first cell and the top left vertex of the second cell, and the bottom right vertex of the first cell and the bottom left vertex of the second cell; the second type of matching points includes the bottom left vertex of the first cell and the top left vertex of the second cell, and the bottom right vertex of the first cell and the top right vertex of the second cell; then the first matching coordinate difference includes: the first row and column coordinate difference between the top right vertex of the first cell and the top left vertex of the second cell, and the second row and column coordinate difference between the bottom right vertex of the first cell and the bottom left vertex of the second cell; the second matching coordinate difference includes: the bottom left vertex of the first cell and the bottom left vertex of the second cell; the first matching coordinate difference includes: the first row and column coordinate difference between the top right vertex of the first cell and the top left vertex of the second cell; the second matching coordinate difference includes: the first row and column coordinate difference between the bottom left vertex of the first cell and the bottom left vertex of the second cell; the second matching coordinate difference includes: the first row and column coordinate difference between the top right ... third matching coordinate difference includes: the first row and column coordinate difference between the top right vertex of the first cell and the bottom left vertex of the second cell; the fourth matching coordinate difference includes: the first row and column coordinate difference between the top right vertex of the first cell and the bottom left vertex of the second cell; the fifth matching coordinate difference includes: the first row and column coordinate difference between the top right vertex of the first cell and the bottom left vertex of the second cell; the sixth matching coordinate difference includes: the first row and column coordinate difference between the top right vertex of the first cell and the bottom left vertex of the second cell; the seventh matching coordinate difference includes: the first row and column coordinate difference between the top right vertex of the first cell and the bottom left vertex of the second cell; the eighth matching coordinate difference The first cell and the second cell are associated within the same row if the first row and the second cell have the same row and column coordinates, and the second cell have the same row and column coordinates, respectively. If, in the row direction, both the first row and the second row and column coordinate differences are less than the length error threshold in the row direction and both are less than the width error threshold in the column direction, then the first cell and the second cell are associated within the same row. Similarly, if, in the column direction, both the third row and the fourth row and column coordinate differences are less than the length error threshold in the row direction and both are less than the width error threshold in the column direction, then the first cell and the second cell are associated within the same column. Alternatively, the first type of matching point and the second type of matching point are the cell center points; then the first matching coordinate difference and the second matching coordinate difference are both the cell center point coordinate differences between the first cell and the second cell; the first error is obtained by subtracting the row-direction coordinate difference from the average length, and the second error is obtained by subtracting the column-direction coordinate difference from the average width; in the row direction, if the first error is less than the length error threshold, then the first cell and the second cell are marked as being associated within the same row; in the column direction, if the second error is less than the width error threshold, then the first cell and the second cell are marked as being associated within the same column.

2. The method for establishing table topological relationships according to claim 1, characterized in that, The tolerance rate can be a fixed tolerance rate or a dynamic tolerance rate; The formula for calculating the fixed tolerance rate is: ； in, This represents the fixed tolerance rate, and N represents the total number of cells in the analysis table image. This is the adjustment coefficient; The formula for calculating the dynamic tolerance rate is: ； in, This represents the dynamic tolerance rate. Where Q is the base tolerance, S is the image quality coefficient, M is the table structure complexity coefficient, and M is the real-time matching success rate. , and These are the weighting coefficients.

3. The method for establishing table topological relationships according to claim 2, characterized in that, The steps for calculating the image quality coefficient include: The image of the table to be analyzed is divided into blocks and the standard deviation of the brightness of each block is calculated as the illumination uniformity. The gradient magnitude is calculated by convolution using the Sobel operator on the grayscale image of the table to be analyzed. A gradient magnitude matrix is ​​constructed according to the spatial position of the original image, and the gradient variance is calculated based on the gradient magnitude matrix. The edge sharpness is obtained by assigning values ​​based on the preset gradient variance interval. Canny edge detection and Hough line detection are performed on the grayscale image of the table to be analyzed. The angles of the table lines in the image are extracted and the deviations from the reference angles are calculated as the perspective distortion. The image quality coefficient is obtained by normalizing the illumination uniformity, edge sharpness, and perspective distortion, and then weighting and summing them.

4. The method for establishing table topological relationships according to claim 2, characterized in that, The steps for calculating the complexity coefficient of the table structure include: The difference between the total number of cells in the table image to be analyzed and the image area is used as the cell number density; The proportion of nested tables is obtained by calculating the proportion of cells containing sub-tables; The table structure complexity coefficient is obtained by normalizing the cell density and the proportion of nested tables and then weighting and summing them.

5. The method for establishing table topological relationships according to claim 2, characterized in that, The method further includes: when the real-time matching success rate is less than a set value, increasing the dynamic tolerance rate by a preset ratio to relax the restriction.

6. The method for establishing table topological relationships according to claim 1, characterized in that, The average length is replaced by the median or mode of the length of each cell; the average width is replaced by the median or mode of the width of each cell; the table structure recognition model is the Cycle-CenterNet model.

7. A table topology relationship establishment device, comprising a processor, a memory, and a computer program / instructions stored in the memory, characterized in that, The processor is configured to execute the computer program / instructions, and when the computer program / instructions are executed, the device implements the steps of the method as described in any one of claims 1 to 6.

8. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method described in any one of claims 1 to 6.

Images