Table topological relation establishing method and device and computer program product

Abstract

The invention provides a table topological relation establishing method and device and a computer program product, and relates to the technical field of electric digital data processing. Vertex coordinates of cells are recognized, matching point positions are selected to calculate a coordinate difference value, and the coordinate difference value is compared with a length error threshold value and a width error threshold value; according to the method, the association relationship between the cells in the row direction and the column direction is identified, the coordinate values of the specified mark point positions are sorted in the row or the column, and row and column division and spatial relationship reconstruction are directly performed based on geometric rules, so that the problem of performance reduction caused by overlarge image size is reduced, and the method does not depend on the integrity or definition of a table frame; the method can be suitable for processing a frameless table and a complex table scene containing a large number of empty rows and empty columns, and the processing speed is remarkably improved.

Description

Technical Field

[0001] The present invention relates to the technical field of electronic digital data processing, and in particular to a method and device for establishing a table topological relationship, and a computer program product. Background Art

[0002] In the field of smart finance, automatic parsing of financial statements is a key step in achieving intelligent processing of corporate financial data. Financial statements are typically presented in tabular format and contain large amounts of structured data, requiring the mining of table topological relationships. Traditional table structure recognition methods primarily rely on deep learning technologies, such as models based on object detection or semantic segmentation. These methods analyze the overall features of table images, identify table areas, segment cells, and then use subsequent algorithms to restore the table structure information. However, for long or complex tables, such as those with tilted or curved borders or that span multiple pages, deep learning models face numerous challenges in practical application.

[0003] The current mainstream table structure recognition technologies are mainly divided into the following categories: 1) Object detection-based methods, such as DeepTabStR and TableNet, treat table rows and columns as objects to be detected and use object detection networks for detection. These methods assume that tables are well-aligned and that the bounding boxes of detected cells are rectangular, making them difficult to handle tables in natural scenes. 2) Semantic segmentation-based methods, such as TableNet, use semantic segmentation to obtain column position information within a table and then use heuristic methods to obtain row information to identify the table structure. This approach is often particularly effective for identifying and extracting tables from complex documents, but the unclear boundaries and sparse visual cues of table regions limit these methods' ability to accurately locate table segments. 3) Graph network-based methods, such as TIES and GFTE, treat text blocks as graph vertices and the relationships between text blocks as graph edges. Graph neural networks are used to predict the row and column positional relationships between text blocks. These methods do not rely on information such as table borders or image features, and thus have stronger generalization capabilities. However, due to the limitations of the graph network structure, they generally do not fully utilize spatial feature information, and local spatial information is easily lost during graph network inference. 4) Text box-based graph modeling methods, such as GEAN-TSR, first use OCR results as a basis to create a graph structure model for the text box. Graph neural networks are then used for inference to restore the table structure. This method does not rely on table borders, but has poor anti-interference capabilities and is difficult to handle complex structures in tables.

[0004] As can be seen, table processing solutions based on deep learning models require a large amount of labeled data for training, are highly data-dependent, have high segmentation error rates for complex tables, and are prone to losing local spatial information, leading to misjudgments. Furthermore, the model has a large number of parameters, consumes a lot of resources when processing long tables, and has poor real-time performance. Furthermore, they are poorly adaptable to special scenarios, such as borderless tables and tables with large numbers of blank rows and columns. Therefore, a new solution for processing table topology relationships is urgently needed. Summary of the Invention

[0005] In view of this, embodiments of the present invention provide a method, device and computer program product for establishing table topological relationships to eliminate or improve one or more defects existing in the prior art and solve the problem that the prior art requires high computing resources during table processing and has poor adaptability to complex tables.

[0006] In one aspect, the present invention provides a method for establishing a table topology relationship, the method comprising the following steps: Extracting identification points and coordinates of each cell in the table image to be analyzed based on optical character recognition or a preset table structure recognition model, wherein the coordinates of the identification points include vertex coordinates and / or center coordinates; the vertex coordinates include the upper left vertex, the lower left vertex, the upper right vertex, and the lower right vertex; Calculate the average length of each cell in the row direction and the average width of each cell in the column direction based on the vertex coordinates of each cell, and multiply them by the tolerance rate to obtain the length error threshold and the width error threshold; Traverse all cells and calculate, between each pair of cells, a first matching coordinate difference between first-type matching points corresponding to positions of a first cell and a second cell in the row direction, and a second matching coordinate difference between second-type matching points corresponding to positions in the column direction; in the row direction, if there is a first matching coordinate difference that is less than the length error threshold in the row direction and less than the width error threshold in the column direction, then mark the first cell and the second cell as being associated in the same row; in the column direction, if there is a second matching coordinate difference that is less than the length error threshold in the row direction and less than the width error threshold in the column direction, then mark the first cell and the second cell as being associated in the same column; Sort multiple cells associated in the same row first along the horizontal coordinate and then along the vertical coordinate based on the first designated marking point, and sort multiple cells associated in the same column first along the vertical coordinate and then along the horizontal coordinate based on the second designated marking point, and reconstruct a topological structure according to the arrangement relationship between the rows and columns of each cell in the table image to be analyzed; The first type of matching points, the second type of matching points, the first designated marking points and the second designated marking points are selected from the identification points.

[0007] In some embodiments, the tolerance rate adopts a fixed tolerance rate or a dynamic tolerance rate; The calculation formula of the fixed tolerance rate is: ; in, represents the fixed tolerance rate, N represents the total number of cells in the analysis table image, is the adjustment coefficient; The calculation formula of the dynamic tolerance rate is: ; in, represents the dynamic tolerance rate, is the basic tolerance rate, Q is the image quality coefficient, S is the table structure complexity coefficient, and M is the real-time matching success rate; 、 and is the weight coefficient.

[0008] In some embodiments, the step of calculating the image quality coefficient includes: Dividing the table image to be analyzed into blocks and calculating the standard deviation of the brightness of the blocks as the illumination uniformity; The grayscaled table image to be analyzed is convolved with the Sobel operator to calculate the gradient amplitude, a gradient amplitude matrix is ​​constructed according to the spatial position of the original image, and the gradient variance is calculated based on the gradient amplitude matrix, and the edge clarity is obtained by performing an assignment based on a preset gradient variance interval; Perform Canny edge detection and Hough line detection on the grayscaled table image to be analyzed, extract the angle of the straight lines in the table in the image, and calculate the deviation from the reference angle as the perspective distortion; The illumination uniformity, the edge clarity, and the perspective distortion are normalized and then weighted and summed to obtain the image quality coefficient.

[0009] In some embodiments, the step of calculating the table complexity coefficient includes: Calculating the difference between the total number of cells in the table image to be analyzed and the image area as the cell number density; Count the proportion of cells containing subtables to get the proportion of nested tables; The cell number density and the nested table ratio are normalized and then weighted summed to obtain the table complexity coefficient.

[0010] In some embodiments, the method further includes: when the real-time matching success rate is less than a set value, increasing the dynamic tolerance rate according to a preset ratio to relax restrictions.

[0011] In some embodiments, the first type of matching points includes the upper right vertex of the first cell and the upper left vertex of the second cell, and the lower right vertex of the first cell and the lower left vertex of the second cell; The second type of matching points includes the lower left vertex of the first cell and the upper left vertex of the second cell, and the lower right vertex of the first cell and the upper right vertex of the second cell; The first matching coordinate difference includes: a first row and column coordinate difference between the upper right vertex of the first cell and the upper left vertex of the second cell, and a second row and column coordinate difference between the lower right vertex of the first cell and the lower left vertex of the second cell; The second matching coordinate difference includes: a third row and column coordinate difference between the lower left vertex of the first cell and the upper left vertex of the second cell, and a fourth row and column coordinate difference between the lower right vertex of the first cell and the upper right vertex of the second cell; In the row direction, if the first row and column coordinate difference and the second row and column coordinate difference are both smaller than the length error threshold in the row direction and smaller than the width error threshold in the column direction, then the first cell and the second cell are marked as associated in the same row; in the column direction, if the third row and column coordinate difference and the fourth row and column coordinate difference are both smaller than the length error threshold in the row direction and smaller than the width error threshold in the column direction, then the first cell and the second cell are marked as associated in the same column.

[0012] In some embodiments, the first type of matching points and the second type of matching points are cell center points; The first matching coordinate difference and the second matching coordinate difference are both cell center coordinate differences between the first cell and the second cell; a first error is obtained by subtracting a row-wise coordinate difference from the average length in the cell center coordinate differences, and a second error is obtained by subtracting a column-wise coordinate difference from the average width in the cell center coordinate differences; In the row direction, if the first error is less than the length error threshold, the first cell and the second cell are marked as associated in the same row; in the column direction, if the second error is less than the width error threshold, the first cell and the second cell are marked as associated in the same column.

[0013] In some embodiments, the average length is replaced by the median or mode of each cell length; the average width is replaced by the median or mode of each cell width; and the table structure recognition model is a Cycle-CenterNet model.

[0014] On the other hand, the present invention also provides a device for establishing table topology relationships, comprising a processor, a memory, and a computer program / instruction stored in the memory, wherein the processor is used to execute the computer program / instruction, and when the computer program / instruction is executed, the device implements the steps of the above method.

[0015] On the other hand, the present invention also provides a computer program product, comprising a computer program / instruction, which implements the steps of the above method when executed by a processor.

[0016] The table topology relationship establishment method, device and computer program product described in the present invention identify the vertex coordinates of cells, select matching points to calculate the coordinate difference and compare it with the length error threshold and width error threshold, identify the association between cells in the row and column directions, and sort within the row or column by specifying the coordinate values ​​of the marked points. The row and column division and spatial relationship reconstruction are directly based on geometric rules, reducing the performance degradation problem caused by excessive image size, not relying on the integrity or clarity of the table border, and can adapt to processing borderless tables and complex table scenarios containing a large number of empty rows and columns, and the processing speed is significantly improved.

[0017] Additional advantages, objects, and features of the present invention will be set forth in part in the following description and will become apparent to those skilled in the art upon examination of the following or may be learned from practice of the present invention. The objects and other advantages of the present invention may be realized and obtained by the structures particularly pointed out in the description and drawings.

[0018] Those skilled in the art will understand that the purposes and advantages that can be achieved by the present invention are not limited to the above specific descriptions, and the above and other purposes that can be achieved by the present invention will be more clearly understood based on the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The drawings described herein are used to provide a further understanding of the present invention, constitute a part of this application, and do not constitute a limitation of the present invention. In the drawings: Figure 1 The figure is a flowchart of a method for establishing a table topology relationship according to an embodiment of the present invention.

[0020] Figure 2 The figure is a schematic diagram of the process of edge clarity analysis of a table image to be analyzed in the table topology relationship establishment method according to one embodiment of the present invention.

[0021] Figure 3 The figure is a schematic diagram of a process for analyzing the perspective distortion of a table image to be analyzed in a method for establishing a table topological relationship according to an embodiment of the present invention. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments and the accompanying drawings. Here, the exemplary embodiments of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention.

[0023] It should also be noted that, in order to avoid obscuring the present invention due to unnecessary details, the accompanying drawings only show structures and / or processing steps closely related to the solutions according to the present invention, while other details that are not closely related to the present invention are omitted.

[0024] It should be emphasized that the term "include / comprises" when used herein refers to the existence of features, elements, steps or components, but does not exclude the existence or addition of one or more other features, elements, steps or components.

[0025] Existing technologies for table processing based on deep neural networks (DNNs) identify the topological relationships between cells within a table. This requires a large amount of labeled data for pre-training, resulting in weak generalization capabilities. The resulting models are sensitive to differences in table layouts and are difficult to adapt to different scenarios. Among them, graphical model methods based on text boxes are prone to losing local spatial information during inference, leading to structural misjudgments. Furthermore, due to the large number of model parameters, resource consumption is high when processing long tables, and real-time performance is poor. For example, deep learning models such as DCNNs require a huge amount of computation when processing long tables, making it difficult to meet the needs of actual applications.

[0026] In view of this, the present invention provides a method for establishing a table topology relationship, such as Figure 1 As shown, the method includes the following steps S101 to S104: Step S101: Based on optical character recognition or a preset table structure recognition model, the identification points and coordinates of each cell in the table image to be analyzed are extracted, where the coordinates of the identification points include vertex coordinates and / or center point coordinates; the vertex coordinates include the upper left vertex, lower left vertex, upper right vertex and lower right vertex.

[0027] Step S102: Calculate the average length of each cell in the row direction and the average width of each cell in the column direction based on the vertex coordinates of each cell, and multiply them by the tolerance rate to obtain a length error threshold and a width error threshold.

[0028] Step S103: traverse all cells, and calculate the first matching coordinate difference between the first type of matching points corresponding to the positions of the first cell and the second cell in the row direction, and the second matching coordinate difference between the second type of matching points corresponding to the positions in the column direction between each two cells; in the row direction, if there is a first matching coordinate difference that is less than the length error threshold in the row direction and less than the width error threshold in the column direction, then mark the first cell and the second cell as associated in the same row; in the column direction, if there is a second matching coordinate difference that is less than the length error threshold in the row direction and less than the width error threshold in the column direction, then mark the first cell and the second cell as associated in the same column.

[0029] Step S104: Sort multiple cells associated in the same row first along the horizontal coordinate and then along the vertical coordinate based on the first specified marking point, and sort multiple cells associated in the same column first along the vertical coordinate and then along the horizontal coordinate based on the second specified marking point, and reconstruct the topological structure according to the arrangement relationship between the rows and columns of each cell in the table image to be analyzed.

[0030] Among them, the first type of matching points, the second type of matching points, the first designated marking points and the second designated marking points are selected from the identification points.

[0031] In step S101, the vertex coordinates of the cell are identified based on optical character recognition (OCR) or the Cycle-CenterNet table structure recognition model. Optical character recognition (OCR) is a technology that converts text information in an image into editable text, and is widely used in the digital processing of documents such as bills, certificates, tables, and books. Cycle-CenterNet is a wired table structure recognition model that focuses on extracting the physical coordinates (four vertices) and logical structure (row and column numbers) of table cells from images. The parameter scale of Cycle-CenterNet is relatively small, and it is a lightweight model that is suitable for deployment in resource-constrained environments. It mainly identifies typical identification points that have the ability to locate cells, including four vertices and the center point of the cell.

[0032] In step S102, based on the identified upper left vertex, lower left vertex, upper right vertex, and lower right vertex of each cell, the length in the row direction and the width in the column direction are calculated for each cell. The average length and average width are calculated, and based on these, a length error threshold and a width error threshold are calculated. In other embodiments, the median or mode of the length or width of each cell may also be used to calculate the length error threshold and the width error threshold.

[0033] Furthermore, when calculating the length error threshold and the width error threshold, the tolerance rate may adopt a fixed tolerance rate or a dynamic tolerance rate.

[0034] The calculation formula for the fixed tolerance rate is: ; in, represents a fixed tolerance rate, N represents the total number of cells in the analyzed table image, For example, the fixed tolerance rate may be set to 0.05.

[0035] In order to better adapt to the processing requirements in different scenarios and obtain higher accuracy, the calculation formula of the dynamic tolerance rate can be set as: ; in, represents the dynamic tolerance rate, is the basic tolerance rate, Q is the image quality coefficient, S is the table structure complexity coefficient, and M is the real-time matching success rate; 、 and is the weight coefficient.

[0036] In this embodiment, the dynamic tolerance rate is adjusted by introducing the image quality coefficient, the table structure complexity coefficient, and the real-time matching success rate in combination with the basic tolerance rate. The specific coefficient acquisition method is as follows.

[0037] In some embodiments, the step of calculating the image quality coefficient includes steps S1021 to S1024: Step S1021: Divide the table image to be analyzed into blocks and calculate the standard deviation of the brightness of the blocks as the illumination uniformity.

[0038] Step S1022: Figure 2 As shown in the figure, the Sobel operator is used to convolve the grayscale image to be analyzed to calculate the gradient amplitude, the gradient amplitude matrix is ​​constructed according to the spatial position of the original image, and the gradient variance is calculated based on the gradient amplitude matrix. The edge clarity is obtained by assigning a value based on the preset gradient variance interval.

[0039] Step S1023: Figure 3 As shown, Canny edge detection and Hough line detection are performed on the grayscaled table image to be analyzed, and the angle of the straight line of the table in the image is extracted and the deviation from the reference angle is calculated as the perspective distortion.

[0040] Step S1024: normalize the illumination uniformity, edge clarity, and perspective distortion, and then perform weighted summation to obtain an image quality coefficient.

[0041] In step S1021, the input table image to be analyzed is divided into several equal-sized subregions. The average brightness of all pixels within each subregion is then calculated. Next, the standard deviation of the brightness averages of these subregions is calculated, and this standard deviation is used as an indicator of the image's illumination uniformity. A larger standard deviation indicates a more significant brightness difference between different image regions, meaning the illumination is more uneven. Conversely, a smaller standard deviation indicates a more even distribution of illumination across the entire image.

[0042] In step S1022, the table image to be analyzed is first converted into a grayscale image. The Sobel operator is then applied to the grayscale image for convolution, calculating the gradient amplitude of each pixel in the image. This reflects the intensity of the pixel brightness change, i.e., the edge strength, thereby obtaining a gradient amplitude image. Next, the variance of the gradient amplitudes of all pixels in this gradient amplitude image is calculated. The magnitude of this variance directly reflects the changes in the overall clarity of the image edge. A larger variance generally means that the image contains more clear, sharp edges and fewer blurred or smooth areas, i.e., clearer edges. Conversely, a smaller variance indicates blurred edges or low contrast. Finally, based on a preset gradient variance range, this calculated gradient variance value is mapped or assigned to a standardized score, which serves as a quantitative edge clarity indicator.

[0043] In step S1023, based on the grayscaled table image, the Canny edge detection algorithm is first used to identify the main edge contours in the image. Then, based on these detected edges, the Hough line detection algorithm is applied to identify and extract all significant straight line segments in the image. These straight line segments generally correspond to the frame lines of the table. Next, the angular deviation between these detected straight line segments and a preset reference angle is calculated. The reference angle is generally an ideal horizontal angle of 0° and a vertical angle of 90°. Finally, a statistical value, such as an average value, is calculated for the angular deviations of all detected table straight line segments. This statistical value is defined as the perspective distortion of the image. A greater degree of distortion indicates a more severe tilt or distortion of the table image due to shooting angle or lens distortion. A smaller degree of distortion indicates that the table is closer to being shot from a straight-on perspective and has less geometric distortion.

[0044] In step S1024, the three indicators of different physical meanings and dimensions—illumination uniformity obtained in step S1021, edge clarity obtained in step S1022, and perspective distortion obtained in step S1023—are normalized separately. Typically, each is mapped to the range [0, 1] to eliminate dimensionality effects and facilitate comparison and weighting. Preset weight coefficients are then assigned to each of these normalized indicators. Finally, the three weighted values ​​are summed to obtain the final image quality coefficient. This coefficient is a comprehensive value; higher values ​​indicate better overall image quality; lower values ​​indicate poorer image quality, and a larger tolerance may be required in subsequent table structure recognition to account for interference caused by image defects.

[0045] In some embodiments, the calculation step of the table complexity coefficient includes steps S1205 to S1027: Step S1205: Calculate the difference between the total number of cells in the table image to be analyzed and the image area as the cell number density.

[0046] Step S1206: Count the proportion of cells containing sub-tables to obtain the proportion of nested tables.

[0047] Step S1207: normalize the cell number density and the nested table ratio and then perform weighted summation to obtain the table complexity coefficient.

[0048] In step S1025, the complexity of the table structure is assessed by quantifying the cell distribution density. Specifically, the total number of cells identified in the table image to be analyzed is counted, and the total image area is calculated, which can be expressed in pixels. The total number of cells is divided by the total image area to obtain the number of cells per unit area, i.e., the cell density. A higher value indicates a denser table structure, smaller cell spacing, and greater difficulty in establishing topological relationships; conversely, a lower value indicates a sparser table structure and lower parsing difficulty.

[0049] In step S1026, nested table structures are detected to assess hierarchical complexity. All identified cells are traversed, and whether they are nested cells is determined by analyzing the internal area of ​​the cell to see whether there are sub-table features, such as internal lines or sub-cell outlines; the number of cells containing sub-tables is counted and divided by the total number of cells to obtain the nested table ratio. For example, a 100-cell table with 5 sub-tables has a nesting ratio of 0.05. The higher this ratio, the more complex the table hierarchy.

[0050] In step S1027, the cell density and nested table ratio are normalized separately, with the density mapped to the [0, 1] range and the nested table ratio converted directly to a [0, 1] value as a percentage. The two normalization results are weighted and summed according to preset weights, ultimately outputting a table complexity coefficient ranging from 0 to 1. The closer the coefficient is to 1, the more complex the table structure is. Subsequent topology establishment requires a higher tolerance threshold to mitigate matching interference caused by dense cells or nested relationships.

[0051] Furthermore, the real-time matching success rate is the ratio of detected associations within the same row and the same column between cells, which is initially set to 100% and is recalculated after each round of matching action in step S103 is executed.

[0052] In some embodiments, the method further includes: when the real-time matching success rate is less than a set value, increasing the dynamic tolerance rate according to a preset ratio to relax the restriction.

[0053] In step S103, there are at least three types of positional relationships between cells, the first being an associated connection in the row direction, the second being an associated connection in the column direction, and the third being no connection. In order to identify the positional relationship between cells, the present invention makes a judgment based on the relative positions of matching points between cells. Specifically, matching points are points where the relative positions of two adjacent cells are determined, such as overlapping points. Exemplarily, for two cells associated along the row direction, the upper right vertex and lower right vertex of the first cell overlap with the upper left vertex and lower left vertex of the second cell, respectively. For two cells associated along the column direction, the lower left vertex and lower right vertex of the first cell overlap with the upper left vertex and upper right vertex of the second cell, respectively.

[0054] Therefore, in some embodiments, the first type of matching points includes the upper right vertex of the first cell and the upper left vertex of the second cell, as well as the lower right vertex of the first cell and the lower left vertex of the second cell. The second type of matching points includes the lower left vertex of the first cell and the upper left vertex of the second cell, as well as the lower right vertex of the first cell and the upper right vertex of the second cell.

[0055] The first matching coordinate difference includes: a first row and column coordinate difference between the upper right vertex of the first cell and the upper left vertex of the second cell, and a second row and column coordinate difference between the lower right vertex of the first cell and the lower left vertex of the second cell.

[0056] The second matching coordinate difference includes: the third row and column coordinate difference between the lower left vertex of the first cell and the upper left vertex of the second cell, and the fourth row and column coordinate difference between the lower right vertex of the first cell and the upper right vertex of the second cell.

[0057] It should be emphasized that the first row and column coordinate difference, the second row and column coordinate difference, the third row and column coordinate difference, and the fourth row and column coordinate difference all include differences along the row direction and differences along the column direction.

[0058] In the row direction, if the first row and column coordinate difference and the second row and column coordinate difference are both smaller than the length error threshold in the row direction and smaller than the width error threshold in the column direction, then the first cell and the second cell are marked as associated in the same row; in the column direction, if the third row and column coordinate difference and the fourth row and column coordinate difference are both smaller than the length error threshold in the row direction and smaller than the width error threshold in the column direction, then the first cell and the second cell are marked as associated in the same column.

[0059] In some embodiments, the first type of matching point and the second type of matching point are cell center points; then the first matching coordinate difference and the second matching coordinate difference are both cell center coordinate differences between the first cell and the second cell; the coordinate difference in the row direction of the cell center coordinate difference is subtracted from the average length to obtain a first error, and the coordinate difference in the column direction of the cell center coordinate difference is subtracted from the average width to obtain a second error; in the row direction, if the first error is less than the length error threshold, the first cell and the second cell are marked as associated in the same row; in the column direction, if the second error is less than the width error threshold, the first cell and the second cell are marked as associated in the same column.

[0060] In step S104, all cells marked as associated in the same row are grouped, with each group representing a row; the horizontal coordinate of the upper left vertex of each cell in the group is taken as the first designated marking point, and the cells are arranged in ascending order along the horizontal coordinate and then along the vertical coordinate according to the coordinate to form a continuous row sequence from left to right.

[0061] Group all cells marked as associated in the same column, with each group representing a column. Take the ordinate of the upper left vertex of each cell in the group as the second designated marking point, and arrange the cells in ascending order along the ordinate and then along the abscissa according to the coordinates to form a continuous column sequence from bottom to top.

[0062] Using the row and column sequence as the skeleton, each cell is mapped to its row and column index, generating a two-dimensional matrix structure. For cells that merge across rows or columns, the index is dynamically expanded based on the row and column range covered by the vertex coordinates, ultimately outputting a table topology with merged relationships.

[0063] On the other hand, the present invention also provides a device for establishing table topology relationships, comprising a processor, a memory, and a computer program / instruction stored in the memory, wherein the processor is used to execute the computer program / instruction, and when the computer program / instruction is executed, the device implements the steps of the above method.

[0064] On the other hand, the present invention also provides a computer program product, comprising a computer program / instruction, which implements the steps of the above method when executed by a processor.

[0065] The present invention will be described below in conjunction with a specific embodiment: This embodiment provides a table structure topology reconstruction method based on vertex coordinate relationships, which specifically includes the following steps S1 to S4: Step S1: Vertex coordinate acquisition: First, use OCR or a table structure recognition model to obtain the vertex coordinates of each cell. The formats are (upper left x, upper left y), (upper right x, upper right y), (lower right x, lower right y), and (lower left x, lower left y). The vertex coordinates can be replaced by the coordinates of the cell center point.

[0066] Step S2: Setting the error threshold: Based on the average cell size avg_cell_size (i.e., the average diagonal length), set the error threshold δ = 0.05 × avg_cell_size. This dynamic tolerance can adapt to the processing requirements of tables of different sizes.

[0067] Step S3: Spatial continuity determination, by comparing the spatial relationship between the vertex coordinates of adjacent cells to determine whether they belong to the same row or column.

[0068] For row determination: compare the coordinates of the two vertices on the right side of the current cell with the coordinates of the two vertices on the left side of the next cell. If they are the same or the difference is within the error threshold δ, they are determined to be in the same row.

[0069] Specifically, all cells are traversed and the following processing is performed on each cell: the coordinates of the top right and bottom right vertices of the current cell are taken, the coordinates of the top left and bottom left vertices of the next possible cell are found, the coordinate difference between the right vertex of the current cell and the left vertex of the next cell is calculated; if the difference is less than or equal to the error threshold δ, the cells are determined to be in the same row; the determination result is recorded, and a row association relationship is established between the cells. In some embodiments, other vertex combinations can also be used. The distance between coordinates can use Manhattan distance or Euclidean distance.

[0070] For column determination: compare the coordinates of the lower left and lower right vertices of the current cell with the coordinates of the upper left and upper right vertices of the next cell. If they are the same or the difference is within the error threshold δ, they are determined to be in the same column.

[0071] Traverse all cells and perform the following processing on each cell: obtain the coordinates of the lower left and lower right vertices of the current cell; find the next possible cell and obtain the coordinates of its upper left and upper right vertices; calculate the coordinate difference between the lower vertex of the current cell and the upper vertex of the next cell; if the difference is less than or equal to the error threshold δ, determine that they are in the same column; record the judgment result and establish a column association relationship between cells.

[0072] In some embodiments, an adjacency list storage structure may be used instead of vertex coordinate comparison.

[0073] Step S4: Topology Reconstruction: Based on the spatial continuity determination results, the table's row and column topology is reconstructed. Specifically, rows are sorted in ascending order, first by x and then by y, to ensure the correct order of cells within the row. Columns are sorted in ascending order, first by y and then by x, to ensure the correct order of cells within the column. Bubble sort or insertion sort can be used for sorting.

[0074] For cells that are determined to be in the same row, sort them according to the following rules: first, sort them in ascending order according to the x-coordinate of the upper left vertex; if the x-coordinates are the same, sort them in ascending order according to the y-coordinate of the upper left vertex; this ensures the correct order of cells within the row.

[0075] For cells that are determined to be in the same column, sort them according to the following rules: first, sort them in ascending order according to the y coordinate of the upper left vertex; if the y coordinates are the same, sort them in ascending order according to the x coordinate of the upper left vertex; this ensures the correct order of cells within the column.

[0076] Based on the row association and column association determination results, the row and column topology of the table is reconstructed: the row-associated cells are organized into rows according to the row sorting results, and the column-associated cells are organized into columns according to the column sorting results, ultimately forming the row and column logical structure of the table.

[0077] This embodiment reconstructs the spatial relationship of vertex coordinates and directly divides rows and columns based on geometric rules, thereby reducing the performance degradation problem caused by excessive image size. Spatial relationships are determined directly through the vertex coordinates of the cells, without relying on the integrity or clarity of the table border. By reconstructing the spatial relationship of vertex coordinates, it is possible to handle tilted or curved table layouts that are common in practical applications. By reconstructing the spatial relationship of vertex coordinates, the table structure is directly reconstructed according to geometric rules, reducing the amount of calculation. Experiments show that the processing speed of the method of this embodiment is 3 to 5 times faster than that of traditional methods, and is particularly suitable for application scenarios with high real-time requirements. At the same time, it can effectively handle special scenarios such as borderless tables and tables containing a large number of blank rows and columns.

[0078] In summary, the table topology relationship establishment method, device and computer program product described in the present invention identify the vertex coordinates of the cells, select matching points to calculate the coordinate difference and compare it with the length error threshold and width error threshold, identify the association between the cells in the row and column directions, and sort them within the row or column by specifying the coordinate values ​​of the marked points. It directly divides the rows and columns and reconstructs the spatial relationship based on geometric rules, reduces the performance degradation problem caused by the large image size, does not rely on the integrity or clarity of the table border, can adapt to the processing of borderless tables and complex table scenes containing a large number of empty rows and columns, and significantly improves the processing speed.

[0079] Corresponding to the above method, the present invention also provides an apparatus / system, which includes a computer device, the computer device includes a processor and a memory, the memory stores computer instructions, and the processor is used to execute the computer instructions stored in the memory. When the computer instructions are executed by the processor, the apparatus / system implements the steps of the method described above.

[0080] An embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon. When executed by a processor, the computer program implements the steps of the aforementioned edge computing server deployment method. The computer-readable storage medium can be a tangible storage medium, such as a random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, register, floppy disk, hard disk, removable storage disk, CD-ROM, or any other form of storage medium known in the art.

[0081] It should be understood by those skilled in the art that the various exemplary components, systems, and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software, or a combination of both. Whether to implement the system in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present invention. When implemented in hardware, it may be, for example, an electronic circuit, an application-specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present invention are programs or code segments used to perform the required tasks. The programs or code segments can be stored in a machine-readable medium or transmitted over a transmission medium or communication link via a data signal carried in a carrier wave.

[0082] It should be understood that the present invention is not limited to the specific configurations and processes described above and illustrated in the figures. For the sake of brevity, a detailed description of known methods is omitted. In the above embodiments, several specific steps are described and illustrated as examples. However, the method of the present invention is not limited to the specific steps described and illustrated. Those skilled in the art may make various changes, modifications, and additions, or change the order of the steps after understanding the spirit of the present invention.

[0083] In the present invention, features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, and / or combined with or replace features of other embodiments.

[0084] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of protection of the present invention.

Claims

1. A method for establishing a table topology relationship, characterized in that: The method comprises the following steps: Extracting identification points and coordinates of each cell in the table image to be analyzed based on optical character recognition or a preset table structure recognition model, wherein the coordinates of the identification points include vertex coordinates and / or center coordinates; the vertex coordinates include the upper left vertex, the lower left vertex, the upper right vertex, and the lower right vertex; Calculate the average length of each cell in the row direction and the average width of each cell in the column direction based on the vertex coordinates of each cell, and multiply them by the tolerance rate to obtain the length error threshold and the width error threshold; Traverse all cells and calculate, between each pair of cells, a first matching coordinate difference between first-category matching points corresponding to positions of a first cell and a second cell in the row direction, and a second matching coordinate difference between second-category matching points corresponding to positions in the column direction; in the row direction, if there is a first matching coordinate difference that is less than the length error threshold in the row direction and less than the width error threshold in the column direction, then mark the first cell and the second cell as being associated in the same row; in the column direction, if there is a second matching coordinate difference that is less than the length error threshold in the row direction and less than the width error threshold in the column direction, then mark the first cell and the second cell as being associated in the same column; Sort multiple cells associated in the same row first along the horizontal coordinate and then along the vertical coordinate based on the first designated marking point, and sort multiple cells associated in the same column first along the vertical coordinate and then along the horizontal coordinate based on the second designated marking point, and reconstruct a topological structure according to the arrangement relationship between the rows and columns of each cell in the table image to be analyzed; The first type of matching points, the second type of matching points, the first designated marking points and the second designated marking points are selected from the identification points.

2. The method for establishing a table topology relationship according to claim 1, wherein: The tolerance rate adopts a fixed tolerance rate or a dynamic tolerance rate; The calculation formula of the fixed tolerance rate is: ； in, represents the fixed tolerance rate, N represents the total number of cells in the analysis table image, is the adjustment coefficient; The calculation formula of the dynamic tolerance rate is: ； in, represents the dynamic tolerance rate, is the basic tolerance rate, Q is the image quality coefficient, S is the table structure complexity coefficient, and M is the real-time matching success rate; 、 and is the weight coefficient.

3. The method for establishing a table topology relationship according to claim 2, wherein: The steps of calculating the image quality coefficient include: Dividing the table image to be analyzed into blocks and calculating the standard deviation of the brightness of the blocks as the illumination uniformity; The grayscaled table image to be analyzed is convolved with the Sobel operator to calculate the gradient amplitude, a gradient amplitude matrix is ​​constructed according to the spatial position of the original image, and the gradient variance is calculated based on the gradient amplitude matrix, and the edge clarity is obtained by performing an assignment based on a preset gradient variance interval; Perform Canny edge detection and Hough line detection on the grayscaled table image to be analyzed, extract the angle of the straight lines in the table in the image, and calculate the deviation from the reference angle as the perspective distortion; The illumination uniformity, the edge clarity, and the perspective distortion are normalized and then weighted and summed to obtain the image quality coefficient.

4. The method for establishing a table topology relationship according to claim 2, wherein: The calculation steps of the table complexity coefficient include: Calculating the difference between the total number of cells in the table image to be analyzed and the image area as the cell number density; Count the proportion of cells containing subtables to get the proportion of nested tables; The cell number density and the nested table ratio are normalized and then weighted summed to obtain the table complexity coefficient.

5. The method for establishing a table topology relationship according to claim 2, wherein: The method further includes: when the real-time matching success rate is less than a set value, increasing the dynamic tolerance rate according to a preset ratio to relax restrictions.

6. The method for establishing a table topology relationship according to claim 1, wherein: The first type of matching points includes the upper right vertex of the first cell and the upper left vertex of the second cell, and the lower right vertex of the first cell and the lower left vertex of the second cell; The second type of matching points includes the lower left vertex of the first cell and the upper left vertex of the second cell, and the lower right vertex of the first cell and the upper right vertex of the second cell; The first matching coordinate difference includes: a first row and column coordinate difference between the upper right vertex of the first cell and the upper left vertex of the second cell, and a second row and column coordinate difference between the lower right vertex of the first cell and the lower left vertex of the second cell; The second matching coordinate difference includes: a third row and column coordinate difference between the lower left vertex of the first cell and the upper left vertex of the second cell, and a fourth row and column coordinate difference between the lower right vertex of the first cell and the upper right vertex of the second cell; In the row direction, if the first row and column coordinate difference and the second row and column coordinate difference are both smaller than the length error threshold in the row direction and smaller than the width error threshold in the column direction, then the first cell and the second cell are marked as associated in the same row; in the column direction, if the third row and column coordinate difference and the fourth row and column coordinate difference are both smaller than the length error threshold in the row direction and smaller than the width error threshold in the column direction, then the first cell and the second cell are marked as associated in the same column.

7. The method for establishing a table topology relationship according to claim 1, wherein: The first type of matching points and the second type of matching points are cell center points; The first matching coordinate difference and the second matching coordinate difference are both cell center coordinate differences between the first cell and the second cell; a first error is obtained by subtracting a row-wise coordinate difference from the average length in the cell center coordinate differences, and a second error is obtained by subtracting a column-wise coordinate difference from the average width in the cell center coordinate differences; In the row direction, if the first error is less than the length error threshold, the first cell and the second cell are marked as associated in the same row; in the column direction, if the second error is less than the width error threshold, the first cell and the second cell are marked as associated in the same column.

8. The method for establishing a table topology relationship according to claim 1, wherein: The average length is replaced by the median or mode of each cell length; the average width is replaced by the median or mode of each cell width; and the table structure recognition model is a Cycle-CenterNet model.

9. A device for establishing a table topology relationship, comprising a processor, a memory, and a computer program / instruction stored in the memory, characterized in that: The processor is configured to execute the computer program / instructions. When the computer program / instructions are executed, the device implements the steps of the method according to any one of claims 1 to 8.

10. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images