A method, system, device and storage medium for table structure recognition

By using the pre-trained semantic segmentation model deeplabV3+ for semantic segmentation and correction transformation of tabular structures, the problem of degradation of recognition accuracy caused by table structure deformation is solved, and a higher accuracy of table structure recognition is achieved.

CN114419643BActive Publication Date: 2024-12-03SOUTH CHINA UNIV OF TECH
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
CN202111563490.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-12-20
Publication Date
2024-12-03
Estimated Expiration
2041-12-20

AI Technical Summary

Technical Problem

When the existing table structure recognition method faces the deformation of the table structure, the recognition accuracy rate decreases and it is impossible to effectively deal with non-rectangular table structures.

Method used

The pre-trained semantic segmentation model deeplabV3+ is used to perform semantic segmentation of cell areas, table box line areas and table areas. By fusion of segmentation graphs and extracting control points, the structural priors are restored, and the connected domain analysis is performed to obtain the table structure. information.

Benefits of technology

It improves the accuracy of table structure recognition, can effectively handle the deformed table structure, and improves the recognition accuracy.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114419643B_ABST
    Figure CN114419643B_ABST
Patent Text Reader

Abstract

The present invention relates to the field of artificial intelligence and computers, and is a method, system, electronic device and storage medium for table structure recognition, the method comprising: using a pre-trained semantic segmentation model to perform semantic segmentation on a cell area, a table frame area and a table area of ​​an image to be recognized, and obtaining a cell area segmentation map, a table frame area segmentation map and a table area segmentation map; fusing the cell area segmentation map and the table frame area segmentation map to obtain a fused cell area segmentation map; extracting control points from the table area segmentation map to obtain a correction transformation; correcting the fused cell segmentation map using the correction transformation to obtain an aligned cell area segmentation map; performing connected domain extraction and analysis on the aligned cell area segmentation map, and obtaining table structure information according to matching conditions. The present invention can solve the impact of table structure deformation in table structure recognition scenarios and improve the accuracy of table structure recognition.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the fields of artificial intelligence and computers, and in particular to a method, system, device and storage medium for table structure recognition. Background Art

[0002] Tables are a form of data organization that is commonly used in daily life. Tables appear in a variety of scenarios, including electronic documents such as PDF, Excel, and scanned documents, as well as printed documents such as receipts and bills. In addition, there are more complex natural scenes such as food packaging and street posters. The structure of photographed tables in these scenes often has relatively serious deformation, which is the more challenging part of table structure recognition technology.

[0003] At present, mainstream table structure recognition methods, whether based on traditional image processing or deep learning, must be based on strong structural priors, that is, the table must be rectangular. Once deformation conditions such as those mentioned above are encountered, the structural prior is lost and the recognition accuracy will be seriously reduced. Summary of the invention

[0004] In order to solve the technical problems existing in the prior art, the present invention provides a method, system, device and storage medium for table structure recognition, which can solve the impact of table structure deformation in table structure recognition scenarios and improve the accuracy of table structure recognition.

[0005] The first object of the present invention is to provide a method for identifying a table structure.

[0006] A second object of the present invention is to provide a system for table structure recognition.

[0007] A third object of the present invention is to provide a computer device.

[0008] A fourth object of the present invention is to provide a storage medium.

[0009] The first object of the present invention can be achieved by adopting the following technical solutions:

[0010] A method for identifying a table structure, the method comprising:

[0011] S1. Use the pre-trained semantic segmentation model to perform semantic segmentation on the cell area, table border area and table area of ​​the image to be recognized, and obtain a cell area segmentation map, a table border area segmentation map, and a table area segmentation map;

[0012] S2, fusing the cell region segmentation map and the table border region segmentation map to obtain a fused cell region segmentation map;

[0013] S3, extracting control points from the table area segmentation map to obtain a correction transformation;

[0014] S4, using a correction transformation to correct the fused cell segmentation map to obtain an aligned cell region segmentation map;

[0015] S5. Perform connected domain extraction and analysis on the aligned cell region segmentation map, and obtain table structure information according to matching conditions.

[0016] Specifically, the pre-trained semantic segmentation model used is a neural network deeplabV3+, and step S1 specifically includes:

[0017] Change the multi-classification output of the last layer of the neural network deeplabV3+ to three binary classification outputs. The loss function is calculated relative to the cell area, table border area, and the annotated image of the table area, and the gradient is back-propagated.

[0018] When the neural network deeplabV3+ is forward-passed, it simultaneously outputs the semantic segmentation maps of the cell area, table border area, and table area.

[0019] Furthermore, the step S2 specifically includes:

[0020] S21, performing a pixel-level inversion operation on the table frame line region segmentation map to obtain an inverted table frame line region segmentation map;

[0021] S22, performing a pixel-level AND operation on the cell region segmentation map and the inverted table border region segmentation map to obtain a fused cell region segmentation map.

[0022] The second object of the present invention can be achieved by adopting the following technical solutions:

[0023] A system for table structure recognition, the system comprising:

[0024] A semantic segmentation module is used to use a pre-trained semantic segmentation model to perform semantic segmentation on the cell area, table frame area and table area of ​​the image to be recognized, and obtain a cell area segmentation map, a table frame area segmentation map and a table area segmentation map;

[0025] The segmentation map fusion module is used to fuse the cell area segmentation map and the table border area segmentation map to obtain a fused cell area segmentation map;

[0026] A control point extraction module is used to extract control points from the table area segmentation map to obtain a correction transformation;

[0027] The segmentation map correction module is used to correct the fused cell segmentation map using the correction transformation to obtain the aligned cell region segmentation map;

[0028] The extraction and analysis module is used to extract and analyze the connected domain of the aligned cell area segmentation map and obtain the table structure information according to the matching conditions.

[0029] The third object of the present invention can be achieved by adopting the following technical solutions:

[0030] A computer device comprises a processor and a memory for storing a program executable by the processor. When the processor executes the program stored in the memory, the above-mentioned table structure recognition method is implemented.

[0031] The fourth object of the present invention can be achieved by adopting the following technical solutions:

[0032] A storage medium stores a program, and when the program is executed by a processor, the above-mentioned table structure recognition method is implemented.

[0033] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0034] The present invention fuses the segmentation maps of the cell area and the table frame area to obtain a fused cell area segmentation map, and the cell area and the table frame area segmentation maps complement each other to improve the accuracy of table structure recognition; the table area segmentation map is subjected to control point extraction to obtain a correction transformation, and the structural prior is restored, which is conducive to the next step of processing and improves the accuracy of table structure recognition; the fused cell segmentation map is corrected using the correction transformation to obtain an aligned cell area segmentation map, and the aligned cell area segmentation map is subjected to connected domain extraction and analysis to obtain table structure information according to matching conditions. The present invention can solve the impact of table structure deformation in table structure recognition scenarios and improve the accuracy of table structure recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.

[0036] Figure 1 is an overall flow chart of the table structure recognition method in Embodiment 1 of the present invention;

[0037] Figure 2 Schematic diagram of a cell area segmentation diagram, a table frame line segmentation diagram, and a table area segmentation diagram in Example 1 of the present invention;

[0038] Figure 3A flow chart for generating a fused cell segmentation map in Example 1 of the present invention;

[0039] Figure 4 is a rendering of an image to be recognized in Example 1 of the present invention;

[0040] Figure 5 The cell region segmentation map obtained in Example 1 of the present invention;

[0041] Figure 6 The table frame line segmentation diagram obtained in Example 1 of the present invention;

[0042] Figure 7 Schematic diagram of the fused cell region segmentation map obtained in Example 1 of the present invention;

[0043] Figure 8 Schematic diagram of the cell area segmentation map before and after correction in Example 1 of the present invention;

[0044] Fig. 9 This is a specific flow chart of obtaining table structure information in Example 1 of the present invention. DETAILED DESCRIPTION

[0045] The technical solution of the present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments, and the implementation of the present invention is not limited thereto. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0046] Example 1

[0047] like Figure 1 As shown, this embodiment provides a table structure recognition method, comprising the following steps:

[0048] S1. Use the pre-trained semantic segmentation model to perform semantic segmentation on the cell area, table border area and table area of ​​the image to be recognized, and obtain the cell area segmentation map, table border area segmentation map and table area segmentation map.

[0049] The image to be identified in this embodiment is a scene image containing a table, including photos containing a table taken by a camera, such as printed receipts, bills, food packaging, street posters, etc., and also includes electronic documents such as PDF, Excel, and scanned documents.

[0050] The pre-trained semantic segmentation model described in this embodiment is a deep learning-based neural network deeplabV3+, which can classify and label images at the pixel level.

[0051] In this embodiment, the training data of the pre-trained semantic segmentation model is obtained from an existing public dataset.

[0052] Furthermore, the specific steps of simultaneously performing semantic segmentation on the cell area, table border area and table area of ​​the image to be identified include:

[0053] Change the purpose of the last layer of multi-classification output of deeplabV3+ to make it three binary classification outputs. The loss function is calculated relative to the cell area, table border area, and annotated image of the table area, and gradient backpropagation is performed.

[0054] When the deeplabV3+ network is forward-passed, it simultaneously outputs semantic segmentation maps of the cell area, table border area, and table area.

[0055] like Figure 2 As shown, from left to right are the cell area segmentation diagram, table border area segmentation diagram, and table area segmentation diagram.

[0056] S2. Fusing the cell region segmentation map and the table border region segmentation map to obtain a fused cell region segmentation map.

[0057] like Figure 3 As shown, the fusion of the cell area segmentation map and the table frame line segmentation map specifically includes the steps of:

[0058] S21, performing a pixel-level inversion operation on the table frame line region segmentation map to obtain an inverted table frame line region segmentation map;

[0059] S22, performing a pixel-level AND operation on the cell region segmentation map and the inverted table border region segmentation map to obtain a fused cell region segmentation map.

[0060] like Figure 4-7 As shown, the steps S1 and S2 are performed on an image to be identified to obtain a fused cell area segmentation map. Figure 4 is a rendering of an image to be recognized, such as Figure 5 As shown, due to the nature of the table itself, the cells in the cell area segmentation map are arranged very densely, and there are adhesions between different cells in the segmentation map. This situation will cause the adhered cells to be divided into the same cell in subsequent operations, affecting the accuracy of table structure recognition.

[0061] At the same time, to extract the cell structure from the table border segmentation map, the area needs to be completely surrounded by the table border mark. Figure 6 As shown, the table frame area segmentation map is distributed in an elongated area due to the nature of the table frame itself, and the result of the segmentation model is prone to defects, which destroys the complete encirclement, causes cell loss, and also affects the accuracy of table structure recognition.

[0062] like Figure 7 As shown, it is a schematic diagram of the fused cell area segmentation map. The cell area segmentation map is fused with the table frame line segmentation map to obtain a fused cell area segmentation map. The cell area and table frame line area segmentation maps complement each other to improve the accuracy of table structure recognition.

[0063] S3. Extract control points from the table region segmentation map to obtain a correction transformation.

[0064] In actual application scenarios, the table to be detected often appears tilted or distorted, and the original structural prior is destroyed. At this time, it is necessary to correct the table area and restore the structural prior, which is conducive to the next step of processing and improves the accuracy of table structure recognition.

[0065] The control points of the table region segmentation map are extracted to obtain the correction transformation, which specifically includes the following steps:

[0066] S31. Use a polygon fitting algorithm to extract four vertices of the circumscribed quadrilateral of the table region segmentation graph.

[0067] S32, determining the center point of the table area and the width and height of the table area according to the four vertices, and finding an intermediate control point according to a preset ratio, the intermediate control point and the four vertices forming a first control point set.

[0068] S33: Generate a second set of control points according to the width and height of the table area and a preset ratio.

[0069] S34. Generate a correction transformation using a thin plate spline interpolation algorithm according to the first control point set and the second control point set.

[0070] The correction transformation used in this embodiment is a thin plate spline interpolation algorithm, which is a two-dimensional interpolation method used for image correction or image registration. Given a pair of control point sets, the first control point set is used to describe the outline of the table area, and the second control point set is used to describe the target of the correction transformation. Since it describes a rectangle, after the transformation, the distorted table area can be corrected to a rectangle, and the structural prior is restored.

[0071] According to the first control point set and the second control point set, a corrective transformation can be obtained using a public general thin plate spline interpolation calculation formula.

[0072] S4. Use correction transformation to correct the fused cell segmentation map to obtain an aligned cell region segmentation map.

[0073] In step S3, the correction transformation is obtained by analyzing the table area segmentation map. Since the fused cell segmentation map has the same shape as the table area segmentation map, but is more refined and can distinguish the cells in the table area, the correction transformation is applied to the fused cell segmentation map to obtain an aligned cell area segmentation map corrected to a rectangle, such as Figure 8 As shown, the curved fused cell region segmentation map on the left is corrected to the aligned cell region segmentation map on the right.

[0074] S5. Perform connected domain extraction and analysis on the aligned cell region segmentation map, and obtain table structure information according to matching conditions.

[0075] like Fig. 9 As shown, the aligned cell region segmentation map is used to extract and analyze the connected domain, specifically comprising the following steps:

[0076] S51, using a contour-based marking algorithm to perform image connected domain analysis on the aligned cell region segmentation map to obtain multiple connected regions, and convert them into multiple marking boxes.

[0077] S52. Convert all connected areas into marking boxes respectively, where the marking boxes are defined by the coordinates of the upper left corner, width, and height of the circumscribed rectangle of the connected area.

[0078] The definition of the marker box is the circumscribed rectangle of the connected area, which is described by the coordinates of the upper left corner, width, and height of the circumscribed rectangle, that is, [x, y, w, h]

[0079] S53: Filter the marked boxes using the preset adaptive conditions, and remove the marked boxes that meet the conditions.

[0080] Among them, the preset adaptive conditions are:

[0081] 1) The marking box outside the table area segmentation map;

[0082] 2) The area of ​​the marked box itself is smaller than the preset threshold.

[0083] In this embodiment, the threshold is calculated based on the width, height and number of marking boxes of the table area, and the threshold calculation formula is:

[0084] threshold=W*H / N / 20

[0085] Among them, threshold is the threshold, W is the width of the table area, H is the height of the table area, and N is the number of marking boxes.

[0086] S54, performing horizontal matching and vertical matching on the marked frame, and generating a series of matching lists respectively for the horizontal matching and vertical matching. Performing horizontal matching and vertical matching on the marked frame specifically includes the following steps:

[0087] If the error between the midpoints of the y coordinates of a pair of marker frames does not exceed the adaptive preset value, the horizontal matching is successful;

[0088] The vertical matching is successful if the error between the midpoints of the x-coordinates of a pair of marker boxes does not exceed the adaptive threshold.

[0089] Furthermore, during matching, a pair of marker frames are divided into a source marker frame and a target marker frame. The adaptive threshold is determined according to the coordinates of the target marker frame. If it is a horizontal match, the adaptive threshold is 1 / 2 of the height of the target marker frame.

[0090] If it is a vertical match, the adaptive threshold is 1 / 2 of the width of the target marker box.

[0091] S55, de-duplicate the matching list, count the number of times each mark box appears in the horizontal and vertical matching lists, and obtain the number of cross-columns and cross-rows of each mark box.

[0092] During the matching process, in order to ensure a complete match, each marker box will be used as a source marker box to match all other marker boxes. In the final series of matching lists generated, there will be multiple duplicate matching lists, which need to be removed and only one of them is retained to obtain a matching list that can correctly describe the table structure information.

[0093] Among them, the marking box that crosses rows and columns will appear simultaneously in different row and column matching lists, and the number of occurrences is counted, which is the number of rows and columns that the marking box crosses.

[0094] S56. Sort the matching lists between the matching lists in each row, and sort the marking boxes in the matching lists in each row to obtain table structure information.

[0095] Between each row matching list, the matching lists are sorted and arranged from small to large according to the average of the vertical midpoints of the marker boxes contained therein; within the row matching list, the marker boxes are sorted and arranged from small to large according to the horizontal values ​​of the coordinates of their upper left corners; the table structure information is obtained by combining the number of cross-rows and cross-columns of each marker box in the order from top to bottom between row matching lists and from left to right within the row matching list.

[0096] Embodiment 2:

[0097] This embodiment provides a system for table structure recognition, which includes a semantic segmentation module, a segmentation map fusion module, a control point extraction module, a segmentation map correction module, and a segmentation map fusion module. The specific functions of each module are as follows:

[0098] A semantic segmentation module is used to use a pre-trained semantic segmentation model to perform semantic segmentation on the cell area, table frame area and table area of ​​the image to be recognized, and obtain a cell area segmentation map, a table frame area segmentation map and a table area segmentation map;

[0099] A segmentation map fusion module is used to fuse the cell region segmentation map and the table border region segmentation map to obtain a fused cell region segmentation map;

[0100] A control point extraction module is used to extract control points from the table area segmentation map to obtain a correction transformation;

[0101] The segmentation map correction module is used to correct the fused cell segmentation map using the correction transformation to obtain the aligned cell region segmentation map;

[0102] The extraction and analysis module is used to extract and analyze the connected domain of the aligned cell area segmentation map and obtain the table structure information according to the matching conditions.

[0103] Embodiment 3:

[0104] This embodiment provides a computer device, which may be a server, a computer, etc., and includes a processor, a memory, an input device, a display, and a network interface connected through a system bus. The processor is used to provide computing and control capabilities. The memory includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. When the processor executes the computer program stored in the memory, a method for identifying a table structure in the above embodiment 1 is implemented as follows:

[0105] Use the pre-trained semantic segmentation model to perform semantic segmentation on the cell area, table border area, and table area of ​​the image to be recognized, and obtain a cell area segmentation map, a table border area segmentation map, and a table area segmentation map;

[0106] The cell region segmentation map and the table border region segmentation map are merged to obtain a fused cell region segmentation map;

[0107] Extract control points from the table region segmentation map to obtain a correction transformation;

[0108] Use the correction transformation to correct the fused cell segmentation map to obtain the aligned cell region segmentation map;

[0109] Perform connected domain extraction analysis on the aligned cell region segmentation map, and obtain table structure information based on matching conditions;

[0110] Embodiment 4:

[0111] This embodiment provides a storage medium, which is a computer-readable storage medium, which stores a computer program. When the program is executed by a processor, the processor executes the computer program stored in the memory to implement the table structure recognition method of the above embodiment 1, as follows:

[0112] Use the pre-trained semantic segmentation model to perform semantic segmentation on the cell area, table border area, and table area of ​​the image to be recognized, and obtain a cell area segmentation map, a table border area segmentation map, and a table area segmentation map;

[0113] The cell region segmentation map and the table border region segmentation map are merged to obtain a merged cell region segmentation map;

[0114] Extract control points from the table region segmentation map to obtain a correction transformation;

[0115] Use the correction transformation to correct the fused cell segmentation map to obtain the aligned cell region segmentation map;

[0116] Perform connected domain extraction analysis on the aligned cell region segmentation map, and obtain table structure information based on matching conditions;

[0117] The storage medium described in this embodiment can be a disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), a USB flash drive, a mobile hard disk, or other media.

[0118] In summary, the present invention uses a pre-trained semantic segmentation model to simultaneously perform semantic segmentation on the cell area, table frame area and table area of ​​the image to be identified, fuses the segmentation maps of the cell area and the table frame area, obtains a fused cell area segmentation map, and uses the cell area and the table frame area segmentation maps to complement each other and improve the accuracy of table structure recognition; extracts control points from the table area segmentation map to obtain a correction transformation, restores the structural prior, is conducive to the next step of processing, and improves the accuracy of table structure recognition; uses the correction transformation to correct the fused cell segmentation map to obtain an aligned cell area segmentation map, performs connected domain extraction and analysis on the aligned cell area segmentation map, and obtains table structure information according to matching conditions. The present invention can solve the impact of table structure deformation in table structure recognition scenarios and improve the accuracy of table structure recognition.

[0119] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications made without departing from the spirit and principles of the present invention shall be equivalent replacement methods and shall be included in the protection scope of the present invention.

Claims

1. A table structure recognition method, characterized in that: The method comprises the steps of: S1. Use the pre-trained semantic segmentation model to perform semantic segmentation on the cell area, table border area and table area of ​​the image to be recognized, and obtain a cell area segmentation map, a table border area segmentation map, and a table area segmentation map; S2, merging the cell region segmentation map and the table border region segmentation map to obtain a fused cell region segmentation map; the step S2 specifically includes: S21, performing a pixel-level inversion operation on the table frame line region segmentation map to obtain an inverted table frame line region segmentation map; S22, performing a pixel-level AND operation on the cell region segmentation map and the inverted table border region segmentation map to obtain a fused cell region segmentation map; S3, extracting control points from the table area segmentation map to obtain a correction transformation; The step S3 specifically includes: S31, extracting four vertices of the circumscribed quadrilateral of the table region segmentation graph using a polygon fitting algorithm; S32, determining the center point of the table area and the width and height of the table area according to the four vertices, finding an intermediate control point according to a preset ratio, and the intermediate control point and the four vertices form a first control point set; S33, generating a second control point set according to the width and height of the table area and a preset ratio; S34, generating a correction transformation using a thin plate spline interpolation algorithm according to the first control point and the second control point; S4, using a correction transformation to correct the fused cell segmentation map to obtain an aligned cell region segmentation map; S5, extracting and analyzing the connected domain of the aligned cell region segmentation map, and obtaining table structure information according to the matching conditions; The step S5 specifically includes: S51, performing image connected domain analysis on the aligned cell region segmentation map using a contour-based marking algorithm to obtain a plurality of connected regions; S52, converting all connected areas into marking boxes respectively, where the marking boxes are defined by the coordinates of the upper left corner, width, and height of the circumscribed rectangle of the connected area; S53, filtering the marked boxes using the preset adaptive conditions, and removing the marked boxes that meet the preset adaptive conditions; S54, performing horizontal matching and vertical matching on the marked frame, and generating a series of matching lists respectively for the horizontal matching and the vertical matching; S55, de-duplicate the matching list, count the number of times each mark box appears in the horizontal and vertical matching lists, and obtain the number of cross-columns and cross-rows of each mark box; S56. Sort the matching lists between the matching lists in each row, and sort the marking boxes in the matching lists in each row to obtain table structure information.

2. A table structure recognition method as claimed in claim 1, characterized in that: The pre-trained semantic segmentation model used is a neural network deeplabV3+, and step S1 specifically includes: Change the multi-classification output of the last layer of the neural network deeplabV3+ to three binary classification outputs. The loss function is calculated relative to the cell area, table border area, and the annotated image of the table area, and the gradient is back-propagated. When the neural network deeplabV3+ is forward-passed, it simultaneously outputs the semantic segmentation maps of the cell area, table border area, and table area.

3. A table structure recognition method as claimed in claim 1, characterized in that: The preset adaptive condition used in step S53 is: 1) The marking box outside the table area segmentation map; 2) The area of ​​the marked box itself is smaller than the preset threshold.

4. The table structure recognition method according to claim 3, characterized in that: The calculation formula of the preset threshold is: threshold = W*H / N / 20 Among them, threshold is the threshold, W is the width of the table area, H is the height of the table area, and N is the number of marking boxes.

5. A table structure recognition system, characterized in that: The system comprises: A semantic segmentation module is used to use a pre-trained semantic segmentation model to perform semantic segmentation on the cell area, table frame area and table area of ​​the image to be recognized, and obtain a cell area segmentation map, a table frame area segmentation map and a table area segmentation map; The segmentation map fusion module is used to fuse the cell region segmentation map and the table border region segmentation map to obtain a fused cell region segmentation map; perform a pixel-level inversion operation on the table border region segmentation map to obtain an inverted table border region segmentation map; perform a pixel-level AND operation on the cell region segmentation map and the inverted table border region segmentation map to obtain a fused cell region segmentation map; The control point extraction module is used to extract control points from the table area segmentation map to obtain a correction transformation; use a polygon fitting algorithm to extract four vertices of the circumscribed quadrilateral of the table area segmentation map; determine the center point of the table area and the width and height of the table area according to the four vertices, find the intermediate control point according to a preset ratio, and the intermediate control point and the four vertices form a first control point set; generate a second control point set according to the width and height of the table area and the preset ratio; and generate a correction transformation according to the first control point and the second control point using a thin plate spline interpolation algorithm; The segmentation map correction module is used to correct the fused cell segmentation map using the correction transformation to obtain the aligned cell region segmentation map; The extraction and analysis module is used to perform connected domain extraction and analysis on the aligned cell area segmentation map, and obtain table structure information according to matching conditions; use a contour-based marking algorithm to perform image connected domain analysis on the aligned cell area segmentation map to obtain multiple connected areas; convert all connected areas into marking boxes, which are defined by the upper left corner coordinates, width and height of the circumscribed rectangle of the connected area; use preset adaptive conditions to filter the marking boxes and remove the marking boxes that meet the preset adaptive conditions; perform horizontal and vertical matching on the marking boxes, and generate a series of matching lists for the horizontal matching and vertical matching respectively; deduplicate the matching lists, count the number of times each marking box appears in the horizontal and vertical matching lists, and obtain the number of cross-columns and cross-rows of each marking box; sort the matching lists between the row matching lists, and sort the marking boxes within the row matching lists to obtain table structure information.

6. A computer device comprising a processor and a memory for storing a program executable by the processor, characterized in that: When the processor executes the program stored in the memory, a table structure recognition method as described in any one of claims 1 to 4 is implemented.

7. A storage medium storing a program, characterized in that: When the program is executed by a processor, a table structure recognition method as described in any one of claims 1 to 4 is implemented.

Citation Information

Patent Citations

  • Table structure identification method and device, electronic equipment and storage medium

    CN112528863A

  • Table analysis method and device, electronic equipment and storage medium

    CN113221743A

  • Text recognition method and apparatus, and electronic device and storage medium

    WO2021147221A1