A table detection method and system based on deep learning

Abstract

The present invention provides a table detection method and system based on deep learning in the field of table detection technology. The method includes the following steps: step S10, obtaining a table image, detecting table parameters including a bounding box, key points, and direction on the table image, and obtaining a table area image; step S20, correcting the table area image; step S30, detecting the table in the corrected table area image to obtain a detection result; step S40, performing cell filtering, table filtering, and row and column filtering based on the detection result to obtain table data; step S50, identifying text in the table image, generating a table file based on the text and table data, to complete table detection of the table image. The advantage of the present invention is that it greatly improves the accuracy and quality of table detection.

Description

Technical Field

[0001] The present invention relates to the field of table detection technology, and in particular to a table detection method and system based on deep learning. Background Art

[0002] With the widespread use of documents containing forms and tables, such as invoices and lists, converting paper documents into electronic data and preserving and managing them has become a necessity for many companies. However, traditional manual data entry methods are inefficient, error-prone, and time-consuming, and are no longer able to meet current needs. Automating the structured display of forms and images can significantly reduce costs, improve efficiency, and enhance user experience.

[0003] With the popularization of scanning technology, images can be converted into editable documents through mobile phones. However, current scanning technology is often not so sensitive when processing text with table types, and often breaks up complete tables into scattered individuals that are difficult to use. Subsequent manual adjustments are required, which is very cumbersome and provides a poor user experience.

[0004] Therefore, how to provide a table detection method and system based on deep learning to improve the accuracy and quality of table detection has become a technical problem that needs to be solved urgently. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to provide a table detection method and system based on deep learning to improve the accuracy and quality of table detection.

[0006] In a first aspect, the present invention provides a table detection method based on deep learning, comprising the following steps:

[0007] Step S10: Acquire a table image, and detect table parameters including a bounding box, key points, and direction on the table image to obtain a table area image;

[0008] Step S20: Correcting the table area image;

[0009] Step S30: Detect the table in the corrected table area image to obtain a detection result;

[0010] Step S40: performing cell filtering, table filtering, and row and column filtering based on the detection results to obtain table data;

[0011] Step S50: Identify the text in the table image, and generate a table file based on the text and table data to complete table detection of the table image.

[0012] Furthermore, the step S10 specifically includes:

[0013] Step S11: obtaining a table image to be detected, detecting the borders in the table image using an object detection algorithm, and selecting the border with the highest confidence as the bounding box of the table;

[0014] Step S12: cropping the table image based on the bounding box to obtain a table area image, performing semantic segmentation on the table area image using a semantic segmentation algorithm, identifying key points in the segmented table area image using an OpenCV edge contour processing algorithm, and drawing a table boundary based on each key point;

[0015] Step S13: Detecting the direction of the table in the table area image based on the table boundary and the LCNet algorithm.

[0016] Furthermore, the step S20 is specifically as follows:

[0017] The table area image is perspective transformed by OpenCV, and then the table area image is corrected.

[0018] Furthermore, the step S30 is specifically as follows:

[0019] Detecting the table rows, table columns, and cell types of the table in the corrected table area image using a target detection algorithm to obtain a detection result;

[0020] The cell type is a single cell, a cell merged across rows, or a cell merged across columns; the detection result carries the width and height of each cell.

[0021] Furthermore, in step S40, the cell filtering is specifically performed based on the prior information that the height of the cell merged across rows is greater than the height of the row where it is located, and the width of the cell merged across columns is greater than the width of the column where it is located;

[0022] The table filtering specifically includes: performing table filtering based on prior information that a table image has only one table;

[0023] The row and column filtering specifically includes: performing row and column filtering based on prior information of the correlation between the width and height of the rows and columns of the table.

[0024] In a second aspect, the present invention provides a table detection system based on deep learning, comprising the following modules:

[0025] A table parameter detection module is used to obtain a table image, detect table parameters including a bounding box, key points, and direction on the table image, and obtain a table area image;

[0026] An image correction module, used for correcting the image in the table area;

[0027] A table detection module is used to detect the table in the corrected table area image and obtain a detection result;

[0028] A filtering module, configured to perform cell filtering, table filtering, and row and column filtering operations based on the detection results, thereby obtaining table data;

[0029] The table file generation module is used to identify the text in the table image and generate a table file based on the text and table data to complete the table detection of the table image.

[0030] Furthermore, the table parameter detection module specifically includes:

[0031] A bounding box detection unit is used to obtain a table image to be detected, detect the border in the table image using an object detection algorithm, and select the border with the highest confidence as the bounding box of the table;

[0032] a key point detection unit, configured to crop the table image based on the bounding box to obtain a table area image, perform semantic segmentation on the table area image using a semantic segmentation algorithm, identify key points within the segmented table area image using an OpenCV edge contour processing algorithm, and draw a table boundary based on each of the key points;

[0033] A direction detection unit is used to detect the direction of the table in the table area picture based on the table boundary and the LCNet algorithm.

[0034] Furthermore, the image correction module is specifically used to:

[0035] The table area image is perspective transformed by OpenCV, and then the table area image is corrected.

[0036] Furthermore, the table detection module is specifically used to:

[0037] Detecting the table rows, table columns, and cell types of the table in the corrected table area image using a target detection algorithm to obtain a detection result;

[0038] The cell type is a single cell, a cell merged across rows, or a cell merged across columns; the detection result carries the width and height of each cell.

[0039] Furthermore, in the filtering module, the cell filtering is specifically performed based on the prior information that the height of the cell merged across rows is greater than the height of the row where it is located, and the width of the cell merged across columns is greater than the width of the column where it is located;

[0040] The table filtering specifically includes: performing table filtering based on prior information that a table image has only one table;

[0041] The row and column filtering specifically includes: performing row and column filtering based on prior information of the correlation between the width and height of the rows and columns of the table.

[0042] The advantages of the present invention are:

[0043] By detecting the table parameters including bounding box, key points and direction of the acquired table image, a table area image is obtained, and then the table area image is corrected by perspective transformation using OpenCV. Then, the table rows, table columns and cell types of the table in the corrected table area image are detected using the target detection algorithm to obtain the detection results containing the width and height of each cell. Based on the detection results, cell filtering, table filtering and row and column filtering operations are performed to obtain the table data. Finally, the text in the table image is recognized, and a table file is generated based on the text and table data, that is, the table image is converted into an editable table file. Since bounding box detection utilizes the target detection algorithm, key point detection utilizes the semantic segmentation algorithm and the OpenCV edge contour processing algorithm, and direction detection utilizes the LCNet algorithm, it can accurately identify the table area and table direction from the table image, and use OpenCV to perform perspective transformation to correct the table area image. It is suitable for images taken at different angles, ensuring the accuracy of table detection, and filtering out low-quality cells, tables, rows and columns based on relevant prior information. It also generates user-editable table files based on text and table data, eliminating the need for users to make secondary adjustments, ultimately greatly improving the accuracy and quality of table detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0045] Figure 1 This is a flow chart of a table detection method based on deep learning of the present invention.

[0046] Figure 2 It is a structural diagram of a table detection system based on deep learning in the present invention.

[0047] Figure 3 It is a schematic diagram of obtaining a picture of a table area according to the present invention.

[0048] Figure 4 It is a schematic diagram of a table picture of the present invention.

[0049] Figure 5 Schematic diagram of bounding box detection of the present invention.

[0050] Figure 6 Schematic diagram of the table area image after correction according to the present invention.

[0051] Figure 7 It is a schematic diagram of the table detection of the present invention. DETAILED DESCRIPTION

[0052] The technical solution in the embodiments of the present application has the following overall idea: using the target detection algorithm to perform bounding box detection, using the semantic segmentation algorithm and the OpenCV edge contour processing algorithm to perform key point detection, and using the LCNet algorithm to perform direction detection, so as to accurately identify the table area and table direction from the table image, and use OpenCV to perform perspective transformation to correct the table area image. It is suitable for pictures taken at different angles, filters out low-quality cells, tables, rows and columns based on relevant prior information, and directly generates a user-editable table file based on text and table data, thereby improving the accuracy and quality of table detection.

[0053] Please refer to Figures 1 to 7 As shown, a preferred embodiment of a table detection method based on deep learning of the present invention includes the following steps:

[0054] Step S10: Acquire a table image, and detect table parameters including a bounding box, key points, and direction on the table image to obtain a table area image;

[0055] Step S20: Correcting the table area image;

[0056] Step S30: Detect the table in the corrected table area image to obtain a detection result;

[0057] Step S40: performing cell filtering, table filtering, and row and column filtering based on the detection results to obtain table data, that is, filtering low-quality cells, tables, rows and columns to improve detection accuracy without significantly reducing the recall rate;

[0058] Step S50: Identify the text in the table image, and generate a table file based on the text and table data to complete table detection of the table image.

[0059] The present invention provides an overall process for refined table detection, which is suitable for table detection in pictures taken at arbitrary angles and for detection of different types of tables. Thanks to the multi-model structure, it has high accuracy and robustness.

[0060] The step S10 specifically includes:

[0061] Step S11: obtaining a table image to be detected, detecting the border in the table image using a target detection algorithm, and selecting the border with the highest confidence as the bounding box of the table; the target detection algorithm is preferably a YOLO algorithm;

[0062] Step S12: cropping the table image based on the bounding box to obtain a table area image, performing semantic segmentation on the table area image using a semantic segmentation algorithm, identifying key points in the segmented table area image using an OpenCV edge contour processing algorithm, and drawing a table boundary based on each key point; the semantic segmentation algorithm is preferably a DBnet algorithm;

[0063] In order to eliminate the influence of noise on the segmentation effect, the bounding box is corrected by the minimum envelope quadrilateral algorithm before semantic segmentation;

[0064] Step S13: Detect the direction of the table in the table area image based on the table boundary and the LCNet algorithm, and map the geometric contour points to semantic contour points according to the predefined correspondence between geometric contour points (key points) and semantic contour points.

[0065] The step S20 is specifically as follows:

[0066] The table area image is perspective transformed by OpenCV, and then the table area image is corrected.

[0067] The step S30 is specifically as follows:

[0068] Detecting the table rows, table columns, and cell types of the table in the corrected table area image using a target detection algorithm to obtain a detection result;

[0069] The cell type is a single cell, a cell merged across rows, or a cell merged across columns; the detection result carries the width and height of each cell.

[0070] In step S40, the cell filtering is specifically performed based on the prior information that the height of a cell merged across rows is greater than the height of the row in which it is located, and the width of a cell merged across columns is greater than the width of the column in which it is located, so as to solve the problem that it is difficult to distinguish between merged cells and single cells.

[0071] The table filtering specifically includes: performing table filtering based on the prior information that a table image contains only one table, so as to improve the recall rate of the table;

[0072] The row and column filtering is specifically: performing row and column filtering based on prior information of the correlation between the width and height of the rows and columns of the table, so as to solve the false detection problem caused by short edge fluctuations.

[0073] A preferred embodiment of a table detection system based on deep learning of the present invention includes the following modules:

[0074] A table parameter detection module is used to obtain a table image, detect table parameters including a bounding box, key points, and direction on the table image, and obtain a table area image;

[0075] An image correction module, used for correcting the image in the table area;

[0076] A table detection module is used to detect the table in the corrected table area image and obtain a detection result;

[0077] A filtering module is used to perform cell filtering, table filtering, and row and column filtering operations based on the detection results to obtain tabular data, that is, to filter low-quality cells, tables, rows and columns to improve detection accuracy without significantly reducing the recall rate;

[0078] The table file generation module is used to identify the text in the table image and generate a table file based on the text and table data to complete the table detection of the table image.

[0079] The present invention provides an overall process for refined table detection, which is suitable for table detection in pictures taken at arbitrary angles and for detection of different types of tables. Thanks to the multi-model structure, it has high accuracy and robustness.

[0080] The table parameter detection module specifically includes:

[0081] a bounding box detection unit, configured to obtain a table image to be detected, detect a border in the table image using a target detection algorithm, and select the border with the highest confidence as the bounding box of the table; the target detection algorithm is preferably a YOLO algorithm;

[0082] a key point detection unit, configured to crop the table image based on the bounding box to obtain a table area image, perform semantic segmentation on the table area image using a semantic segmentation algorithm, identify key points within the segmented table area image using an OpenCV edge contour processing algorithm, and draw a table boundary based on each key point; the semantic segmentation algorithm is preferably a DBnet algorithm;

[0083] In order to eliminate the influence of noise on the segmentation effect, the bounding box is corrected by the minimum envelope quadrilateral algorithm before semantic segmentation;

[0084] A direction detection unit is used to detect the direction of the table in the table area picture based on the table boundary and the LCNet algorithm, and map the geometric contour points to semantic contour points according to the predefined correspondence between the geometric contour points (key points) and the semantic contour points.

[0085] The image correction module is specifically used for:

[0086] The table area image is perspective transformed by OpenCV, and then the table area image is corrected.

[0087] The table detection module is specifically used for:

[0088] Detecting the table rows, table columns, and cell types of the table in the corrected table area image using a target detection algorithm to obtain a detection result;

[0089] The cell type is a single cell, a cell merged across rows, or a cell merged across columns; the detection result carries the width and height of each cell.

[0090] In the filtering module, the cell filtering is specifically performed based on the prior information that the height of a cell merged across rows is greater than the height of the row in which it is located, and the width of a cell merged across columns is greater than the width of the column in which it is located, so as to solve the problem that it is difficult to distinguish between merged cells and single cells;

[0091] The table filtering specifically includes: performing table filtering based on the prior information that a table image contains only one table, so as to improve the recall rate of the table;

[0092] The row and column filtering is specifically: performing row and column filtering based on prior information of the correlation between the width and height of the rows and columns of the table, so as to solve the false detection problem caused by short edge fluctuations.

[0093] In summary, the advantages of the present invention are:

[0094] By detecting the table parameters including bounding box, key points and direction of the acquired table image, a table area image is obtained, and then the table area image is corrected by perspective transformation using OpenCV. Then, the table rows, table columns and cell types of the table in the corrected table area image are detected using the target detection algorithm to obtain the detection results containing the width and height of each cell. Based on the detection results, cell filtering, table filtering and row and column filtering operations are performed to obtain the table data. Finally, the text in the table image is recognized, and a table file is generated based on the text and table data, that is, the table image is converted into an editable table file. Since bounding box detection utilizes the target detection algorithm, key point detection utilizes the semantic segmentation algorithm and the OpenCV edge contour processing algorithm, and direction detection utilizes the LCNet algorithm, it can accurately identify the table area and table direction from the table image, and use OpenCV to perform perspective transformation to correct the table area image. It is suitable for images taken at different angles, ensuring the accuracy of table detection, and filtering out low-quality cells, tables, rows and columns based on relevant prior information. It also generates user-editable table files based on text and table data, eliminating the need for users to make secondary adjustments, ultimately greatly improving the accuracy and quality of table detection.

[0095] Although the specific embodiments of the present invention are described above, those skilled in the art should understand that the specific embodiments described are merely illustrative and are not intended to limit the scope of the present invention. Equivalent modifications and changes made by those skilled in the art in accordance with the spirit of the present invention should be included within the scope of protection of the claims of the present invention.

Claims

1. A table detection method based on deep learning, characterized by: The steps include: Step S10: Acquire a table image, and detect table parameters including a bounding box, key points, and direction on the table image to obtain a table area image; Step S20: Correcting the table area image; Step S30: Detect the table in the corrected table area image to obtain a detection result; Step S40: performing cell filtering, table filtering, and row and column filtering based on the detection results to obtain table data; Step S50: identifying the text in the table image, and generating a table file based on the text and table data to complete table detection of the table image; The step S10 specifically includes: Step S11: obtaining a table image to be detected, detecting the borders in the table image using an object detection algorithm, and selecting the border with the highest confidence as the bounding box of the table; Step S12: cropping the table image based on the bounding box to obtain a table area image, performing semantic segmentation on the table area image using a semantic segmentation algorithm, identifying key points in the segmented table area image using an OpenCV edge contour processing algorithm, and drawing a table boundary based on each key point; Step S13: Detecting the direction of the table in the table area image based on the table boundary and the LCNet algorithm.

2. The table detection method based on deep learning according to claim 1, characterized in that: The step S20 is specifically as follows: The table area image is perspective transformed by OpenCV, and then the table area image is corrected.

3. The table detection method based on deep learning according to claim 1, characterized in that: The step S30 is specifically as follows: Detecting the table rows, table columns, and cell types of the table in the corrected table area image using a target detection algorithm to obtain a detection result; The cell type is a single cell, a merged cell across rows, or a merged cell across columns; The detection result carries the width and height of each cell.

4. The table detection method based on deep learning according to claim 1, characterized in that: In the step S40, the cell filtering is specifically performed based on the prior information that the height of the cell merged across rows is greater than the height of the row where it is located, and the width of the cell merged across columns is greater than the width of the column where it is located; The table filtering specifically includes: performing table filtering based on prior information that a table image has only one table; The row and column filtering specifically includes: performing row and column filtering based on prior information of the correlation between the width and height of the rows and columns of the table.

5. A table detection system based on deep learning, characterized by: Includes the following modules: A table parameter detection module is used to obtain a table image, detect table parameters including a bounding box, key points, and direction on the table image, and obtain a table area image; An image correction module, used for correcting the image in the table area; A table detection module is used to detect the table in the corrected table area image and obtain a detection result; A filtering module, configured to perform cell filtering, table filtering, and row and column filtering operations based on the detection results, thereby obtaining table data; A table file generation module is used to identify the text in the table image and generate a table file based on the text and table data to complete the table detection of the table image; The table parameter detection module specifically includes: A bounding box detection unit is used to obtain a table image to be detected, detect the border in the table image using an object detection algorithm, and select the border with the highest confidence as the bounding box of the table; a key point detection unit, configured to crop the table image based on the bounding box to obtain a table area image, perform semantic segmentation on the table area image using a semantic segmentation algorithm, identify key points within the segmented table area image using an OpenCV edge contour processing algorithm, and draw a table boundary based on each of the key points; A direction detection unit is used to detect the direction of the table in the table area picture based on the table boundary and the LCNet algorithm.

6. The table detection system based on deep learning according to claim 5, characterized in that: The image correction module is specifically used for: The table area image is perspective transformed by OpenCV, and then the table area image is corrected.

7. The table detection system based on deep learning according to claim 5, characterized in that: The table detection module is specifically used for: Detecting the table rows, table columns, and cell types of the table in the corrected table area image using a target detection algorithm to obtain a detection result; The cell type is a single cell, a merged cell across rows, or a merged cell across columns; The detection result carries the width and height of each cell.

8. The table detection system based on deep learning according to claim 5, characterized in that: In the filtering module, the cell filtering is specifically performed based on the prior information that the height of the cell merged across rows is greater than the height of the row where it is located, and the width of the cell merged across columns is greater than the width of the column where it is located; The table filtering specifically includes: performing table filtering based on prior information that a table image has only one table; The row and column filtering specifically includes: performing row and column filtering based on prior information of the correlation between the width and height of the rows and columns of the table.

Images