Table text detection and identification method and system based on deep learning
Through the deep learning-based table text detection and recognition method, cell detection, tilt correction and clustering algorithms, the problem of complex layout and multi-table recognition efficiency in the existing technology is solved, and efficient and accurate table text recognition is achieved. Reduces computational complexity.
Patent Information
- Application Number
- CN202410924916.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-11
- Publication Date
- 2024-11-15
AI Technical Summary
When processing documents containing merged cells, skewed tables or complex layouts, existing tables have low recognition efficiency and difficult to accurately distinguish text areas. The calculation complexity and memory requirements are high, which limits its complexity in complex layout tables. on the application.
Using deep learning-based methods, optimized cell detection model and text recognition model are constructed, and the efficiency of table text detection and recognition is improved through cell detection, tilt correction, clustering processing, Cell coordinate aggregation algorithm and density clustering algorithm. Accuracy, reduce algorithm complexity.
On the basis of ensuring recognition accuracy, the efficiency of table text detection and recognition is improved, and it can effectively handle skewed tables, complex layouts and multiple tables, reducing the computing resource requirements, and improving the recognition efficiency and accuracy.
Smart Images

Figure CN118968533A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of table text detection and recognition, and specifically to a table text detection and recognition method and system based on deep learning. Background Art
[0002] In today's digital information age, table text detection and recognition technology is widely used in financial and accounting document processing, medical records, government and public services, commercial retail, compliance review and other fields. At present, the technology for table text detection and recognition has made some progress, and there are mainly three implementation strategies:
[0003] 1. Traditional two-stage method: Early mainstream methods usually adopted a two-stage strategy. First, text area detection was performed through traditional edge detection, text area segmentation, feature extraction and other methods. Then, the text content of the text area detected in the previous stage was recognized using a text recognition model.
[0004] 2. Traditional end-to-end model: This type of method uses traditional machine learning models for text recognition. The relevant models usually combine object detection and sequence recognition tasks, and simultaneously handle cell positioning and character recognition in a unified framework, which improves the overall efficiency and accuracy compared to the two-stage method.
[0005] 3. Methods based on deep learning: Deep learning models such as convolutional neural networks and Transformer have also been introduced into table text recognition due to their powerful modeling capabilities, and have further improved recognition performance. Deep learning models usually also adopt an end-to-end recognition strategy.
[0006] Although the above technologies have made significant progress in table text detection and recognition, there are still some significant shortcomings:
[0007] 1. The two-stage method for detecting text areas based on traditional algorithms often performs poorly for tables with merged cells, tilted tables, or complex layouts; in particular, when the text content does not completely match the table border or the table content overlaps with each other, it is difficult to accurately distinguish and identify different text areas and content.
[0008] 2. Although traditional end-to-end models and deep learning-based methods have improved accuracy, their computational complexity and memory requirements are high. Therefore, these strategies have low recognition efficiency and limit their application to complex layout tables, especially these models usually cannot handle the situation where there are multiple tables in one image in a timely manner. Summary of the invention
[0009] In response to the needs and shortcomings of current technological development, the present invention provides a table text detection and recognition method and system based on deep learning, aiming to improve the efficiency and accuracy of table text detection and recognition by improving the detection and recognition process and optimizing the algorithm.
[0010] In the first aspect, the present invention provides a table text detection and recognition method based on deep learning. The technical solution adopted to solve the above technical problems is as follows:
[0011] A method for detecting and recognizing table text based on deep learning, comprising the following steps:
[0012] S1. Based on the deep learning framework, a cell detection model and a text recognition model are constructed and optimized. The cell detection model is used to detect tables in images and extract cells, and the text recognition model is used to identify and extract text boxes and the text information they contain.
[0013] S2, using the cell detection model to detect the table in the input image, extracting all cells contained in the table, and performing tilt correction;
[0014] S3, using the text recognition model to identify the text box and its text information in the input image, and at the same time, clustering the cells to identify all independent tables on the image;
[0015] S4, applying the Cell coordinate aggregation algorithm, matching the text box with the cell, and obtaining the cell text information;
[0016] S5. Use the density clustering algorithm to merge the table lines into vertical lines and horizontal lines; sort the vertical lines and horizontal lines, and number the indexes of the horizontal lines and vertical lines starting from 0; sort the cells according to the row index, and output the row and column information of each cell.
[0017] Optionally, the step S2 specifically includes the following operations:
[0018] First, obtain the coordinate information of the four vertices of all cells and present it in a list. Each cell consists of four vertices, and the coordinate format is: [[(x i1 ,y i1 ),(x i2 ,y i2 ),(x i3 ,y i3 ),(x i4 ,y i4 )];
[0019] Subsequently, the tilt angle of the table is determined by calculating the average slope of the horizontal lines of the table and the corresponding rotation matrix is generated;
[0020] Finally, use the matrix to rotate the table and straighten it.
[0021] Further optionally, step S3 is performed to recognize the text box and its text information in the input image using a text recognition model, specifically including:
[0022] Get the coordinates of all text boxes and the text information in the text boxes, and output them in the form of a two-dimensional list, where each output element includes the coordinates of the four vertices of the text box and the text content;
[0023] Execute step S3 to perform clustering processing on the cells to identify all independent tables on the image, specifically including:
[0024] S3.1, using the spatial index data structure r-tree to obtain the adjacent cells of each cell, and forming a cell cluster through depth-first traversal;
[0025] S3.2, set a threshold a. When the number of cells in a cluster is less than the threshold a, determine that the cells in the cluster cannot form a complete table. If not, merge the cells in the cluster to form a complete table object.
[0026] S3.3. Obtain all independent tables in the input image by executing step S3.2.
[0027] Further optionally, the step S4 specifically includes the following operations:
[0028] S4.1. Given the coordinates of the cell and the text box, use the spatial index data structure r-tree to obtain the text box adjacent to the cell and calculate their intersection-over-union (IOU) ratio.
[0029] S4.2, set the IOU threshold b, and when the IOU is greater than the threshold b, the cell is considered to contain a text box;
[0030] S4.3, further determine the number of text boxes contained in the cell,
[0031] (a) If the cell contains only one text box, its text information is used as the content of the cell;
[0032] (b) If the cell contains at least two text boxes, the text contents are spliced according to their coordinate relationship.
[0033] Further optionally, the step S5 specifically includes the following operations:
[0034] S5.1. Calculate the center coordinates of each cell based on the four sides of the cell;
[0035] S5.2. Use the density clustering algorithm to perform density clustering on the left and right sides and the top and bottom sides of all cells, and identify all horizontal and vertical lines in the table;
[0036] S5.3. Use the scan line algorithm in computational geometry or the topological sorting algorithm of the graph to sort the horizontal and vertical lines and identify them with an index starting from 0;
[0037] S5.4. Sort the cells by row index and output the row and column information of each cell.
[0038] In the second aspect, the present invention provides a table text detection and recognition system based on deep learning. The technical solution adopted to solve the above technical problems is as follows:
[0039] A table text detection and recognition system based on deep learning, comprising:
[0040] Build an optimization module to build and optimize a cell detection model and a text recognition model based on a deep learning framework. The cell detection model is used to detect tables in images and extract cells, and the text recognition model is used to identify and extract text boxes and the text information they contain.
[0041] The cell detection model is used to detect the table in the input image, extract all the cells contained in the table, and perform tilt correction;
[0042] A text recognition model is used to identify text boxes and their text information in the input image;
[0043] The clustering recognition module is used to cluster cells to identify all independent tables on the image;
[0044] The matching acquisition module is used to match the text box with the cell using the Cell coordinate aggregation algorithm to obtain the cell text information;
[0045] The clustering processing module is used to first merge the table lines into vertical lines and horizontal lines using a density clustering algorithm, then sort the vertical lines and horizontal lines, number the indexes of the horizontal lines and vertical lines starting from 0, and finally sort the cells according to the row index and output the row and column information of each cell.
[0046] Optionally, the cell detection model involved specifically includes:
[0047] The information acquisition unit is used to obtain the coordinate information of the four vertices of all cells and present it in a list form. Each cell consists of four vertices, and the coordinate form is: [[(x i1 ,y i1 ),(x i2 ,y i2),(x i3 ,y i3 ),(x i4 ,y i4 )];
[0048] A calculation generating unit, used for determining the inclination angle of the table by calculating the average slope of the horizontal line of the table, and then generating a corresponding rotation matrix;
[0049] The rotation transformation unit is used to rotate the table using a rotation matrix to straighten the table.
[0050] Further optionally, the text recognition model involved recognizes the text boxes and text information in the input image, specifically including: obtaining the coordinates of all text boxes and the text information in the text boxes, and outputting them in the form of a two-dimensional list, wherein each output element includes the coordinates of four vertices of the text box and the text content;
[0051] The cluster identification modules involved specifically include:
[0052] Obtaining a traversal unit, which is used to obtain adjacent cells of each cell using the spatial index data structure r-tree, and forming a cell cluster through depth-first traversal;
[0053] A threshold setting unit 1, used for setting a threshold a;
[0054] A judgment processing unit, used to judge whether the number of cells in the cluster is less than a threshold value a. If so, it is determined that the cells of the cluster cannot form a complete table. If not, the cells in the cluster are merged to form a complete table object.
[0055] The loop acquisition module is used to loop the judgment processing unit until all independent tables in the input image are acquired.
[0056] Further optionally, the matching acquisition module involved specifically includes:
[0057] Get the calculation unit. When the coordinates of the cell and the text box are known, use the spatial index data structure r-tree to get the text box adjacent to the cell and calculate its intersection-over-union (IOU) ratio.
[0058] Threshold setting unit 2, used for setting threshold b;
[0059] A judgment unit is used to judge whether the calculated intersection-over-union ratio IOU is greater than a threshold b. If so, it is determined that the cell contains a text box;
[0060] The re-judgment processing unit is used to further judge the number of text boxes contained in the cell according to the identification result of the judgment and identification unit, and when the cell contains only one text box, its text information is used as the content of the cell, and when the cell contains at least two text boxes, the text content is spliced according to their coordinate relationship.
[0061] Further optionally, the clustering processing module involved specifically includes:
[0062] A center calculation unit, used to calculate the center coordinates of each cell based on the four sides of each cell;
[0063] The density clustering unit is used to use the density clustering algorithm to perform density clustering on the left and right sides and the top and bottom sides of all cells, and identify all horizontal and vertical lines in the table;
[0064] A calculation sorting unit is used to sort the horizontal lines and vertical lines by using a scanning line algorithm in computational geometry or a topological sorting algorithm of a graph, and identify them by an index starting from 0;
[0065] The sort output unit is used to sort cells by row index and output the row and column information of each cell.
[0066] The present invention provides a method and system for detecting and recognizing table text based on deep learning, which has the following beneficial effects compared with the prior art:
[0067] 1. The present invention aims to improve the efficiency and accuracy of table text detection and recognition by improving the detection and recognition process and optimizing the algorithm, solve the problem of poor detection of tilted tables, complex table layouts, and multiple tables in one picture, and reduce the complexity of the algorithm while ensuring the recognition accuracy, thereby reducing the requirements for computing resources and improving the recognition efficiency;
[0068] 2. Even when faced with tables with challenges such as merged cells, complex structure layouts, and text tilt, the present invention can still provide excellent table detection and recognition effects, accurately extract row and column information and text content of complex layout tables, and can simultaneously detect and recognize text in multiple independent tables. BRIEF DESCRIPTION OF THE DRAWINGS
[0069] Attached Figure 1 is a flow chart of a method according to Embodiment 1 of the present invention;
[0070] Attached Figure 2 This is a module connection block diagram of the second embodiment of the present invention. DETAILED DESCRIPTION
[0071] In order to make the technical solution, the technical problem solved and the technical effect of the present invention more clearly understood, the technical solution of the present invention is clearly and completely described below in conjunction with specific embodiments.
[0072] Embodiment 1:
[0073] Combined with Figure 1 , this embodiment proposes a table text detection and recognition method based on deep learning, which includes the following steps:
[0074] S1. Based on the deep learning framework, a cell detection model and a text recognition model are constructed and optimized. The cell detection model is used to detect tables in images and extract cells, and the text recognition model is used to identify and extract text boxes and the text information contained therein. This embodiment is specifically an OCR text recognition model.
[0075] In this step, the size of the constructed model is reduced through quantization and other methods to achieve model optimization.
[0076] S2. Use the cell detection model to detect the table in the input image, extract all cells contained in the table, and perform tilt correction, which specifically includes the following operations:
[0077] S2.1. Get the coordinate information of the four vertices of all cells and present it in a list. Each cell consists of four vertices, and the coordinate format is: [[(x i1 ,y i1 ),(x i2 ,y i2 ),(x i3 ,y i3 ),(x i4 ,y i4 )],x i1 、x i2 、x i3 、x i4 Represents the x-axis coordinates of the four vertices of the cell, i1 ,y i2 ,y i3 ,y i4 Respectively represent the y-axis coordinates of the four vertices of the cell;
[0078] S2.2, then, the tilt angle of the table is determined by calculating the average slope of the horizontal lines of the table, and a corresponding rotation matrix is generated;
[0079] S2.3. Use the matrix to rotate the table and straighten it.
[0080] S3, using a text recognition model to recognize text boxes and text information in the input image, specifically including:
[0081] Get the coordinates of all text boxes and the text information in the text boxes, and output them in the form of a two-dimensional list, where each output element includes the coordinates of the four vertices of the text box and the text content;
[0082] At the same time, the cells are clustered to identify all independent tables on the image, including:
[0083] S3.1, using the spatial index data structure r-tree to obtain the adjacent cells of each cell, and forming a cell cluster through depth-first traversal;
[0084] S3.2, set a threshold a. When the number of cells in a cluster is less than the threshold a, determine that the cells in the cluster cannot form a complete table. If not, merge the cells in the cluster to form a complete table object.
[0085] S3.3. Obtain all independent tables in the input image by executing step S3.2.
[0086] S4. Apply the Cell coordinate aggregation algorithm to match the text box with the cell and obtain the cell text information, which specifically includes the following operations:
[0087] S4.1. Given the coordinates of the cell and the text box, use the spatial index data structure r-tree to obtain the text box adjacent to the cell and calculate their intersection-over-union (IOU) ratio.
[0088] S4.2, set the IOU threshold b, and when the IOU is greater than the threshold b, the cell is considered to contain a text box;
[0089] S4.3, further determine the number of text boxes contained in the cell,
[0090] (a) If the cell contains only one text box, its text information is used as the content of the cell;
[0091] (b) If the cell contains at least two text boxes, the text contents are spliced according to their coordinate relationship.
[0092] S5. Calculate the center coordinates of each cell based on the four sides of each cell;
[0093] Use the density clustering algorithm to perform density clustering on the left and right sides and the top and bottom sides of all cells, and identify all horizontal and vertical lines in the table;
[0094] Use the scanning line algorithm in computational geometry or the topological sorting algorithm of the graph to sort the horizontal and vertical lines and identify them with an index starting from 0;
[0095] Sort cells by row index and output row and column information for each cell.
[0096] It should be added that, in step S3 of this embodiment, a specific code example of clustering cells to identify all independent tables on the picture is as follows:
[0097]
[0098]
[0099]
[0100] The specific code example of step S5 in this embodiment is as follows:
[0101]
[0102]
[0103]
[0104] Note: In the above code, tab_df_part is a dataframe structure that stores the coordinate information of all cells. The format of tab_df_part is shown in Table 1 (only the first two rows are taken from Table 1):
[0105] Table 1
[0106]
[0107] Among them, 0, 1, 2, 3, 4, 5, 6, and 7 represent the coordinate values of the four points of the cell: x1, y1, x2, y2, x3, y3, x4, and y4.
[0108] Embodiment 2:
[0109] Combined with Figure 2 , this embodiment proposes a table text detection and recognition system based on deep learning, which includes:
[0110] Construct an optimization module, which is used to construct and optimize a cell detection model and a text recognition model based on a deep learning framework, wherein the cell detection model is used to detect tables in an image and extract cells, and the text recognition model is used to identify and extract text boxes and the text information contained therein, and this embodiment is specifically an OCR text recognition model;
[0111] The cell detection model is used to detect the table in the input image, extract all the cells contained in the table, and perform tilt correction;
[0112] A text recognition model is used to identify text boxes and their text information in the input image;
[0113] The clustering recognition module is used to cluster cells to identify all independent tables on the image;
[0114] The matching acquisition module is used to match the text box with the cell using the Cell coordinate aggregation algorithm to obtain the cell text information;
[0115] The clustering processing module is used to first merge the table lines into vertical lines and horizontal lines using a density clustering algorithm, then sort the vertical lines and horizontal lines, number the indexes of the horizontal lines and vertical lines starting from 0, and finally sort the cells according to the row index and output the row and column information of each cell.
[0116] In this embodiment, the construction optimization module involved reduces the volume of the constructed model through quantization and other methods to achieve model optimization.
[0117] In this embodiment, the cell detection model involved specifically includes:
[0118] The information acquisition unit is used to obtain the coordinate information of the four vertices of all cells and present it in a list form. Each cell consists of four vertices, and the coordinate form is: [[(x i1 ,y i1 ),(x i2 ,y i2 ),(x i3 ,y i3 ),(x i4 ,y i4 )],x i1 、x i2 、x i3 、x i4 Represents the x-axis coordinates of the four vertices of the cell, i1 ,y i2 ,y i3 ,y i4 Respectively represent the y-axis coordinates of the four vertices of the cell;
[0119] A calculation generating unit, used for determining the inclination angle of the table by calculating the average slope of the horizontal lines of the table, and then generating a corresponding rotation matrix;
[0120] The rotation transformation unit is used to rotate the table using a rotation matrix to straighten the table.
[0121] In this embodiment, the text recognition model involved recognizes the text boxes and text information in the input image, specifically including: obtaining the coordinates of all text boxes and the text information in the text boxes, and outputting them in the form of a two-dimensional list, wherein each output element includes the coordinates of four vertices of the text box and the text content;
[0122] In this embodiment, the cluster identification module involved specifically includes:
[0123] Obtaining a traversal unit, which is used to obtain adjacent cells of each cell using the spatial index data structure r-tree, and forming a cell cluster through depth-first traversal;
[0124] A threshold setting unit 1, used for setting a threshold a;
[0125] A judgment processing unit, used to judge whether the number of cells in the cluster is less than a threshold value a. If so, it is determined that the cells of the cluster cannot form a complete table. If not, the cells in the cluster are merged to form a complete table object.
[0126] The loop acquisition module is used to loop the judgment processing unit until all independent tables in the input image are acquired.
[0127] In this embodiment, the matching acquisition module involved specifically includes:
[0128] Get the calculation unit. When the coordinates of the cell and the text box are known, use the spatial index data structure r-tree to get the text box adjacent to the cell and calculate its intersection-over-union (IOU) ratio.
[0129] Threshold setting unit 2, used for setting threshold b;
[0130] A judgment unit is used to judge whether the calculated intersection-over-union ratio IOU is greater than a threshold b. If so, it is determined that the cell contains a text box;
[0131] The re-judgment processing unit is used to further judge the number of text boxes contained in the cell according to the identification result of the judgment and identification unit, and when the cell contains only one text box, its text information is used as the content of the cell, and when the cell contains at least two text boxes, the text content is spliced according to their coordinate relationship.
[0132] In this embodiment, the clustering processing module involved specifically includes:
[0133] A center calculation unit, used to calculate the center coordinates of each cell based on the four sides of each cell;
[0134] The density clustering unit is used to use the density clustering algorithm to perform density clustering on the left and right sides and the top and bottom sides of all cells, and identify all horizontal and vertical lines in the table;
[0135] A calculation sorting unit is used to sort the horizontal lines and vertical lines by using a scanning line algorithm in computational geometry or a topological sorting algorithm of a graph, and identify them by an index starting from 0;
[0136] The sort output unit is used to sort cells by row index and output the row and column information of each cell.
[0137] In summary, the use of a table text detection and recognition method and system based on deep learning of the present invention can improve the efficiency and accuracy of table text detection and recognition, reduce the complexity of the algorithm while ensuring recognition accuracy, and solve the problems of poor detection of tilted tables, complex table layouts, and multiple tables in one picture.
[0138] The above specific examples are used to explain the principles and implementation methods of the present invention in detail. These examples are only used to help understand the core technical content of the present invention. Based on the above specific embodiments of the present invention, any improvements and modifications made by technicians in this technical field without departing from the principles of the present invention should fall within the scope of patent protection of the present invention.
Claims
1. A table text detection and recognition method based on deep learning, characterized in that: The steps include: S1. Based on the deep learning framework, a cell detection model and a text recognition model are constructed and optimized. The cell detection model is used to detect tables in images and extract cells, and the text recognition model is used to identify and extract text boxes and the text information they contain. S2, using the cell detection model to detect the table in the input image, extracting all cells contained in the table, and performing tilt correction; S3, using the text recognition model to identify the text box and its text information in the input image, and at the same time, clustering the cells to identify all independent tables on the image; S4, applying the Cell coordinate aggregation algorithm, matching the text box with the cell, and obtaining the cell text information; S5. Use the density clustering algorithm to merge the table lines into vertical lines and horizontal lines; sort the vertical lines and horizontal lines, and number the indexes of the horizontal lines and vertical lines starting from 0; sort the cells according to the row index, and output the row and column information of each cell.
2. The method for detecting and recognizing table text based on deep learning according to claim 1, characterized in that: The step S2 specifically includes the following operations: First, obtain the coordinate information of the four vertices of all cells and present it in a list. Each cell consists of four vertices, and the coordinate format is: [[(x i1 ,y i1 ),(x i2 ,y i2 ),(x i3 ,y i3 ),(x i4 ,y i4 )]; Subsequently, the tilt angle of the table is determined by calculating the average slope of the horizontal lines of the table and the corresponding rotation matrix is generated; Finally, use the matrix to rotate the table and straighten it.
3. The method for detecting and recognizing table text based on deep learning according to claim 2, characterized in that: Execute step S3, using the text recognition model to recognize the text box and its text information in the input image, specifically including: Get the coordinates of all text boxes and the text information in the text boxes, and output them in the form of a two-dimensional list, where each output element includes the coordinates of the four vertices of the text box and the text content; Execute step S3 to perform clustering processing on the cells to identify all independent tables on the image, specifically including: S3.1, using the spatial index data structure r-tree to obtain the adjacent cells of each cell, and forming a cell cluster through depth-first traversal; S3.2, set a threshold a. When the number of cells in a cluster is less than the threshold a, determine that the cells in the cluster cannot form a complete table. If not, merge the cells in the cluster to form a complete table object. S3.
3. Obtain all independent tables in the input image by executing step S3.
2.
4. The method for detecting and recognizing table text based on deep learning according to claim 3, characterized in that: The step S4 specifically includes the following operations: S4.
1. Given the coordinates of the cell and the text box, use the spatial index data structure r-tree to obtain the text box adjacent to the cell and calculate their intersection-over-union (IOU) ratio. S4.2, set the IOU threshold b, and when the IOU is greater than the threshold b, the cell is considered to contain a text box; S4.3, further determine the number of text boxes contained in the cell, (a) If the cell contains only one text box, its text information is used as the content of the cell; (b) If the cell contains at least two text boxes, the text contents are spliced according to their coordinate relationship.
5. The method for detecting and recognizing table text based on deep learning according to claim 4, characterized in that: The step S5 specifically includes the following operations: S5.
1. Calculate the center coordinates of each cell based on the four sides of the cell; S5.
2. Use the density clustering algorithm to perform density clustering on the left and right sides and the top and bottom sides of all cells, and identify all horizontal and vertical lines in the table; S5.
3. Use the scan line algorithm in computational geometry or the topological sorting algorithm of the graph to sort the horizontal and vertical lines and identify them with an index starting from 0; S5.
4. Sort the cells by row index and output the row and column information of each cell.
6. A table text detection and recognition system based on deep learning, characterized in that: It includes: Build an optimization module to build and optimize a cell detection model and a text recognition model based on a deep learning framework. The cell detection model is used to detect tables in images and extract cells, and the text recognition model is used to identify and extract text boxes and the text information they contain. The cell detection model is used to detect the table in the input image, extract all the cells contained in the table, and perform tilt correction; A text recognition model is used to identify text boxes and their text information in the input image; The clustering recognition module is used to cluster cells to identify all independent tables on the image; The matching acquisition module is used to match the text box with the cell using the Cell coordinate aggregation algorithm to obtain the cell text information; The clustering processing module is used to first merge the table lines into vertical lines and horizontal lines using a density clustering algorithm, then sort the vertical lines and horizontal lines, number the indexes of the horizontal lines and vertical lines starting from 0, and finally sort the cells according to the row index and output the row and column information of each cell.
7. The table text detection and recognition system based on deep learning according to claim 6, characterized in that: The cell detection model specifically includes: The information acquisition unit is used to obtain the coordinate information of the four vertices of all cells and present it in a list form. Each cell consists of four vertices, and the coordinate form is: [[(x i1 ,y i1 ),(x i2 ,y i2 ),(x i3 ,y i3 ),(x i4 ,y i4 )]; A calculation generating unit, used for determining the inclination angle of the table by calculating the average slope of the horizontal line of the table, and then generating a corresponding rotation matrix; The rotation transformation unit is used to rotate the table using a rotation matrix to straighten the table.
8. The table text detection and recognition system based on deep learning according to claim 7, characterized in that: The text recognition model recognizes text boxes and text information in the input image, specifically including: obtaining the coordinates of all text boxes and the text information in the text boxes, and outputting them in the form of a two-dimensional list, wherein each output element includes the coordinates of four vertices of the text box and the text content; The cluster identification module specifically includes: Obtaining a traversal unit, which is used to obtain adjacent cells of each cell using the spatial index data structure r-tree, and forming a cell cluster through depth-first traversal; A threshold setting unit 1, used for setting a threshold a; A judgment processing unit, used to judge whether the number of cells in the cluster is less than a threshold value a. If so, it is determined that the cells of the cluster cannot form a complete table. If not, the cells in the cluster are merged to form a complete table object. The loop acquisition module is used to loop the judgment processing unit until all independent tables in the input image are acquired.
9. The table text detection and recognition system based on deep learning according to claim 8, characterized in that: The matching acquisition module specifically includes: Get the calculation unit. When the coordinates of the cell and the text box are known, use the spatial index data structure r-tree to get the text box adjacent to the cell and calculate its intersection-over-union (IOU) ratio. Threshold setting unit 2, used for setting threshold b; A judgment unit is used to judge whether the calculated intersection-over-union ratio IOU is greater than a threshold b. If so, it is determined that the cell contains a text box; The re-judgment processing unit is used to further judge the number of text boxes contained in the cell according to the identification result of the judgment and identification unit, and when the cell contains only one text box, its text information is used as the content of the cell, and when the cell contains at least two text boxes, the text content is spliced according to their coordinate relationship.
10. The table text detection and recognition system based on deep learning according to claim 9, characterized in that: The cluster processing module specifically includes: A center calculation unit, used to calculate the center coordinates of each cell based on the four sides of each cell; The density clustering unit is used to use the density clustering algorithm to perform density clustering on the left and right sides and the top and bottom sides of all cells, and identify all horizontal and vertical lines in the table; A calculation sorting unit is used to sort the horizontal lines and vertical lines by using a scanning line algorithm in computational geometry or a topological sorting algorithm of a graph, and identify them by an index starting from 0; The sort output unit is used to sort cells by row index and output the row and column information of each cell.
Citation Information
Patent Citations
Form recognition method and device in image, electronic equipment and storage medium
CN111814722A
Table extraction and identification method and system based on deep learning
CN115661848A
E-commerce table image recognition method and device based on deep learning
CN117710999A
System and Method for Unsupervised Density Based Table Structure Identification
US20210141781A1
Form parsing method based on character location and recognition, and medium and computer device
WO2020164281A1
Cited By
Super-large format document intelligent identification method and system based on block parallelism
CN119625766A
File cross-page table restoration method and device
CN122176735A
