Method and device for reconstructing picture missing line table based on OCR, and storage medium

Abstract

The application discloses a kind of reconstruction method, device and storage medium of picture missing line table based on OCR, belong to picture processing technical field.The reconstruction method includes: S1, all texts in table are clustered according to ordinate, and all texts are divided into different lines;S2, line coordinates are constructed, and the position corresponding to the mean value of the ordinate of the first intermediate coordinate value of two adjacent lines is regarded as the first demarcation of two adjacent lines of text;S3, column coordinates are constructed, and the position corresponding to the mean value of the abscissa of the second intermediate coordinate value of two adjacent columns is regarded as the second demarcation of two adjacent columns of text;S4, the position coordinates of line recognized by OCR are compared with the coordinates of second demarcation, if matching, then display vertical line.The application reconstructs missing line table according to the position information of missing line table text and line, and improves the restoration degree of missing line table.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and specifically to a method, apparatus, and storage medium for reconstructing a table with missing lines in an image based on OCR. Background Technology

[0002] Automatically recognizing images and scanned documents and extracting useful data from them has become a significant concern. Images and scanned documents primarily consist of text, images, tables, formulas, and other content. Among these, table recognition and reconstruction, as an efficient form of data organization and presentation, is a problem that urgently needs to be solved. Table recognition refers to identifying the text content and table borders from an image; table reconstruction refers to using the recognized text and borders to reconstruct the logical structure and cell content of the table, and then inserting the table into a DOCX file for user use. Tables in images or scanned documents can be mainly divided into two categories: tables with complete borders and tables with incomplete borders (i.e., tables with missing borders). Recognizing and reconstructing tables with complete borders is relatively simple, and currently available open-source table extraction tools can achieve high accuracy. However, recognizing and reconstructing tables with missing borders presents more challenges and results in lower fidelity. Summary of the Invention

[0003] The purpose of this invention is to overcome the above-mentioned technical deficiencies and provide a method, apparatus and storage medium for reconstructing missing line tables in images based on OCR, thereby solving the technical problem of low restoration accuracy of missing line tables in the prior art.

[0004] To achieve the above technical objectives, the present invention provides a method for reconstructing a table with missing lines in an image based on OCR, comprising the following steps:

[0005] S1. Cluster all the text in the table according to the vertical axis, and divide all the text into different rows;

[0006] S2. Calculate the coordinate range of different rows of the table using the vertical coordinates of the text in different rows in step S1, construct the row coordinates, calculate the mean of the vertical coordinates of all texts based on the coordinates of the text in different rows, and then take the mean of the most frequent occurrence as the first intermediate coordinate value of the row, and take the position corresponding to the mean of the vertical coordinates of the first intermediate coordinate values ​​of two adjacent rows as the first boundary of the two adjacent rows of text.

[0007] S3. After dividing all the text in the table into rows, use the horizontal coordinate to divide the text in the row into different columns to construct column coordinates. Calculate the mean of the horizontal coordinates of all text based on the coordinates of the text in different columns. Then take the mean value with the highest frequency as the second intermediate coordinate value of the column. Take the position corresponding to the mean of the horizontal coordinates of the second intermediate coordinate values ​​of two adjacent columns as the second boundary between the two adjacent columns.

[0008] S4. Compare the position coordinates of the lines recognized by OCR with the coordinates of the first boundary constructed in step S2. If they match, display a horizontal line. Compare the position coordinates of the lines recognized by OCR with the coordinates of the second boundary constructed in step S3. If they match, display a vertical line.

[0009] Furthermore, in step S1, before dividing all the text into different lines, it also includes: determining whether the text belongs to the same line. If the difference in the vertical coordinates between the texts is less than a set first threshold, then the texts are determined to be in the same line.

[0010] Furthermore, in step S3, before dividing the text in a row into different columns using the horizontal coordinate, it also includes: determining whether the text in the row belongs to the same column. If the difference in the horizontal coordinates between the texts is less than a set second threshold, then the texts are determined to be in the same column.

[0011] Further, in step S4, the vertical coordinate value of the line position coordinates identified by OCR is compared with the vertical coordinate value of the first boundary constructed in step S2. If the difference between the two is less than the set third threshold, then a match is found and the horizontal line is displayed.

[0012] Further, in step S4, the horizontal coordinate value of the line position coordinates identified by OCR is compared with the horizontal coordinate value of the second boundary constructed in step S2. If the difference between the two is less than the set fourth threshold, then a match is found and the vertical line is displayed.

[0013] Furthermore, before step S1, the method includes: determining whether the table is a table with missing lines, determining whether all lines in the table area form a closed structure, and if the table formed by all lines forms a closed structure, then the corresponding table is a complete table; if all lines cannot form a closed structure, then the corresponding table is a table with missing lines.

[0014] Furthermore, after step S4, step S5 is included: constructing a basic table logical structure and an index of different cell contents based on the reconstructed table and line information.

[0015] Furthermore, after step S5, step S6 is also included: detecting the content in different cells, merging blank cells, formatting the text in different cells, merging the characters in the cells into corresponding paragraphs, constructing the final table structure, using the merged cells and the paragraph content in different cells to construct the table logic structure of the cells and the index information of the text paragraphs in the cells, and writing the constructed missing-line table into the DOCX file to restore the table.

[0016] Furthermore, the present invention also proposes a device for reconstructing a table with missing lines in an image, comprising:

[0017] Row partitioning units are used to cluster all the text in a table according to the vertical axis, dividing all the text into different rows;

[0018] The first dividing point is determined by using the vertical coordinates of the text in different rows to calculate the coordinate range of different rows in the table, constructing row coordinates. Based on the coordinates of the text in different rows, the mean of the vertical coordinates of all text is calculated. Then, the mean of the most frequent occurrence is taken as the first intermediate coordinate value of the row. The mean of the vertical coordinates of the first intermediate coordinate values ​​of two adjacent rows is taken as the first dividing point of the text in the two adjacent rows.

[0019] Determine the second dividing point unit, divide the text in the row into different columns using the horizontal coordinate, construct column coordinates, calculate the mean of the horizontal coordinates of all text based on the coordinates of the text in different columns, and then take the mean of the most frequent occurrence as the second intermediate coordinate value of the column. The mean of the horizontal coordinates of the second intermediate coordinate values ​​of two adjacent columns is used as the second dividing point of the text in the two adjacent columns.

[0020] The matching unit is used to compare the position coordinates of the lines recognized by the OCR with the constructed first boundary. If they match, a horizontal line is displayed; the matching unit compares the position coordinates of the lines recognized by the OCR with the constructed second boundary. If they match, a vertical line is displayed.

[0021] Furthermore, the present invention also proposes a storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the above-described OCR-based image missing line table reconstruction method.

[0022] Compared with the prior art, the beneficial effects of the present invention include: the present invention reconstructs the missing-line table based on the position information of the text and lines in the missing-line table, reduces the impact of the missing-line table on the layout, and makes the reconstructed content have a higher degree of fidelity. Attached Figure Description

[0023] Figure 1 This is a flowchart of a method for reconstructing a missing line table in an image based on OCR, according to Embodiment 1 of the present invention;

[0024] Figure 2 This is a structural block diagram of a device for reconstructing a table with missing lines in an image, according to Embodiment 1 of the present invention. Detailed Implementation

[0025] This specific implementation provides a method for reconstructing a missing line table in an image based on OCR, including the following steps:

[0026] S0. Determine if the table is a table with missing lines. Determine if all lines in the table area form a closed structure. If all lines form a closed structure, the corresponding table is a complete table. If all lines cannot form a closed structure, the corresponding table is a table with missing lines.

[0027] S1. Cluster all the text in the table according to the vertical axis, and determine whether the text belongs to the same row. If the difference between the vertical axes of the texts is less than the set first threshold, then the texts are determined to be in the same row, and all the texts are divided into different rows.

[0028] S2. Calculate the coordinate range of different rows of the table using the vertical coordinates of the text in different rows in step S1, construct the row coordinates, calculate the mean of the vertical coordinates of all texts based on the coordinates of the text in different rows, and then take the mean of the most frequent occurrence as the first intermediate coordinate value of the row. The position corresponding to the mean of the vertical coordinates of the first intermediate coordinate values ​​of two adjacent rows is taken as the first boundary of the two adjacent rows of text.

[0029] S3. After dividing all the text in the table into rows, determine whether the text in the row belongs to the same column. If the difference between the horizontal coordinates of the texts is less than the set second threshold, then the texts are determined to be in the same column. Use the horizontal coordinates to divide the text in the row into different columns and construct column coordinates. Calculate the mean of the horizontal coordinates of all texts based on the coordinates of the texts in different columns. Then take the mean with the highest frequency as the second intermediate coordinate value of the column. Take the position corresponding to the mean of the horizontal coordinates of the second intermediate coordinate values ​​of two adjacent columns as the second boundary of the two adjacent columns of text.

[0030] S4. Compare the position coordinates of the lines identified by the OCR with the coordinates of the first boundary constructed in step S2. If they match, display a horizontal line. Further, compare the vertical coordinate of the line identified by the OCR with the vertical coordinate of the first boundary constructed in step S2. If the difference between the two is less than the set third threshold, they match and display a horizontal line. Compare the position coordinates of the lines identified by the OCR with the coordinates of the second boundary constructed in step S3. If they match, display a vertical line. Further, compare the horizontal coordinate of the line identified by the OCR with the horizontal coordinate of the second boundary constructed in step S2. If the difference between the two is less than the set fourth threshold, they match and display a vertical line.

[0031] In some embodiments, step S5 is included after step S4: constructing a basic table logical structure and an index of different cell contents based on the reconstructed table and line information.

[0032] In some embodiments, step S5 is followed by step S6: detecting the content in different cells, merging blank cells, formatting the text in different cells, merging the characters in the cells into corresponding paragraphs, constructing the final table structure, using the merged cells and the paragraph content in different cells to construct the table logic structure of the cells and the index information of the text paragraphs in the cells, and writing the constructed missing-line table into a DOCX file to restore the table.

[0033] This specific embodiment also includes a reconstruction device for a table with missing lines in an image, comprising: a row division unit, used to cluster all text in the table according to the vertical axis, and divide all text into different rows;

[0034] The first dividing point is determined by using the vertical coordinates of the text in different rows to calculate the coordinate range of different rows in the table, constructing row coordinates. Based on the coordinates of the text in different rows, the mean of the vertical coordinates of all text is calculated. Then, the mean of the most frequent occurrence is taken as the first intermediate coordinate value of the row. The mean of the vertical coordinates of the first intermediate coordinate values ​​of two adjacent rows is taken as the first dividing point of the text in the two adjacent rows.

[0035] Determine the second dividing point unit, divide the text in the row into different columns using the horizontal coordinate, construct column coordinates, calculate the mean of the horizontal coordinates of all text based on the coordinates of the text in different columns, and then take the mean of the most frequent occurrence as the second intermediate coordinate value of the column. The mean of the horizontal coordinates of the second intermediate coordinate values ​​of two adjacent columns is used as the second dividing point of the text in the two adjacent columns.

[0036] The matching unit is used to compare the position coordinates of the lines recognized by the OCR with the constructed first boundary. If they match, a horizontal line is displayed; the matching unit compares the position coordinates of the lines recognized by the OCR with the constructed second boundary. If they match, a vertical line is displayed.

[0037] Furthermore, this specific embodiment also proposes a storage medium storing a computer program, which, when executed by a processor, implements the steps of the above-described OCR-based image missing line table reconstruction method.

[0038] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0039] Example 1

[0040] The table with missing lines in the image cannot be restored. Based on the position of the text and lines in the image, the table with missing lines is reconstructed and then inserted into the corresponding position in the DOCX file, so that the reconstructed table has a higher degree of restoration.

[0041] Based on the position coordinates of the table text and borders in the image, the logical structure and cell content of the table with missing lines are reconstructed to achieve a higher degree of fidelity. The specific solution is as follows:

[0042] T1: For image files, first use an OCR tool to identify the content and coordinate position information of the characters in the image, as well as the start and end coordinate information of the lines.

[0043] T2: Based on the lines and their coordinate information recognized by OCR, determine the position of the table in the image. After defining the position of the table, extract all the text and lines within the table, along with their coordinate information.

[0044] T3: To reconstruct a table with missing lines, we first need to determine if the table is indeed a table with missing lines. The main characteristic for this determination is whether all lines in the table area form a closed structure. If all lines can form a closed structure, the corresponding table is a complete table. If all lines cannot form a closed structure, the corresponding table is a table with missing lines.

[0045] T4: After determining that the current table is a table with missing lines, it is necessary to reconstruct the table using the information recognized by OCR. Since a table with missing lines does not have complete table borders, it is impossible to construct the logical structure of the table using lines.

[0046] Combination Figure 1 Based on this, this embodiment proposes a method for reconstructing missing lines in an image based on OCR, including the following steps:

[0047] S0. Determine if the table is a table with missing lines. Determine if all lines in the table area form a closed structure. If all lines form a closed structure, the corresponding table is a complete table. If all lines cannot form a closed structure, the corresponding table is a table with missing lines.

[0048] S1. Cluster all the text in the table according to the vertical axis to determine whether the text belongs to the same row. If the difference between the vertical axes of the texts is less than the set first threshold, the texts are determined to be in the same row, and all texts are divided into different rows. In this embodiment, the first threshold is 6, and the unit is pixel value.

[0049] S2. Calculate the coordinate range of different rows of the table using the y-coordinates of the text in different rows from step S1, constructing row coordinates. Based on the coordinates of the text in different rows, calculate the mean of the y-coordinates of all text, then take the mean of the most frequent occurrence as the first intermediate coordinate value of that row. Take the position corresponding to the mean of the y-coordinates of the first intermediate coordinate values ​​of two adjacent rows as the first boundary between the two adjacent rows of text.

[0050] S3. After dividing all the text in the table by row, determine whether the text in the row belongs to the same column. If the difference in the horizontal coordinates between the texts is less than the set second threshold, then the texts are determined to be in the same column. Use the horizontal coordinates to divide the text in the row into different columns and construct column coordinates. Calculate the mean of the horizontal coordinates of all texts based on the coordinates of the texts in different columns. Then take the mean value with the highest frequency as the second median coordinate value of the column. Take the position corresponding to the mean of the horizontal coordinates of the second median coordinate values ​​of two adjacent columns as the second boundary between the two adjacent columns of text. In this embodiment, the second threshold is 2, and the unit is pixel value.

[0051] S4. Compare the position coordinates of the lines identified by the OCR with the coordinates of the first boundary constructed in step S2. If they match, display a horizontal line. Further, compare the vertical coordinate of the line identified by the OCR with the vertical coordinate of the first boundary constructed in step S2. If the difference between the two is less than the set third threshold, they match, and a horizontal line is displayed. Compare the position coordinates of the lines identified by the OCR with the coordinates of the second boundary constructed in step S3. If they match, display a vertical line. Further, compare the horizontal coordinate of the line identified by the OCR with the horizontal coordinate of the second boundary constructed in step S2. If the difference between the two is less than the set fourth threshold, they match, and a vertical line is displayed. In this embodiment, both the third and fourth thresholds are 2, and the unit is pixel value.

[0052] S5. Based on the reconstructed table and line information, construct the basic table logical structure and indexes of different cell contents.

[0053] S6. Detect the content in different cells, merge blank cells, format the text in different cells, merge the characters in the cells into corresponding paragraphs, construct the final table structure, use the merged cells and the paragraph content in different cells to construct the table logic structure of the cells and the index information of the text paragraphs in the cells, and write the constructed missing-line table into a DOCX file to restore the table.

[0054] Combination Figure 2 This embodiment also includes a device for reconstructing a table with missing lines in an image, comprising:

[0055] Row partitioning units are used to cluster all the text in a table according to the vertical axis, dividing all the text into different rows;

[0056] The first dividing point is determined by using the vertical coordinates of the text in different rows to calculate the coordinate range of different rows in the table and construct the row coordinates. Based on the coordinates of the text in different rows, the mean of the vertical coordinates of all text is calculated. Then, the mean with the highest frequency is taken as the first intermediate coordinate value of the row. The position corresponding to the mean of the vertical coordinates of the first intermediate coordinate values ​​of two adjacent rows is taken as the first dividing point of the text in the two adjacent rows.

[0057] Determine the second dividing point unit, divide the text in the row into different columns using the horizontal coordinate, construct column coordinates, calculate the mean of the horizontal coordinates of all text based on the coordinates of the text in different columns, and then take the mean of the most frequent occurrence as the second intermediate coordinate value of the column. The position corresponding to the mean of the horizontal coordinates of the second intermediate coordinate values ​​of two adjacent columns is taken as the second dividing point of the two adjacent columns of text.

[0058] The matching unit is used to compare the position coordinates of the lines recognized by the OCR with the coordinates of the first boundary constructed. If they match, a horizontal line is displayed; the matching unit compares the position coordinates of the lines recognized by the OCR with the coordinates of the second boundary constructed. If they match, a vertical line is displayed.

[0059] Furthermore, this embodiment also proposes a storage medium storing a computer program, which, when executed by a processor, implements the steps of the OCR-based image missing line table reconstruction method described above.

[0060] This invention provides a method for parsing and reconstructing missing-line tables in image documents. It can automatically identify table text and line information in images, thereby constructing the logical structure of the missing-line table and the content in the cells, ultimately improving the accuracy of the missing-line table reconstruction.

[0061] The specific embodiments of the present invention described above do not constitute a limitation on the scope of protection of the present invention. Any other corresponding changes and modifications made in accordance with the technical concept of the present invention should be included within the scope of protection of the claims of the present invention.

Claims

1. A method for reconstructing a table with missing lines in an image based on OCR, characterized in that, Includes the following steps: S0. Determine if the table is a table with missing lines. Determine if all lines in the table area form a closed structure. If all lines form a closed structure, the corresponding table is a complete table. If all lines cannot form a closed structure, the corresponding table is a table with missing lines. S1. Cluster all the text in the table according to the vertical axis, and divide all the text into different rows; S2. Calculate the coordinate range of different rows of the table using the vertical coordinates of the text in different rows in step S1, construct the row coordinates, calculate the mean of the vertical coordinates of all texts based on the coordinates of the text in different rows, and then take the mean of the most frequent occurrence as the first intermediate coordinate value of the row, and take the position corresponding to the mean of the vertical coordinates of the first intermediate coordinate values ​​of two adjacent rows as the first boundary of the two adjacent rows of text. S3. After dividing all the text in the table into rows, use the horizontal coordinate to divide the text in the row into different columns to construct column coordinates. Calculate the mean of the horizontal coordinates of all text based on the coordinates of the text in different columns. Then take the mean value with the highest frequency as the second intermediate coordinate value of the column. Take the position corresponding to the mean of the horizontal coordinates of the second intermediate coordinate values ​​of two adjacent columns as the second boundary between the two adjacent columns. S4. Compare the position coordinates of the lines recognized by OCR with the coordinates of the first boundary constructed in step S2. If they match, display a horizontal line; compare the position coordinates of the lines recognized by OCR with the coordinates of the second boundary constructed in step S3. If they match, display a vertical line. S5. Based on the reconstructed table and line information, construct the basic table logical structure and indexes of different cell contents; S6. Detect the content in different cells, merge blank cells, format the text in different cells, merge the characters in the cells into corresponding paragraphs, construct the final table structure, use the merged cells and the paragraph content in different cells to construct the table logic structure of the cells and the index information of the text paragraphs in the cells, and write the constructed missing-line table into a DOCX file to restore the table.

2. The method for reconstructing a table with missing lines in an image based on OCR according to claim 1, characterized in that, In step S1, before dividing all text into different lines, it also includes: determining whether the text belongs to the same line. If the difference between the vertical coordinates of the texts is less than a set first threshold, then the texts are determined to be in the same line.

3. The method for reconstructing a table with missing lines in an image based on OCR according to claim 1, characterized in that, In step S3, before dividing the text in a row into different columns using the horizontal coordinate, it also includes: determining whether the text in the row belongs to the same column. If the difference in the horizontal coordinates between the texts is less than a set second threshold, then the texts are determined to be in the same column.

4. The method for reconstructing a table with missing lines in an image based on OCR according to claim 1, characterized in that, In step S4, the vertical coordinate value of the line position coordinates identified by OCR is compared with the vertical coordinate value of the first boundary constructed in step S2. If the difference between the two is less than the set third threshold, then a match is found and a horizontal line is displayed.

5. The method for reconstructing a table with missing lines in an image based on OCR according to claim 1, characterized in that, In step S4, the horizontal coordinate value of the line position coordinates identified by OCR is compared with the horizontal coordinate value of the second boundary constructed in step S2. If the difference between the two is less than the set fourth threshold, then a match is found and the vertical line is displayed.

6. A device for reconstructing a table with missing lines in an image, characterized in that, include: Row partitioning units are used to cluster all the text in a table according to the vertical axis, dividing all the text into different rows; The first dividing point is determined by using the vertical coordinates of the text in different rows to calculate the coordinate range of different rows in the table and construct the row coordinates. Based on the coordinates of the text in different rows, the mean of the vertical coordinates of all text is calculated. Then, the mean of the most frequent occurrence is taken as the first intermediate coordinate value of the row. The position corresponding to the mean of the vertical coordinates of the first intermediate coordinate values ​​of two adjacent rows is taken as the first dividing point of the text in the two adjacent rows. Determine the second dividing point unit, divide the text in the row into different columns using the horizontal coordinate, construct column coordinates, calculate the mean of the horizontal coordinates of all text based on the coordinates of the text in different columns, and then take the mean of the most frequent occurrence as the second intermediate coordinate value of the column. The position corresponding to the mean of the horizontal coordinates of the second intermediate coordinate values ​​of two adjacent columns is taken as the second dividing point of the two adjacent columns of text. The matching unit is used to compare the position coordinates of the lines recognized by OCR with the constructed first boundary. If they match, the horizontal line is displayed. The position coordinates of the lines identified by OCR are compared with the constructed second boundary. If they match, the vertical line is displayed.

7. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the OCR-based image missing line table reconstruction method as described in any one of claims 1 to 5.

Images