Table text recognition method and system based on OCR

Abstract

The application provides an OCR-based table text recognition method and system, comprising: parsing a PDF file, converting the PDF file into image data; performing layout analysis on the image data to obtain a layout analysis result; judging whether there is a table element; if yes, saving coordinate information; if no, performing recognition according to pure text; obtaining the number and area of table detection boxes based on a deep learning network; finding the contour in the image; obtaining the number and area of table detection boxes; judging whether the number of detection boxes obtained by the former is equal to the number of detection boxes obtained by the latter; if yes, using the number of detection boxes obtained by the latter as an input item; if no, using the result with more detection boxes as an input item; returning the text content of each row to obtain result data. The application performs table detection on a page, detects the table area existing in the page, and prepares for cell detection and information extraction; and the method can improve the accuracy of table detection.

Description

Technical Field

[0001] This invention belongs to the field of optical character recognition technology, specifically, it relates to a table text recognition method and system based on OCR. Background Technology

[0002] OCR, or Optical Character Recognition, is a crucial area of ​​computer vision. Traditionally, OCR was generally defined as being geared towards scanning documents. Nowadays, OCR typically refers to scene-based text recognition, primarily targeting natural scenes. OCR technology has a wide range of applications, a typical example being the widespread use of structured text recognition in everyday life, targeting specific subcategories such as license plate recognition, bank card information recognition, ID card information recognition, and train ticket information recognition. These subcategories share the characteristic of fixed formats, making them ideal for automating with OCR technology, significantly reducing labor costs and improving efficiency.

[0003] Table text recognition aims to divide a table area into several cells, extract and recognize the content in each cell, and return the text information consistent with the table's location. This falls under the category of structured document recognition. With the increasing maturity of OCR technology and the strong demand for structured information extraction, various technologies related to intelligent document analysis, such as layout analysis, table recognition, and key information extraction, have received increasing attention and research. Layout analysis mainly involves classifying document images into categories, generally including plain text, titles, tables, and images. Existing methods typically treat different layouts in a document as different targets for detection or segmentation. Table recognition aims to identify and extract table information from a document. The types and styles of tables in text images are complex and diverse, including different row and column combinations and different text types; these challenges make table recognition a persistent research difficulty in the field of document understanding. Summary of the Invention

[0004] In view of the shortcomings of the existing technology, the purpose of this invention is to provide a table text recognition method and system based on OCR.

[0005] According to the present invention, a table text recognition method based on OCR includes:

[0006] Step S1: Parse the PDF file and convert it into image data;

[0007] Step S2: Perform layout analysis on the image data to obtain the layout analysis results;

[0008] Step S3: Based on the layout analysis results, determine whether table elements exist; if yes, save the coordinate information of the table detection area; if no, recognize the text as plain text; based on the deep learning network, determine the number and area of ​​table detection boxes.

[0009] Step S4: Based on the image data, find the contours in the image through morphological processing; and based on the contours, determine the number and region of the table detection boxes;

[0010] Step S5: Determine whether the number of detection boxes obtained in step S3 is equal to the number of detection boxes obtained in step S4. If the result is yes, then use the number of detection boxes obtained in step S4 as the input item; if the result is no, then use the result with more detection boxes as the input item.

[0011] Step S6: Based on the input items, obtain cell information, and filter the cell information to obtain cell coordinate information;

[0012] Step S7: Based on the cell coordinate information, obtain and return the text content of each line through the coordinate information of each text box and the recognized content in the text box, and obtain the result data.

[0013] Preferably, in step S2: according to the actual page layout of the layout analysis results, the layout elements in the layout analysis results are output from top to bottom;

[0014] The layout analysis results include layout elements and their coordinate regions in the image;

[0015] The layout elements include: text, titles, tables, and images.

[0016] Preferably, in step S3, the number and region of table detection boxes are obtained based on deep learning.

[0017] In step S4:

[0018] Step S4.1: After binarizing the image data, perform morphological processing to obtain the horizontal and vertical lines of the table in the image, and calculate the intersection of the horizontal and vertical lines;

[0019] Step S4.2: Based on the intersection of the horizontal and vertical lines, obtain the contours in the image using OpenCV, and derive the number and region of the table detection boxes based on the bounding rectangle table detection method.

[0020] In step S5, if the number of detection boxes obtained by the deep learning method is greater than the number of detection boxes obtained by the bounding rectangle table detection method, and the number of detection boxes obtained by the bounding rectangle table detection method is greater than or equal to one, it is determined whether the position of the box obtained by the bounding rectangle table detection method is within the box obtained by the deep learning method.

[0021] If the result is yes, then the position information of the bounding box detected by the deep learning method will be replaced with the bounding box detected by the bounding rectangle table detection method.

[0022] If the result is negative, the position information of the bounding box detected by deep learning method is used.

[0023] Preferably, in step S6:

[0024] Step S6.1: Based on the table detection results obtained in step S5, perform binarization and morphological processing on each table detection result, i.e. each table, to calculate the binary images of the horizontal and vertical lines of the table, and then calculate the binary images of the table intersection points based on the binary images of the horizontal and vertical lines.

[0025] Step S6.2: Based on the binary graph of the intersection points in the table, obtain the coordinates of each intersection point. Group the intersection point coordinates and the intersection points of each horizontal and vertical line to obtain the grouped intersection point coordinates.

[0026] Step S6.3: Based on the coordinates of the intersection points after grouping, determine the number of horizontal and vertical lines in the table, and iterate through all cells to represent them, thus obtaining the cell information;

[0027] The cell is represented as [starting row, ending row, starting column, ending column];

[0028] Step S6.4: Filter all the displayed cells to obtain the final actual cell coordinate information;

[0029] In step S6.4:

[0030] Step S6.4.1: Obtain the coordinates of each cell using the cell information, and determine whether the four intersection points of the cells satisfy the rectangle condition; if the result is yes, keep it; if the result is no, filter it.

[0031] Step S6.4.2: Filter out duplicate cells to obtain the actual cell coordinate information.

[0032] Preferably, in step S7:

[0033] Step S7.1: Based on the coordinate information of the table area in the cell coordinate information, obtain the image of the non-table area;

[0034] Step S7.2: Perform text recognition on the image outside the table area using PaddleOCR to obtain the coordinate information of each text box in the image and the recognized content in the text box;

[0035] Step S7.3: Based on the vertical coordinate information of each text box, merge the recognized content in the text boxes in the same row, and return the text content of each row to obtain the result data;

[0036] The resulting data is output from top to bottom according to the actual page layout.

[0037] According to the present invention, a table text recognition system based on OCR includes:

[0038] Module M1: Parses PDF files and converts them into image data;

[0039] Module M2: Performs layout analysis on image data to obtain layout analysis results;

[0040] Module M3: Based on the layout analysis results, determine whether table elements exist; if yes, save the coordinate information of the table detection area; if no, recognize it as plain text; based on a deep learning network, determine the number and area of ​​table detection boxes.

[0041] Module M4: Based on image data, it uses morphological processing to find contours in the image; and based on the contours, it determines the number and region of table detection boxes.

[0042] Module M5: Determines whether the number of detection boxes obtained by module M3 is equal to the number of detection boxes obtained by module M4. If the result is yes, the number of detection boxes obtained by module M4 is used as the input; if the result is no, the result with more detection boxes is used as the input.

[0043] Module M6: Based on the input item, obtain cell information, filter the cell information, and obtain cell coordinate information;

[0044] Module M7: Based on cell coordinate information, it obtains and returns the text content of each line by using the coordinate information of each text box and the recognized content in the text box, thus obtaining the result data.

[0045] Preferably, in module M2: according to the actual page layout of the layout analysis results, the layout elements in the layout analysis results are output from top to bottom;

[0046] The layout analysis results include layout elements and their coordinate regions in the image;

[0047] The layout elements include: text, titles, tables, and images.

[0048] Preferably, in module M3, the number and region of table detection boxes are obtained based on deep learning.

[0049] In module M4:

[0050] Module M4.1: After binarizing the image data, perform morphological processing to obtain the horizontal and vertical lines of the table in the image, and calculate the intersection of the horizontal and vertical lines;

[0051] Module M4.2: Based on the intersection of horizontal and vertical lines, it obtains the contours in the image using OpenCV, and derives the number and region of table detection boxes based on the bounding rectangle table detection method.

[0052] In module M5, if the number of detection boxes obtained by the deep learning method is greater than the number of detection boxes obtained by the bounding rectangle table detection method, and the number of detection boxes obtained by the bounding rectangle table detection method is greater than or equal to one, it is determined whether the position of the box obtained by the bounding rectangle table detection method is within the box obtained by the deep learning method.

[0053] If the result is yes, then the position information of the bounding box detected by the deep learning method will be replaced with the bounding box detected by the bounding rectangle table detection method.

[0054] If the result is negative, the position information of the bounding box detected by deep learning method is used.

[0055] Preferably, in module M6:

[0056] Module M6.1: Based on the table detection results obtained in step S5, perform binarization and morphological processing on each table detection result, i.e. each table, and then calculate the binary images of the horizontal and vertical lines of the table. Based on the binary images of the horizontal and vertical lines, calculate the binary image of the intersection point of the table.

[0057] Module M6.2: Based on the binary graph of the intersection points of the table, obtain the coordinates of each intersection point, group the intersection point coordinates and the intersection points of each horizontal and vertical line, and obtain the grouped intersection point coordinates;

[0058] Module M6.3: Based on the coordinates of the intersection points after grouping, determine the number of horizontal and vertical lines in the table, and iterate through all cells to represent them and obtain the cell information;

[0059] The cell is represented as [starting row, ending row, starting column, ending column];

[0060] Module M6.4: Filters all displayed cells to obtain the final actual cell coordinate information;

[0061] In module M6.4:

[0062] Module M6.4.1: Based on the cell information, obtain the coordinates of each cell and determine whether the four intersection points of the cells satisfy the rectangle condition; if the result is yes, keep it; if the result is no, filter it.

[0063] Module M6.4.2: Filters out duplicate cells to obtain the actual cell coordinate information.

[0064] Preferably, in module M7:

[0065] Module M7.1: Obtains the image of a non-table area based on the coordinate information of the table area in the cell coordinate information;

[0066] Module M7.2: Performs text recognition on images outside the table area using PaddleOCR, obtaining the coordinate information of each text box in the image and the recognized content in the text box;

[0067] Module M7.3: Based on the vertical coordinate information of each text box, merge the recognized content of text boxes in the same row, and return the text content of each row to obtain the result data;

[0068] The resulting data is output from top to bottom according to the actual page layout.

[0069] Compared with the prior art, the present invention has the following beneficial effects:

[0070] 1. This invention provides a complete method for recognizing tables and text in PDF files. By analyzing the layout of each page of the file, it provides effective information for subsequent table detection and cell detection.

[0071] 2. This invention performs table detection on a webpage, identifying table areas that exist on the page, thus preparing for cell detection and information extraction; this method can improve the accuracy of table detection.

[0072] 3. This invention detects and extracts information from cells, performs OCR recognition on the detected cells to obtain the text information of each cell, thereby improving the text recognition rate of cells. It also distinguishes between tables and text on the page through a table text layout algorithm, and finally outputs the recognition result. Attached Figure Description

[0073] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0074] Figure 1 This is a schematic diagram of the OCR table text recognition process provided by the present invention. Detailed Implementation

[0075] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make several changes and improvements without departing from the concept of the present invention. These all fall within the scope of protection of the present invention.

[0076] This invention achieves accurate detection of tables in documents. It detects the horizontal and vertical lines of tables using traditional image processing methods and identifies their intersections. The coordinates of cells are then determined based on these intersections, enabling the recognition of tables and text content in PDF documents and returning accurate cell and text content information. Furthermore, this invention performs layout analysis and content formatting in PDF documents, classifying tables and text content using delimiters and outputting the table and text content based on their actual positions on the page.

[0077] According to the present invention, a table text recognition method based on OCR includes:

[0078] Step S1: The input PDF file is parsed using the PDF parsing module, and each page of the PDF file is converted into image data and saved.

[0079] The PDF parsing module is used to parse PDF files and convert them into image data.

[0080] Step S2: Based on the layout analysis module in Paddle, obtain the layout analysis results for each page of images, and output the layout elements in the layout analysis results from top to bottom according to the actual page layout.

[0081] Specifically, the layout analysis results represent the layout structure elements contained in the page, that is, the layout elements and the coordinate areas of the corresponding elements in the image; the layout elements include: text, title, table, and figure.

[0082] Step S3: Based on the layout analysis results of step S2, determine whether there are table elements. If there are table elements, save the coordinate information of the table detection area; if there are no table elements, recognize the page as plain text.

[0083] In other words, based on the layout analysis results, it is determined whether table elements exist; if yes, the coordinate information of the table detection area is saved; if no, it is recognized as plain text; based on deep learning, the number and area of ​​table detection boxes are obtained.

[0084] The deep learning method, specifically a deep learning-based table detection method, includes:

[0085] Step 1: Call the open-source Baidu PaddlePaddle PPstructure interface to obtain the layout analysis results from the input image. The layout analysis results include the layout types included in each page, such as the detection areas of various types such as tables, text, and titles.

[0086] Step 2: When the layout type is table, calculate the coordinates of the layout area and put these coordinates into a list. The list stores the coordinate positions of all the detection boxes.

[0087] Step S4: Preprocess the image, i.e., morphological processing. Use image binarization followed by morphological processing to obtain the horizontal and vertical lines of the table, and calculate the intersection of the horizontal and vertical lines.

[0088] Based on the intersection of horizontal and vertical lines, OpenCV is used to find contours in the image. The outermost rectangle of these contours is the bounding box rectangle of the table. These bounding box rectangles are filtered to remove rectangular areas in the table where the table and contour areas are less than a threshold, and finally the actual coordinates of the table area are retained.

[0089] In other words, based on image data, morphological processing is used to find contours in the image; and based on the contours, the number and region of table detection boxes are obtained using the bounding rectangle table detection method.

[0090] The bounding rectangle table detection method, namely a table detection method based on the bounding rectangle of a table, includes:

[0091] Step 1: Binarize and perform morphological processing on the image to obtain the outline of the table in the image;

[0092] Step 2: Calculate the maximum and minimum x and y coordinates for each contour to obtain the coordinate information of the bounding rectangle of each contour;

[0093] Step 3: Based on these coordinate information, obtain the bounding rectangle of each table, filter out the bounding rectangles that do not meet the conditions, and finally obtain the bounding rectangles that meet the conditions, i.e., the table area.

[0094] The length and width of the circumscribed rectangle are both less than 10 pixels; and it is completely inside another circumscribed rectangle.

[0095] Step S5: By comparing the number of bounding boxes detected by the deep learning table detection method and the bounding rectangle table detection method, the result of the method with more detected boxes is usually used as the subsequent input. In non-typical cases, if the number of bounding boxes is the same, the result of the bounding rectangle table detection method is used. Experiments show that the bounding rectangle table detection method detects table regions more accurately. If the deep learning method detects more boxes and the bounding rectangle table detection method detects at least one box, it is necessary to determine whether the position of the bounding rectangle table detection method's box is within the box detected by the deep learning method. If it is, the position information of this table in the deep learning method's box needs to be replaced. If it is not, no replacement is performed, and the coordinate result of the deep learning method's detected box is used.

[0096] For example, if the deep learning method detects two tables, and the bounding rectangle table detection method detects one table, then the Intersection over Union (IoU) between the bounding rectangle table detection method's bounding box and the deep learning method's bounding box needs to be calculated. If an IoU exceeds a certain threshold and is the maximum value, the coordinates of the corresponding box are replaced with the result detected by the bounding rectangle table detection method. In other words, for each actual sample, the number of bounding rectangle table detection boxes and the actual coordinate values ​​of the deep learning method's bounding box are calculated. By comparing the IoU values ​​of every two boxes, if the condition is met, the deep learning box is replaced with the bounding rectangle table detection box.

[0097] The outer rectangle table detection method is named to distinguish it from deep learning methods.

[0098] The aforementioned bounding rectangle table detection method treats different layouts in a document as different targets for detection or segmentation. The task of table recognition is to identify and extract table information from the document.

[0099] Step S6: Using the table detection results from Step S5 as input for cell information extraction, the algorithm takes the region in the table detection results as the input image, calculates, supplements, and filters the intersection points of the image; through the intersection points, it obtains the number of horizontal and vertical lines in the entire image, the number of points on each line, and the coordinates of each point; it uses the indices of the horizontal and vertical lines to represent all cells, and filters the cells, retaining the smallest actual unit cell, finally obtaining the coordinate position of each cell; using image processing, it extracts the cell region and performs OCR text recognition sequentially to obtain the text content in each cell.

[0100] Specifically, step S6.1: Based on the input table detection results after comparison, judge each table detection result (i.e. each table). First, perform binarization and morphological processing on the table image. Calculate the binary images of the horizontal and vertical lines according to their characteristics. Calculate the binary images of the table intersections based on the binary images of the horizontal and vertical lines.

[0101] Step S6.2: Based on the binary graph of the intersection points in the table, obtain the coordinates of each intersection point, group the intersection point coordinates, determine and group the intersection points on each horizontal or vertical line, and obtain the intersection point coordinates on each horizontal or vertical line.

[0102] Step S6.3: Based on the coordinates of the intersection points after grouping, determine the number of horizontal and vertical lines in the table, represent the cells, and represent each cell with [starting row, ending row, starting column, ending column], and traverse to represent all cells.

[0103] Step S6.4: Filter all the displayed cells. First, filter cells that do not meet the rectangle condition. Calculate the coordinates of each cell using the cell information. If four points of a cell do not meet the rectangle condition, filter it. Second, filter out duplicate cells. In step S6.3, all cells were displayed, but in reality, only the first cell sharing the starting row and column is needed. Therefore, the other cells need to be filtered. After these two filtering steps, the final, actual cell coordinate information is obtained.

[0104] Step S7: Based on the coordinate information of the table area in the image obtained in step S6, the table area is occluded to obtain the image of the non-table area. PaddleOCR is used to perform full-image text recognition on this part of the image to obtain the coordinate information of each text box and the recognition content in the text box. By calculating the vertical coordinate information of each text box, the text box content in the same row is merged, and finally the text content of each row is returned to obtain the result data.

[0105] Based on the relationship between the vertical coordinates of the table area and the coordinates of the text boxes, the table position is arranged, and separators are added to the top and bottom edges of the table.

[0106] Implementation Example 1:

[0107] like Figure 1 As shown, the table text recognition solution used in this invention is mainly divided into three parts.

[0108] The first part is data preprocessing, including PDF parsing, which converts PDF contract documents into images, and preprocessing each page of images by binarization and morphological processing to obtain binary images of the horizontal and vertical lines of the table and the intersection binary images.

[0109] The second part is table and cell detection;

[0110] First, the layout analysis results of each page are obtained through PP-Structure layout analysis. The number of tables and coordinate regions are calculated using the layout analysis results as the detection results of the deep learning method.

[0111] The bounded rectangle table detection method is used to find the outline of the table region in the preprocessed image, and the coordinate position of the real table region is obtained by filtering the outline.

[0112] Then, the table selection algorithm is used to compare the table detection results of the deep learning method and the table detection results of the bounding rectangle table method. The result of which method is adopted as the final result is determined according to the number of tables.

[0113] The third part is the post-processing of the table detection results, including table and text layout and output results. Using a table and text layout algorithm, the specific location information of the tables and text contained on each page is calculated, and the recognition results are output from top to bottom according to the actual page layout.

[0114] The table text layout algorithm distinguishes the detected and recognized table areas from the text in non-table areas using separators.

[0115] For example, the content on a page, from top to bottom, is: table area - non-table text - table area;

[0116] First, calculate the range of the vertical coordinates of the two table regions. For example, the range of the vertical coordinates of Table 1 is [table1_y1, table2_y2], and the range of the vertical coordinates of Table 2 is [table2_y1, table2_y2].

[0117] After masking the table area, perform full-image OCR recognition. At this time, the table area is filled with white. Calculate the vertical coordinate range of the non-table area, i.e., the text part, for example, [text_y1, text_y2].

[0118] Finally, conditional checks are performed on each page; the content is checked, specifically whether it is a table and the relationship between the vertical axes; if no table exists, no separator is inserted; if a table exists, a separator is inserted between the corresponding table and the text, as shown in Table 1:

[0119]

[0120] Table 1

[0121] The present invention also provides an OCR-based table text recognition system, which can be implemented by executing the process steps of the OCR-based table text recognition method. That is, those skilled in the art can understand the OCR-based table text recognition method as a preferred embodiment of the OCR-based table text recognition system.

[0122] According to the present invention, a table text recognition system based on OCR includes:

[0123] Module M1: Parses PDF files and converts them into image data;

[0124] Module M2: Performs layout analysis on image data to obtain layout analysis results;

[0125] Module M3: Based on the layout analysis results, determine whether table elements exist; if yes, save the coordinate information of the table detection area; if no, recognize it as plain text; based on a deep learning network, determine the number and area of ​​table detection boxes.

[0126] Module M4: Based on image data, it uses morphological processing to find contours in the image; and based on the contours, it determines the number and region of table detection boxes.

[0127] Module M5: Determines whether the number of detection boxes obtained by module M3 is equal to the number of detection boxes obtained by module M4. If the result is yes, the number of detection boxes obtained by module M4 is used as the input; if the result is no, the result with more detection boxes is used as the input.

[0128] Module M6: Based on the input item, obtain cell information, filter the cell information, and obtain cell coordinate information;

[0129] Module M7: Based on cell coordinate information, it obtains and returns the text content of each line by using the coordinate information of each text box and the recognized content in the text box, thus obtaining the result data.

[0130] Specifically, in module M2: according to the actual page layout of the layout analysis results, the layout elements in the layout analysis results are output from top to bottom;

[0131] The layout analysis results include layout elements and their coordinate regions in the image;

[0132] The layout elements include: text, titles, tables, and images.

[0133] Specifically, in module M3, the number and region of table detection boxes are obtained based on deep learning.

[0134] In module M4:

[0135] Module M4.1: After binarizing the image data, perform morphological processing to obtain the horizontal and vertical lines of the table in the image, and calculate the intersection of the horizontal and vertical lines;

[0136] Module M4.2: Based on the intersection of horizontal and vertical lines, it obtains the contours in the image using OpenCV, and derives the number and region of table detection boxes based on the bounding rectangle table detection method.

[0137] In module M5, if the number of detection boxes obtained by the deep learning method is greater than the number of detection boxes obtained by the bounding rectangle table detection method, and the number of detection boxes obtained by the bounding rectangle table detection method is greater than or equal to one, it is determined whether the position of the box obtained by the bounding rectangle table detection method is within the box obtained by the deep learning method.

[0138] If the result is yes, then the position information of the bounding box detected by the deep learning method will be replaced with the bounding box detected by the bounding rectangle table detection method.

[0139] If the result is negative, the position information of the bounding box detected by deep learning method is used.

[0140] Specifically, in module M6:

[0141] Module M6.1: Based on the table detection results obtained in step S5, perform binarization and morphological processing on each table detection result, i.e. each table, and then calculate the binary images of the horizontal and vertical lines of the table. Based on the binary images of the horizontal and vertical lines, calculate the binary image of the intersection point of the table.

[0142] Module M6.2: Based on the binary graph of the intersection points of the table, obtain the coordinates of each intersection point, group the intersection point coordinates and the intersection points of each horizontal and vertical line, and obtain the grouped intersection point coordinates;

[0143] Module M6.3: Based on the coordinates of the intersection points after grouping, determine the number of horizontal and vertical lines in the table, and iterate through all cells to represent them and obtain the cell information;

[0144] The cell is represented as [starting row, ending row, starting column, ending column];

[0145] Module M6.4: Filters all displayed cells to obtain the final actual cell coordinate information;

[0146] In module M6.4:

[0147] Module M6.4.1: Based on the cell information, obtain the coordinates of each cell and determine whether the four intersection points of the cells satisfy the rectangle condition; if the result is yes, keep it; if the result is no, filter it.

[0148] Module M6.4.2: Filters out duplicate cells to obtain the actual cell coordinate information.

[0149] Specifically, in module M7:

[0150] Module M7.1: Obtains the image of a non-table area based on the coordinate information of the table area in the cell coordinate information;

[0151] Module M7.2: Performs text recognition on images outside the table area using PaddleOCR, obtaining the coordinate information of each text box in the image and the recognized content in the text box;

[0152] Module M7.3: Based on the vertical coordinate information of each text box, merge the recognized content of text boxes in the same row, and return the text content of each row to obtain the result data;

[0153] The resulting data is output from top to bottom according to the actual page layout.

[0154] Those skilled in the art will understand that, besides implementing the system and its various devices, modules, and units provided by this invention in the form of purely computer-readable program code, the same functions can be achieved entirely through logical programming of the method steps, enabling the system and its various devices, modules, and units to function in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, the system and its various devices, modules, and units provided by this invention can be considered a hardware component, and the devices, modules, and units included therein for implementing various functions can also be considered structures within the hardware component; alternatively, the devices, modules, and units for implementing various functions can be considered both software modules implementing the method and structures within the hardware component.

[0155] Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. Unless otherwise specified, the embodiments and features described in this application can be arbitrarily combined with each other.

Claims

1. A table text recognition method based on OCR, characterized in that, include: Step S1: Parse the PDF file and convert it into image data; Step S2: Perform layout analysis on the image data to obtain the layout analysis results; Step S3: Based on the layout analysis results, determine whether table elements exist; If the result is yes, then save the coordinate information of the table detection area; If the result is negative, then recognition will be performed according to plain text. Based on deep learning networks, the number and region of detection boxes in the table are obtained; Step S4: Based on the image data, use morphological processing to find contours in the image; and based on the contours, determine the number of detection boxes and the regions of the table. Step S5: Determine whether the number of detection boxes obtained in step S3 is equal to the number of detection boxes obtained in step S4. If the result is yes, then use the number of detection boxes obtained in step S4 as the input item. If the result is negative, the detection result with more detection boxes will be used as the input. Step S6: Based on the input items, obtain cell information, and filter the cell information to obtain cell coordinate information; Step S7: Based on the cell coordinate information, obtain and return the text content of each line through the coordinate information of each text box and the recognized content in the text box, and obtain the result data; In step S3, the number and region of table detection boxes are obtained based on deep learning. In step S4: Step S4.1: After binarizing the image data, perform morphological processing to obtain the horizontal and vertical lines of the table in the image, and calculate the intersection of the horizontal and vertical lines; Step S4.2: Based on the intersection of the horizontal and vertical lines, obtain the contours in the image using OpenCV, and determine the number and region of the table detection boxes based on the bounding rectangle table detection method. In step S5, if the number of detection boxes obtained by the deep learning method is greater than the number of detection boxes obtained by the bounding rectangle table detection method, and the number of detection boxes obtained by the bounding rectangle table detection method is greater than or equal to one, it is determined whether the position of the box obtained by the bounding rectangle table detection method is within the box obtained by the deep learning method. If the result is yes, then the position information of the bounding box detected by the deep learning method will be replaced with the bounding box detected by the bounding rectangle table detection method. If the result is negative, the position information of the bounding box detected by deep learning method is used.

2. The OCR-based table text recognition method according to claim 1, characterized in that, In step S2: According to the actual page layout of the layout analysis results, output the layout elements in the layout analysis results from top to bottom; The layout analysis results include layout elements and their coordinate regions in the image; The layout elements include: text, titles, tables, and images.

3. The OCR-based table text recognition method according to claim 1, characterized in that, In step S6: Step S6.1: Based on the table detection results obtained in step S5, perform binarization and morphological processing on each table detection result, i.e. each table, to calculate the binary images of the horizontal and vertical lines of the table, and then calculate the binary images of the table intersection points based on the binary images of the horizontal and vertical lines. Step S6.2: Based on the binary graph of the intersection points in the table, obtain the coordinates of each intersection point. Group the intersection point coordinates and the intersection points of each horizontal and vertical line to obtain the grouped intersection point coordinates. Step S6.3: Based on the coordinates of the intersection points after grouping, determine the number of horizontal and vertical lines in the table, and iterate through all cells to represent them, thus obtaining the cell information; The cell is represented as [starting row, ending row, starting column, ending column]; Step S6.4: Filter all the displayed cells to obtain the final actual cell coordinate information; In step S6.4: Step S6.4.1: Obtain the coordinates of each cell using the cell information, and determine whether the four intersection points of the cells satisfy the rectangle condition; if the result is yes, keep it; if the result is no, filter it. Step S6.4.2: Filter out duplicate cells to obtain the actual cell coordinate information.

4. The OCR-based table text recognition method according to claim 3, characterized in that, In step S7: Step S7.1: Based on the coordinate information of the table area in the cell coordinate information, obtain the image of the non-table area; Step S7.2: Perform text recognition on the image outside the table area using PaddleOCR to obtain the coordinate information of each text box in the image and the recognized content in the text box; Step S7.3: Based on the vertical coordinate information of each text box, merge the recognized content in the text boxes in the same row, and return the text content of each row to obtain the result data; The resulting data is output from top to bottom according to the actual page layout.

5. A table text recognition system based on OCR, characterized in that, include: Module M1: Parses PDF files and converts them into image data; Module M2: Performs layout analysis on image data to obtain layout analysis results; Module M3: Determines whether table elements exist based on layout analysis results; If the result is yes, then save the coordinate information of the table detection area; If the result is negative, then recognition will be performed according to plain text. Based on deep learning networks, the number and region of detection boxes in the table are obtained; Module M4: Based on image data, it uses morphological processing to find contours in the image; and based on the contours, it determines the number of detection boxes in the table. Module M5: Determine whether the number of detection boxes obtained by module M3 is equal to the number of detection boxes obtained by module M4. If the result is yes, then the number of detection boxes obtained by module M4 is used as the input item. If the result is negative, the detection result with more detection boxes will be used as the input. Module M6: Based on the input item, obtain cell information, filter the cell information, and obtain cell coordinate information; Module M7: Based on cell coordinate information, it obtains and returns the text content of each line by using the coordinate information of each text box and the recognized content in the text box, thus obtaining the result data; In module M3, the number and region of table detection boxes are obtained based on deep learning. In module M4: Module M4.1: After binarizing the image data, perform morphological processing to obtain the horizontal and vertical lines of the table in the image, and calculate the intersection of the horizontal and vertical lines; Module M4.2: Based on the intersection of horizontal and vertical lines, the contours in the image are obtained through OpenCV. Based on the bounding rectangle table detection method, the number and region of table detection boxes are obtained. In module M5, if the number of detection boxes obtained by the deep learning method is greater than the number of detection boxes obtained by the bounding rectangle table detection method, and the number of detection boxes obtained by the bounding rectangle table detection method is greater than or equal to one, it is determined whether the position of the box obtained by the bounding rectangle table detection method is within the box obtained by the deep learning method. If the result is yes, then the position information of the bounding box detected by the deep learning method will be replaced with the bounding box detected by the bounding rectangle table detection method. If the result is negative, the position information of the bounding box detected by deep learning method is used.

6. The OCR-based table text recognition system according to claim 5, characterized in that, In module M2: Based on the actual page layout of the layout analysis results, the layout elements in the layout analysis results are output from top to bottom; The layout analysis results include layout elements and their coordinate regions in the image; The layout elements include: text, titles, tables, and images.

7. The OCR-based table text recognition system according to claim 5, characterized in that, In module M6: Module M6.1: Based on the table detection results obtained in step S5, perform binarization and morphological processing on each table detection result, i.e. each table, and then calculate the binary images of the horizontal and vertical lines of the table. Based on the binary images of the horizontal and vertical lines, calculate the binary image of the intersection point of the table. Module M6.2: Based on the binary graph of the intersection points of the table, obtain the coordinates of each intersection point, group the intersection point coordinates and the intersection points of each horizontal and vertical line, and obtain the grouped intersection point coordinates; Module M6.3: Based on the coordinates of the intersection points after grouping, determine the number of horizontal and vertical lines in the table, and iterate through all cells to represent them and obtain the cell information; The cell is represented as [starting row, ending row, starting column, ending column]; Module M6.4: Filters all displayed cells to obtain the final actual cell coordinate information; In module M6.4: Module M6.4.1: Based on cell information, obtain the coordinates of each cell and determine whether the four intersection points of the cells satisfy the rectangle condition; If the result is yes, keep it; if the result is no, filter it. Module M6.4.2: Filters out duplicate cells to obtain the actual cell coordinate information.

8. The OCR-based table text recognition system according to claim 7, characterized in that, In module M7: Module M7.1: Obtains the image of a non-table area based on the coordinate information of the table area in the cell coordinate information; Module M7.2: Performs text recognition on images outside the table area using PaddleOCR, obtaining the coordinate information of each text box and the recognized content within the text box; Module M7.3: Based on the vertical coordinate information of each text box, merge the recognized content of text boxes in the same row, and return the text content of each row to obtain the result data; The resulting data is output from top to bottom according to the actual page layout.

Images