Table extraction method and table extraction system

Abstract

The present invention relates to the field of image processing technology, and discloses a table extraction method and system. The method comprises: segmenting a table image through an image segmentation model to obtain a mask image marked with line attributes at each pixel point; processing the mask image using an image science method to obtain the vertex coordinates, starting row index, starting column index, ending row index, and ending column index of each cell; recognizing the table image through a text recognition model to obtain the vertex coordinates and text content of the minimum circumscribed rectangle of each text; determining the text content in each cell according to the vertex coordinates of each cell and the vertex coordinates of the minimum circumscribed rectangle of each text; and generating a table presented in a target file according to the above-mentioned row index, column index, and text content. The present invention can still accurately extract tables of any format even if the table is not clearly printed or has large noise interference.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a table extraction method and a table extraction system. Background Art

[0002] Table OCR (Optical Character Recognition) refers to extracting the structural information of the table in the image, extracting the text information in the image in combination with OCR, and restoring the information in the table in the form of Excel.

[0003] Traditional table OCR methods can be divided into the following two methods. One method is to use OCR to detect and identify the text in the table, and deduce which rows, columns, and cells need to be merged based on the spatial layout information of the text box, thereby generating a spreadsheet. For documents with non-uniform data table formats, this method requires writing endless rules to structure the recognized text, and it is difficult to achieve universality. Another method is to use image morphological transformation and combine corresponding thresholds to extract table lines, and then combine OCR results to structure the information to generate a spreadsheet. This method has poor anti-interference ability. If the table frame lines are not printed clearly or the noise interference is large, it will affect the table frame line detection results, resulting in the final information being difficult to restore. Summary of the invention

[0004] The object of the present invention is to provide a table extraction method and a table extraction system, which can accurately extract tables of any format even when the table is not clearly printed or has a lot of noise interference.

[0005] In order to achieve the above-mentioned purpose, the first aspect of the present invention provides a table extraction method, which includes: segmenting the table image through an image segmentation model to obtain a mask image marked with line attributes at each pixel point; processing the mask image using an image science method to obtain the vertex coordinates of each cell and the starting row index, starting column index, ending row index and ending column index of each cell; recognizing the table image through a text recognition model to obtain the vertex coordinates and text content of the minimum enclosing rectangle of each text in the table image; determining the text content in each cell according to the vertex coordinates of each cell and the vertex coordinates of the minimum enclosing rectangle of each text; and generating a table presented in a target file according to the starting row index, starting column index, ending row index and ending column index of each cell and the text content in each cell.

[0006] Preferably, the processing of the mask image by an image science method comprises: marking the connected domains of the mask image to obtain each connected domain; obtaining the head coordinates and tail coordinates of four line segments in the minimum circumscribed rectangular box matching each connected domain according to the coordinates of the pixel points in the each connected domain; establishing a cell matrix according to the head coordinates and tail coordinates of the four line segments in the minimum circumscribed rectangular box matching each connected domain; and marking the connected domains of the cell matrix to obtain the vertex coordinates of each cell.

[0007] Preferably, the step of obtaining the head coordinates and tail coordinates of four line segments in the minimum circumscribed rectangular box matching each connected domain comprises: obtaining the vertex coordinates of the minimum circumscribed rectangular box matching each connected domain according to the coordinates of the pixel points in each connected domain; and determining the head coordinates and tail coordinates of each line segment in the minimum circumscribed rectangular box according to the vertex coordinates of the minimum circumscribed rectangular box matching each connected domain and the direction of each line segment in the minimum circumscribed rectangular box.

[0008] Preferably, the determining of the head coordinate and the tail coordinate of each line segment comprises: for a horizontal line segment in the minimum circumscribed rectangular frame, determining the minimum value of the horizontal coordinate of the minimum circumscribed rectangular frame as the head horizontal coordinate, determining the average of the vertical coordinates of the upper left vertex and the lower left vertex of the minimum circumscribed rectangular frame as the head vertical coordinate, determining the maximum value of the horizontal coordinate of the minimum circumscribed rectangular frame as the tail horizontal coordinate, and determining the average of the vertical coordinates of the upper right vertex and the lower right vertex of the minimum circumscribed rectangular frame as the tail vertical coordinate; and for a vertical line segment in the minimum circumscribed rectangular frame, determining the minimum value of the vertical coordinate of the minimum circumscribed rectangular frame as the head vertical coordinate, determining the average of the horizontal coordinates of the upper left vertex and the upper right vertex of the minimum circumscribed rectangular frame as the head horizontal coordinate, determining the maximum value of the vertical coordinate of the minimum circumscribed rectangular frame as the tail vertical coordinate, and determining the average of the horizontal coordinates of the lower left vertex and the lower right vertex of the minimum circumscribed rectangular frame as the tail horizontal coordinate.

[0009] Preferably, the processing of the mask image by an image science method further includes: sorting the horizontal line segments in ascending order according to the head vertical coordinate or the tail vertical coordinate of the horizontal line segments in the minimum circumscribed rectangular box that matches the connected domains to form a first set; sorting the vertical line segments in ascending order according to the head horizontal coordinate or the tail horizontal coordinate of the vertical line segments in the minimum circumscribed rectangular box that matches the connected domains to form a second set; determining the line segment in the first set that is closest to the coordinates of the upper left corner vertex of each cell as the starting row index of each cell, and determining the line segment in the first set that is closest to the coordinates of the lower right corner vertex of each cell as the ending row index of each cell; and determining the line segment in the second set that is closest to the coordinates of the upper left corner vertex of each cell as the starting column index of each cell, and determining the line segment in the second set that is closest to the coordinates of the lower right corner vertex of each cell as the ending column index of each cell. Preferably, determining the text content within each cell includes: traversing each cell, and when the intersection area of ​​the minimum enclosing rectangle of the specific text and the specific cell is greater than a critical threshold, determining that the text content of the minimum enclosing rectangle of the specific text is the text content within the specific cell.

[0010] Preferably, the critical threshold is the product of a preset value and the minimum value of the area of ​​the minimum circumscribed rectangle of the specific text and the area of ​​the specific cell.

[0011] Through the above technical scheme, the present invention creatively first segments the table image through an image segmentation model to obtain a mask image marked with line attributes at each pixel point; then uses an image science method to process the mask image to obtain the vertex coordinates of each cell and the starting row index, starting column index, ending row index and ending column index of each cell; then recognizes the table image through a text recognition model to obtain the vertex coordinates and text content of the minimum circumscribed rectangle of each text in the table image; then determines the text content in each cell according to the vertex coordinates of each cell and the vertex coordinates of the minimum circumscribed rectangle of each text; finally, generates a table presented in a target file according to the starting row index, starting column index, ending row index and ending column index of each cell and the text content in each cell. Therefore, the present invention can also accurately extract tables of any format when the table printing is unclear or the noise interference is large.

[0012] The second aspect of the present invention provides a table extraction system, which includes: a segmentation device for segmenting a table image through an image segmentation model to obtain a mask image marked with line attribute vertical lines at each pixel point; a processing device for processing the mask image using an image science method to obtain the vertex coordinates of each cell and the starting row index, starting column index, ending row index and ending column index of each cell; a text recognition device for recognizing the table image through a text recognition model to obtain the vertex coordinates of the minimum circumscribed rectangle of each text in the table image and the text content text recognition model; a text determination device for determining the text content in each cell according to the vertex coordinates of each cell and the vertex coordinates of the minimum circumscribed rectangle of each text; and a generation device for generating a table presented in a target file according to the starting row index, starting column index, ending row index and ending column index of each cell and the text content in each cell.

[0013] The specific details and benefits of the table extraction system provided by the present invention can be found in the above description of the table extraction method, which will not be repeated here.

[0014] The third aspect of the present invention further provides a machine-readable storage medium, on which instructions are stored, and the instructions are used to enable a machine to execute the above-mentioned table extraction method.

[0015] The fourth aspect of the present invention also provides an electronic device, which includes: a processor; a memory for storing executable instructions of the processor; the processor is used to read the executable instructions from the memory and execute the instructions to implement the above-mentioned table extraction method.

[0016] A fifth aspect of the present invention further provides a computer program product, the computer program product comprising a computer program, and the computer program implements the above-mentioned table extraction method when executed by a processor.

[0017] Other features and advantages of the present invention will be described in detail in the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings are used to provide a further understanding of the embodiments of the present invention and constitute a part of the specification. Together with the following specific embodiments, they are used to explain the embodiments of the present invention, but do not constitute a limitation on the embodiments of the present invention. In the accompanying drawings:

[0019] Figure 1 is a flow chart of a table extraction method provided by an embodiment of the present invention;

[0020] Figure 2is a flow chart of processing the mask image using an image science method provided by an embodiment of the present invention;

[0021] Figure 3 is a line segment annotation sample provided by an embodiment of the present invention;

[0022] Figure 4 This is a line segmentation effect diagram provided by an embodiment of the present invention;

[0023] Figure 5 is a cell calculation effect diagram provided by an embodiment of the present invention; and

[0024] Figure 6 It is a flow chart of a table extraction method provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0025] The specific implementation of the present invention is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described here is only used to illustrate and explain the present invention, and is not used to limit the present invention.

[0026] Figure 1 FIG. 1 is a flow chart of a table extraction method provided by an embodiment of the present invention. Figure 1 As shown, the table extraction method may include steps S101-S105.

[0027] Before executing step S101, the training and verification of the image segmentation model are briefly described.

[0028] First, label the table border training data using labelme (a labeling tool for object segmentation). The labeling principle is: label the horizontal line (i.e. horizontal line segment) with two points from left to right (labeled as 1), and label the vertical line (i.e. vertical line segment) with two points from top to bottom (labeled as 2), so as to generate the training set. The labeling file is as follows: Figure 3 shown.

[0029] Secondly, the training set is input into the image segmentation model (such as the DarkNet model, specifically the DarkNet YOLOv4 model) for training; different segmentation models are obtained through training, and the model selection is based on the effect in the validation set. The verification standard is to calculate the IOU value of the predicted result and the actual result of the validation set. Among them, the IOU value is an indicator to measure the accuracy of target detection. The IOU value is calculated as the ratio of the intersection and union of the "predicted edge area" and the "real edge area". The higher the IOU value, the more accurate the image segmentation model. In one embodiment, the image segmentation model with the largest IOU value can be selected as the image segmentation model described below.

[0030] Step S101 , segmenting the table image using an image segmentation model to obtain a mask image marked with line attributes at each pixel point.

[0031] The line attribute at each pixel point indicates whether the line at each pixel point is a horizontal line or a vertical line. For example, the flag "1" may be used to indicate that the line is a horizontal line or a vertical line, and "0" may be used to indicate that the line is not a horizontal line or a vertical line.

[0032] The table image to be extracted is input into the image segmentation model for segmentation by the image segmentation model. The output results contain two matrices, the dimensions of which are the same as the image size: each element in one matrix represents whether the pixel at this position in the original image is a horizontal line ("1" for yes; "0" for no), and each element in the other matrix represents whether the pixel at this position in the original image is a vertical line ("1" for yes; "0" for no). Figure 4 It is the result of superimposing the mask image of two matrices output by the image segmentation model, where white represents line segments and black represents non-line segments.

[0033] In step S101, using a deep learning model instead of traditional image processing methods can greatly improve the accuracy and robustness of frame segmentation.

[0034] The mask image of the line segment is obtained through step S101, which is equivalent to only obtaining a set of points belonging to the line segment area, but not obtaining the head and tail coordinates of each line segment and the coordinates of the cell. The mask image will be processed using an imaging method below.

[0035] Step S102 , using an image science method to process the mask image to obtain the vertex coordinates of each cell and the starting row index, starting column index, ending row index and ending column index of each cell.

[0036] For step S102, the processing of the mask image using an image processing method may include steps S201-S204, such as Figure 2 shown.

[0037] Step S201 : marking connected domains of the mask image to obtain connected domains.

[0038] Use image processing methods such as dilation and erosion (such as skimage.measure.label()) to mark the connected domains of the mask image and obtain all the connected domains.

[0039] Step S202 : acquiring head coordinates and tail coordinates of four line segments in a minimum circumscribed rectangular frame matching each connected domain according to the coordinates of the pixel points in each connected domain.

[0040] For step S202, the step of obtaining the head coordinates and tail coordinates of the four line segments in the minimum circumscribed rectangular box that matches the connected domains includes: obtaining the vertex coordinates of the minimum circumscribed rectangular box that matches the connected domains according to the coordinates of the pixel points in the connected domains; and determining the head coordinates and tail coordinates of each line segment in the minimum circumscribed rectangular box according to the vertex coordinates of the minimum circumscribed rectangular box that matches the connected domains and the direction of each line segment in the minimum circumscribed rectangular box.

[0041] Among them, determining the head coordinate and the tail coordinate of each line segment may include: for the horizontal line segment in the minimum circumscribed rectangular frame, determining the minimum value of the horizontal coordinate of the minimum circumscribed rectangular frame as the head horizontal coordinate, determining the average of the vertical coordinates of the upper left corner vertex and the lower left corner vertex of the minimum circumscribed rectangular frame as the head vertical coordinate, determining the maximum value of the horizontal coordinate of the minimum circumscribed rectangular frame as the tail horizontal coordinate, and determining the average of the vertical coordinates of the upper right corner vertex and the lower right corner vertex of the minimum circumscribed rectangular frame as the tail vertical coordinate; and for the vertical line segment in the minimum circumscribed rectangular frame, determining the minimum value of the vertical coordinate of the minimum circumscribed rectangular frame as the head vertical coordinate, determining the average of the horizontal coordinates of the upper left corner vertex and the upper right corner vertex of the minimum circumscribed rectangular frame as the head horizontal coordinate, determining the maximum value of the vertical coordinate of the minimum circumscribed rectangular frame as the tail vertical coordinate, and determining the average of the horizontal coordinates of the lower left corner vertex and the lower right corner vertex of the minimum circumscribed rectangular frame as the tail horizontal coordinate.

[0042] Specifically, for any connected domain, first, take the coordinates of all pixel points in this connected domain to obtain the point set coords about the connected domain; then, find the minimum circumscribed rectangular box (referred to as box for short) for the point set coords, and obtain the coordinates of the upper left corner, upper right corner, lower right corner and lower left corner of the box (i.e., vertex coordinates). Next, for the horizontal line of the box, take the minimum value of the horizontal coordinate in the box as the horizontal coordinate value of the head, take the average of the vertical coordinates of the upper left vertex and the lower left vertex of the box as the vertical coordinate value of the head, take the maximum value of the horizontal coordinate in the box as the horizontal coordinate value of the tail, and take the average of the vertical coordinates of the upper right vertex and the lower right vertex of the box as the vertical coordinate value of the tail; for the vertical line of the box, take the minimum value of the vertical coordinate in the box as the vertical coordinate value of the head, take the average of the vertical coordinates of the upper left vertex and the upper right vertex of the box as the horizontal coordinate value of the head, take the maximum value of the vertical coordinate in the box as the vertical coordinate value of the tail, and take the average of the vertical coordinates of the lower left vertex and the lower right vertex of the box as the horizontal coordinate value of the tail, so as to obtain the head and tail coordinates of each line segment.

[0043] The above process is performed for all connected domains to determine the head coordinates and tail coordinates of the four line segments in the minimum circumscribed rectangular frame of each connected domain.

[0044] Step S203: establishing a cell matrix according to the head coordinates and tail coordinates of the four line segments in the minimum circumscribed rectangular frame matching the connected domains.

[0045] Each line segment is placed at a corresponding position in the empty matrix according to the head and tail coordinates of the line segment, that is, a cell matrix is ​​established. The empty matrix may have the same size as the input image.

[0046] Step S204: marking the connected domain of the cell matrix to obtain the vertex coordinates of each cell.

[0047] According to the cell matrix, the connected domain with pixel value 0 is calculated to obtain the four-point coordinates of each connected domain corresponding to the cell, that is, the coordinates of each cell are obtained, such as Figure 5 shown.

[0048] The above content only obtains the coordinates of each cell, but there is no coordinate information in the target file such as Excel. Therefore, if you need to present the table in the form of a target file such as Excel, you need to obtain the starting and ending rows and columns of each cell. To solve this problem, in this embodiment, it is necessary to combine the sorting of the segments in the horizontal segment set and the vertical segment set to determine the row and column index of the cell.

[0049] For step S102, the processing of the mask image by using an image science method further includes:

[0050] According to the head ordinate or tail ordinate of the horizontal line segments in the minimum circumscribed rectangular box matched by each connected domain, the horizontal line segments are sorted in ascending order to form a first set; according to the head ordinate or tail ordinate of the vertical line segments in the minimum circumscribed rectangular box matched by each connected domain, the vertical line segments are sorted in ascending order to form a second set; the line segment in the first set that is closest to the coordinates of the upper left corner vertex of each cell is determined as the starting row index of each cell, and the line segment in the first set that is closest to the coordinates of the lower right corner vertex of each cell is determined as the ending row index of each cell; and the line segment in the second set that is closest to the coordinates of the upper left corner vertex of each cell is determined as the starting column index of each cell, and the line segment in the second set that is closest to the coordinates of the lower right corner vertex of each cell is determined as the ending column index of each cell. First, all horizontal line segments are sorted from top to bottom according to the head vertical coordinate (or tail vertical coordinate) of the horizontal line segment to obtain the horizontal line set RowsLines{horizontal line 0, horizontal line 1, horizontal line 2, ...}; all vertical line segments are sorted from left to right according to the head horizontal coordinate (or tail horizontal coordinate) of the vertical line segment to obtain the vertical line set ColsLines{vertical line 0, vertical line 1, vertical line 2, ...}. Then, traverse each cell and perform the following operations: determine the upper left vertex p1 and the lower right vertex p2 of each cell, calculate and determine the line closest to p1 in the horizontal line set RowsLines, and this line is the starting row row_st; calculate and determine the line closest to p2 in the horizontal line set RowsLines, and this line is the ending row row_end; calculate and determine the line closest to p1 in the vertical line set ColsLines, and this line is the starting column col_st; calculate and determine the line closest to p2 in the vertical line set ColsLines, and this line is the ending column col_end, thereby obtaining the row and column indexes of each cell.

[0051] Therefore, the above step S102 uses image science methods such as dilation and erosion to merge the frame lines, and obtains the relative position relationship of each cell and the coordinates of the four vertices.

[0052] Step S103, recognizing the table image by a text recognition model to obtain the vertex coordinates and text content of the minimum circumscribed rectangle of each text in the table image.

[0053] For step S103, the recognition of the table image through the text recognition model may include: recognizing the table image through the detection module in the text recognition model to obtain the vertex coordinates of the minimum enclosing rectangle of each text; and recognizing each text through the recognition module in the text recognition model to obtain the text content within the minimum enclosing rectangle of each text.

[0054] Among them, the detection module may be a detection module based on CornerNet; and the recognition module may be a recognition module based on ResNet.

[0055] First, the table image is recognized by a detection module based on CornerNet to obtain the coordinate information of the minimum bounding rectangle of each text. Then, each text is recognized by a recognition module based on ResNet to obtain the text content within the minimum bounding rectangle of each text.

[0056] Step S104 , determining the text content in each cell according to the vertex coordinates of each cell and the vertex coordinates of the minimum circumscribed rectangle of each text.

[0057] For step S104, determining the text content within each cell may include: traversing each cell, and when the intersection area of ​​the minimum enclosing rectangle of the specific text and the specific cell is greater than a critical threshold, determining that the text content of the minimum enclosing rectangle of the specific text is the text content within the specific cell.

[0058] The critical threshold is the product of a minimum value of the area of ​​the minimum circumscribed rectangle of the specific text and the area of ​​the specific cell and a preset value.

[0059] For any one of all the recognized texts (for example, a specific text A), traverse each cell and calculate the improved IOU value between the minimum bounding rectangle of the specific text A and each cell according to the following formula:

[0060]

[0061] If the improved IOU value between the minimum bounding rectangle of the specific text A and the specific cell is greater than a preset value (e.g., 0.8), the specific text A is considered to belong to the specific cell. Thus, the method of determining the text content by the improved IOU value can avoid missing text with a smaller area. The above similar process is performed for other texts in all the recognized texts to determine the text content in each cell.

[0062] Step S105 , generating a table presented in a target file according to the starting row index, starting column index, ending row index and ending column index of each cell and the position coordinates and text content of each text box.

[0063] Wherein, the target file is an Excel file.

[0064] First, use a table creation module (such as xlsxwriter) to create an Excel file object. Then, based on the row and column indexes of each cell, determine the number of cells in the Excel file occupied by the cells in each image. Assuming that the starting row and column of a cell in the image is (0, 0) and the ending row and column is (2, 2), the cells of the image correspond to the cells in the first row and first column, the first row and second column, the second row and first column, and the second row and second column in the Excel file. Next, fill the text content in each cell into the Excel file, thereby completing the OCR process of the entire table. That is, use a conversion script to process the table information to convert it into an Excel file.

[0065] Specifically, the table extraction process is briefly described below. Figure 6 shown.

[0066] The table extraction process may include the following steps S601-S608.

[0067] Step S601, input the table image to be extracted, and execute steps S602 and S605.

[0068] Step S602, loading the trained image segmentation model.

[0069] Step S603: output the segmentation result through the image segmentation model and calculate the vertex coordinates of each cell.

[0070] Step S604, calculating the starting row index, starting column index, ending row index and ending column index of each cell.

[0071] Step S605, loading the trained text recognition model.

[0072] Step S606: Output the vertex coordinates and text content of the minimum circumscribed rectangle of each text through the text recognition model.

[0073] Step S607, determining the text content of each cell in the target file.

[0074] Step S608, convert the script to obtain an Excel file.

[0075] In summary, the present invention creatively first segments the table image through an image segmentation model to obtain a mask image marked with line attributes at each pixel point; then uses an image science method to process the mask image to obtain the vertex coordinates of each cell and the starting row index, starting column index, ending row index and ending column index of each cell; then recognizes the table image through a text recognition model to obtain the vertex coordinates and text content of the minimum circumscribed rectangle of each text in the table image; then determines the text content in each cell based on the vertex coordinates of each cell and the vertex coordinates of the minimum circumscribed rectangle of each text; finally, generates a table presented in a target file based on the starting row index, starting column index, ending row index and ending column index of each cell and the text content in each cell. Therefore, the present invention can accurately extract tables of any format even if the table is not printed clearly or has large noise interference.

[0076] An embodiment of the present invention also provides a table extraction system, which includes: a segmentation device for segmenting a table image through an image segmentation model to obtain a mask image marked with line attribute vertical lines at each pixel point; a processing device for processing the mask image using an image science method to obtain the vertex coordinates of each cell and the starting row index, starting column index, ending row index and ending column index of each cell; a text recognition device for recognizing the table image through a text recognition model to obtain the vertex coordinates of the minimum circumscribed rectangle of each text in the table image and the text content text recognition model; a text determination device for determining the text content in each cell according to the vertex coordinates of each cell and the vertex coordinates of the minimum circumscribed rectangle of each text; and a generation device for generating a table presented in a target file according to the starting row index, starting column index, ending row index and ending column index of each cell and the text content in each cell.

[0077] The specific details and benefits of the table extraction system provided by the present invention can be found in the above description of the table extraction method, which will not be repeated here.

[0078] An embodiment of the present invention further provides a machine-readable storage medium, on which instructions are stored, and the instructions are used to enable a machine to execute the above table extraction method.

[0079] An embodiment of the present invention further provides an electronic device, comprising: a processor; a memory for storing instructions executable by the processor; the processor is configured to read the executable instructions from the memory and execute the instructions to implement the above-mentioned table extraction method.

[0080] An embodiment of the present invention further provides a computer program product, the computer program product comprising a computer program, and the computer program implements the above table extraction method when executed by a processor.

[0081] The optional implementation modes of the embodiments of the present invention are described in detail above in conjunction with the accompanying drawings. However, the embodiments of the present invention are not limited to the specific details in the above implementation modes. Within the technical concept of the embodiments of the present invention, various simple modifications can be made to the technical scheme of the embodiments of the present invention, and these simple modifications all belong to the protection scope of the embodiments of the present invention.

[0082] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. To avoid unnecessary repetition, the embodiments of the present invention will not further describe various possible combinations.

[0083] Those skilled in the art can understand that all or part of the steps in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a program, and the program is stored in a storage medium, including a number of instructions to enable a single-chip microcomputer, a chip or a processor to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0084] In addition, various implementation modes of the embodiments of the present invention may be arbitrarily combined, and as long as they do not violate the concept of the embodiments of the present invention, they should also be regarded as the contents disclosed by the embodiments of the present invention.

Claims

1. A table extraction method, characterized in that: The table extraction method comprises: The table image is segmented by an image segmentation model to obtain a mask image marked with line attributes at each pixel point; Processing the mask image using an image science method to obtain the vertex coordinates of each cell and the starting row index, starting column index, ending row index, and ending column index of each cell; Recognize the table image by using a text recognition model to obtain the vertex coordinates and text content of the minimum circumscribed rectangle of each text in the table image; Determine the text content in each cell according to the vertex coordinates of each cell and the vertex coordinates of the minimum circumscribed rectangle of each text; and According to the starting row index, starting column index, ending row index and ending column index of each cell and the text content in each cell, a table presented in a target file is generated, wherein the processing of the mask image by an image science method comprises: Marking connected domains of the mask image to obtain each connected domain; According to the coordinates of the pixel points in each connected domain and the direction of each line segment in the minimum circumscribed rectangular frame matching each connected domain, the head coordinates and the tail coordinates of four line segments in the minimum circumscribed rectangular frame matching each connected domain are obtained; Establishing a cell matrix according to the head coordinates and tail coordinates of four line segments in the minimum circumscribed rectangular frame matching each of the connected domains; and The cell matrix is ​​marked as a connected region to obtain the vertex coordinates of each cell.

2. The table extraction method according to claim 1, characterized in that: The step of obtaining the head coordinates and tail coordinates of four line segments in the minimum circumscribed rectangular frame matching each of the connected domains includes: According to the coordinates of the pixel points in each connected domain, the vertex coordinates of the minimum circumscribed rectangular box matching each connected domain are obtained; and The head coordinate and the tail coordinate of each line segment are determined according to the vertex coordinates of the minimum circumscribed rectangular frame matching the connected domains and the direction of each line segment in the minimum circumscribed rectangular frame.

3. The table extraction method according to claim 2, characterized in that: Determining the head coordinate and the tail coordinate of each line segment includes: For a horizontal line segment in the minimum circumscribed rectangular frame, the minimum value of the horizontal coordinate of the minimum circumscribed rectangular frame is determined as the head horizontal coordinate, the average value of the vertical coordinate of the upper left corner vertex and the vertical coordinate of the lower left corner vertex of the minimum circumscribed rectangular frame is determined as the head vertical coordinate, the maximum value of the horizontal coordinate of the minimum circumscribed rectangular frame is determined as the tail horizontal coordinate, and the average value of the vertical coordinate of the upper right corner vertex and the vertical coordinate of the lower right corner vertex of the minimum circumscribed rectangular frame is determined as the tail vertical coordinate; and For the vertical line segments in the minimum circumscribed rectangular frame, the minimum value of the vertical coordinate of the minimum circumscribed rectangular frame is determined as the head vertical coordinate, the average value of the horizontal coordinate of the upper left corner vertex and the horizontal coordinate of the upper right corner vertex of the minimum circumscribed rectangular frame is determined as the head horizontal coordinate, the maximum value of the vertical coordinate of the minimum circumscribed rectangular frame is determined as the tail vertical coordinate, and the average value of the horizontal coordinate of the lower left corner vertex and the horizontal coordinate of the lower right corner vertex of the minimum circumscribed rectangular frame is determined as the tail horizontal coordinate.

4. The table extraction method according to claim 3, characterized in that: The processing of the mask image by an image science method further comprises: According to the head ordinate or the tail ordinate of the lateral line segment in the minimum circumscribed rectangular frame matched by each connected domain, the lateral line segments are sorted in ascending order to form a first set; According to the head horizontal coordinate or the tail horizontal coordinate of the vertical line segment in the minimum circumscribed rectangular frame matched by each connected domain, the vertical line segments are sorted in ascending order to form a second set; Determine the line segment in the first set that is closest to the coordinates of the upper left corner vertex of each cell as the start row index of each cell, and determine the line segment in the first set that is closest to the coordinates of the lower right corner vertex of each cell as the end row index of each cell; and The line segment in the second set closest to the coordinates of the upper left corner vertex of each cell is determined as the starting column index of each cell, and the line segment in the second set closest to the coordinates of the lower right corner vertex of each cell is determined as the ending column index of each cell.

5. The table extraction method according to claim 1, characterized in that: Determining the text content in each cell includes: Each cell is traversed, and when the intersection area of ​​the minimum bounding rectangle of the specific text and the specific cell is greater than a critical threshold, the text content of the minimum bounding rectangle of the specific text is determined to be the text content in the specific cell.

6. The table extraction method according to claim 1, characterized in that: The critical threshold is a product of a minimum value of an area of ​​a minimum circumscribed rectangle of the specific text and an area of ​​the specific cell and a preset value.

7. A table extraction system, characterized in that: The table extraction system comprises: A segmentation device, used for segmenting the table image by using an image segmentation model to obtain a mask image marked with line attribute vertical lines at each pixel point; A processing device, used for processing the mask image by an image science method to obtain the vertex coordinates of each cell and the starting row index, starting column index, ending row index and ending column index of each cell; A text recognition device, used for recognizing the table image through a text recognition model to obtain the vertex coordinates of the minimum circumscribed rectangle of each text in the table image and a text recognition model of the text content; A text determination device, used for determining the text content in each cell according to the vertex coordinates of each cell and the vertex coordinates of the minimum circumscribed rectangle of each text; and A generating device is used to generate a table presented in a target file according to the starting row index, starting column index, ending row index and ending column index of each cell and the text content in each cell, The processing of the mask image by an image science method comprises: Marking connected domains of the mask image to obtain each connected domain; According to the coordinates of the pixel points in each connected domain and the direction of each line segment in the minimum circumscribed rectangular frame matching each connected domain, the head coordinates and the tail coordinates of four line segments in the minimum circumscribed rectangular frame matching each connected domain are obtained; Establishing a cell matrix according to the head coordinates and tail coordinates of four line segments in the minimum circumscribed rectangular frame matching each of the connected domains; and Connected domain marking is performed on the cell matrix to obtain vertex coordinates of each cell.

8. A machine-readable storage medium, characterized in that: The machine-readable storage medium stores instructions for causing a machine to execute the table extraction method according to any one of claims 1 to 6.

9. An electronic device, characterized in that: The electronic device comprises: processor; a memory for storing instructions executable by the processor; The processor is used to read the executable instructions from the memory and execute the instructions to implement the table extraction method according to any one of claims 1 to 6.

Images