Table recognition, document acquisition methods and storage media

Abstract

This invention belongs to the field of image processing technology, specifically providing a table recognition and document acquisition method and storage medium, aiming to solve the problem of accurately recognizing tables with different positions and appearance structures in an image. To this end, the method of this invention includes: acquiring information such as text rows, table regions, frame foreground images, and text foreground images from the image of the table to be recognized; obtaining the table frame structure based on the frame foreground images; obtaining the table text structure based on the text foreground images; and then obtaining the recognized table based on the table frame structure, table text structure, and table regions. Through this method, text position assists in table structure extraction, improving the robustness of the recognition method; text row information assists in table text structure extraction, improving the generalization and accuracy of the recognition method; and through the two-layer extraction and combination of frame structure and text structure, it flexibly realizes the extraction of tables with various structures, enhancing the practicality of this invention.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, and specifically provides a method for table recognition, document acquisition, and storage medium. Background Technology

[0002] With the development of the digital age, a large amount of document image data is uploaded daily via electronic devices, leading to a growing demand for the rapid and accurate extraction of document information from raw document images. Representing information in documents using tables is one of the most common and effective methods, with various table structures widely used in finance, administration, research, and other fields. Therefore, automatically recognizing table structures and information is an important research direction for document analysis through image processing.

[0003] However, retrieving table information through image recognition faces numerous challenges. For example, table structures vary; some tables have table lines, while others have none, or only a few. Furthermore, the distribution of table regions within images differs; some tables are located within paragraphs, some occupy entire pages, and others are surrounded by significant background, leaving only a small effective table area. These factors pose substantial challenges to table recognition. Therefore, a robust and accurate method for structured table image recognition in everyday scenarios is needed.

[0004] Accordingly, a new solution is needed in this field to address the aforementioned problems. Summary of the Invention

[0005] The present invention aims to solve the above-mentioned technical problem, namely, how to accurately identify tables with different positions and appearances in an image.

[0006] In a first aspect, the present invention provides a table recognition method, the method comprising:

[0007] Obtain the text line information of the text lines in the table image to be recognized;

[0008] Obtain the table region in the table image to be identified, wherein the table region includes at least one of a table region with multiple borders, a table region with few borders, and a table region without borders;

[0009] Obtain the foreground image of the table border and the foreground image of the table text in the image of the table to be identified;

[0010] Based on the foreground image of the table border, extract the table border structure of the table area;

[0011] Based on the foreground image of the table text, extract the table text structure of the table area;

[0012] Based on the table border structure and the table text structure, and in conjunction with the table area, a fully recognized table is obtained.

[0013] In one embodiment of the above table recognition method, the text line information includes at least the text line position, single character content, and single character position. The method for "obtaining the text line information of the text lines in the table image to be recognized" includes:

[0014] The text row position in the table image to be identified is obtained by using a text detection model;

[0015] Based on the position of the text line, the text line image is cropped from the table image to be recognized;

[0016] The text recognition model is used to obtain the content and position of each character in the text line image.

[0017] In one embodiment of the above table recognition method, the method for "obtaining the table region in the table image to be recognized" includes:

[0018] The table region is obtained by using a trained table region detection model;

[0019] The table region detection model is trained using sample images with completed region annotations, wherein the region annotations include table category and location annotations, the table category includes at least one of multi-bordered table, few-bordered table, and no-bordered table, and the location annotation is the bounding rectangle containing the table region;

[0020] The table region detection model is one of the following: YOLO network, Faster R-CNN network, or SSD network.

[0021] In one embodiment of the above table recognition method, the method of "extracting the table frame structure of the table region based on the table frame foreground image" includes:

[0022] The horizontal and vertical outlines in the foreground image of the table frame are obtained through morphological operations. The point where the horizontal and vertical outlines intersect is called the intersection point. The starting / ending point of the horizontal outline and the starting / ending point of the vertical outline are collectively referred to as the endpoints.

[0023] Obtain the four-way connection index of the intersection point and the endpoint in the direction of the horizontal contour line and the direction of the vertical contour line;

[0024] Based on the horizontal contour line, the vertical contour line, and the table area, a table candidate area is obtained;

[0025] The horizontal and vertical contour lines within the candidate area of ​​the table are sorted to obtain the first horizontal contour line number and the first vertical contour line number.

[0026] Based on the horizontal contour line deviation threshold and the vertical contour line deviation threshold, the first horizontal contour line number and the first vertical contour line number are merged and updated respectively to obtain the second horizontal contour line number and the second vertical contour line number.

[0027] Based on the horizontal contour line, the vertical contour line, the second horizontal contour line number, and the second vertical contour line number, a contour point map is obtained;

[0028] The points in the contour point map are connected according to the four-way connection index to obtain the table frame structure.

[0029] In one embodiment of the above table recognition method, the method for "extracting the table text structure of the table region based on the table text foreground image" includes:

[0030] Based on the position of the text line, determine the table row in the table area;

[0031] Obtain column splitting candidate points in the table rows, including head splitting points and tail splitting points;

[0032] Based on the candidate column splitting points, a virtual column splitting line is obtained, wherein the virtual column splitting line includes a head virtual splitting line and a tail virtual splitting line, the head virtual splitting line is obtained by connecting the head cutting points, and the tail virtual splitting line is obtained by connecting the tail cutting points;

[0033] The credibility of the virtual dividing line of the column is judged to obtain the credible virtual dividing line;

[0034] Based on the trusted virtual dividing line, the virtual wireless table column lines are obtained;

[0035] Based on two adjacent rows of the table, a virtual wireless table row line is obtained;

[0036] The table text structure is obtained based on the virtual wireless table column lines, the virtual wireless table row lines, and the text line information.

[0037] In one embodiment of the above table recognition method, the method for "determining the table row in the table area based on the text line position" includes:

[0038] Obtain the vertical overlap ratio of the first text line and the second text line, where the first text line and the second text line are any two text lines in the table area;

[0039] When the overlap ratio is greater than the overlap ratio threshold, it is determined that the first text line and the second text line belong to the same table row;

[0040] Sort the text lines contained in the table rows and create a text line index.

[0041] In one embodiment of the above table recognition method, the head cutting point further includes the head cutting point at the end of the text line and the head cutting point in the middle of the text line, and the tail cutting point further includes the tail cutting point at the end of the text line and the tail cutting point in the middle of the text line. The method for "obtaining column splitting candidate points in the table row" includes:

[0042] Insert a beginning cutoff point at the left of the starting character of the text line;

[0043] Insert a tail cutoff point at the end of the text line to the right of the ending character of the text line;

[0044] The column segmentation candidate points are inserted into the text line based on the content or position of the single character.

[0045] Sort the column splitting candidate points contained in the table rows and create a column splitting candidate point index;

[0046] The method for "inserting the column segmentation candidate point in the text line based on the content or position of the single character" includes:

[0047] Get the first character spacing between the selected character and the character to its left adjacent to the selected character.

[0048] Get the second character spacing between the selected character and the character adjacent to the right of the selected character.

[0049] When the ratio of the first character spacing to the second character spacing is greater than the character spacing threshold, a middle tail cutoff point and a middle head cutoff point of the text line are inserted to the left of the selected character.

[0050] When the ratio of the first character spacing to the second character spacing is less than the reciprocal of the character spacing threshold, insert the middle end cut point and the middle beginning cut point of the text line to the right of the selected character.

[0051] When the single-word content transitions from a numeric string to a text string, a middle end cutoff point and a middle beginning cutoff point of the text line are inserted sequentially between the numeric string and the text string.

[0052] In one embodiment of the above table recognition method, the head cutting point includes a first head cutting point and a second head cutting point, and the tail cutting point includes a first tail cutting point and a second tail cutting point. The method of "obtaining virtual cutting lines based on the column cutting candidate points" includes:

[0053] Obtain the horizontal distance and vertical distance between the first head cutting point and the second head cutting point. When the horizontal distance between the head cutting points is less than or equal to a first horizontal distance threshold and the vertical distance between the head cutting points is less than or equal to a first vertical distance threshold, connect the first head cutting point and the second head cutting point with a straight line to obtain the virtual head cutting line.

[0054] Obtain the horizontal distance and vertical distance between the first tail cutting point and the second tail cutting point. When the horizontal distance between the tail cutting points is less than or equal to a second horizontal distance threshold and the vertical distance between the tail cutting points is less than or equal to a second vertical distance threshold, connect the first tail cutting point and the second tail cutting point with a straight line to obtain the tail virtual cutting line.

[0055] In one embodiment of the above table recognition method, the method for "judging the credibility of the column virtual dividing line to obtain a credible virtual dividing line" includes:

[0056] Perform linear fitting on the candidate points of the column on the virtual dividing line to obtain the virtual dividing fitting line;

[0057] Obtain the fitting position deviation between the candidate points of the column segmentation on the virtual segmentation line and the fitting line of the virtual segmentation;

[0058] Obtain the first proportion of the column splitting candidate points whose fitting position deviation is less than or equal to the fitting position deviation threshold among all the column splitting candidate points on the column virtual splitting line;

[0059] When the first ratio is greater than or equal to the first confidence threshold, the column virtual dividing line is the confidence virtual dividing line;

[0060] When the first proportion is less than the first confidence threshold, obtain the second proportion of the text line end head cut point or the text line end tail cut point among all the column segmentation candidate points on the column virtual segmentation line;

[0061] When the second ratio is greater than or equal to the second confidence threshold, the column virtual dividing line is the confidence virtual dividing line.

[0062] In one embodiment of the above table recognition method, the trusted virtual segmentation line includes a trusted head virtual segmentation line and a trusted tail virtual segmentation line. The trusted head virtual segmentation line is the trusted virtual segmentation line corresponding to the head virtual segmentation line, and the trusted tail virtual segmentation line is the trusted virtual segmentation line corresponding to the tail virtual segmentation line. The method for "obtaining virtual wireless table column lines based on the trusted virtual segmentation lines" includes:

[0063] Sort the trusted virtual dividing lines;

[0064] The virtual dividing line of the trusted header with consecutive serial numbers is the starting line of the table column;

[0065] The credible tail virtual dividing line for consecutive merged sequence numbers is the table column termination line;

[0066] The adjacent end lines and start lines of the table columns are merged to obtain the virtual wireless table column lines, wherein the end lines of the table columns are located to the left of the start lines of the table columns.

[0067] In one embodiment of the above table recognition method, the method of "obtaining the table text structure based on the virtual wireless table column lines, the virtual wireless table row lines, and the text row information" includes:

[0068] The virtual wireless table is obtained based on the column lines and row lines of the virtual wireless table.

[0069] Based on the single character content and position in the text line information, the characters in the text line are placed one by one into the virtual wireless cells of the virtual wireless table to obtain the table text structure.

[0070] In one embodiment of the above table recognition method, the method of "obtaining the recognized table based on the table border structure and the table text structure, and in combination with the table area" includes:

[0071] When the table area is the multi-border table area, the text line is added based on the table border structure;

[0072] When the table area is the table area with few borders, obtain the correspondence between each virtual wireless table row and the text row in the table text structure, and fill the text row corresponding to the virtual wireless table row into the table border structure row by row;

[0073] When the table area is a borderless table area, the table text structure is directly added to the table border structure.

[0074] In a second aspect, the present invention proposes a document acquisition method, the method comprising:

[0075] Obtain the image of the document to be processed;

[0076] According to the table recognition method described in any of the above schemes, the table and text lines in the document image to be processed are obtained, and the number of the table is one or more;

[0077] The document that has been identified is obtained based on the table and the text lines outside the table area.

[0078] In a third aspect, the present invention provides a storage medium adapted to store a plurality of program codes, the program codes being adapted to be loaded and run by a processor to perform the table recognition method described in any of the preceding schemes, or the document acquisition method described in any of the preceding schemes.

[0079] By employing the above technical solution, this invention acquires information such as text rows, table regions, table border foreground images, and table text foreground images from the image of the table to be recognized. Based on the table border foreground images, it obtains the table border structure; based on the table text foreground images, it obtains the table text structure; and then, based on the table border structure and table text structure, combined with the table regions, it obtains the recognized table. Through this method, text location assists in table structure extraction, improving the robustness of the recognition method; text recognition information assists in table text structure extraction, improving the generalization and accuracy of the recognition method; and through the two-layer extraction and combination of table border structure and table text structure, it flexibly achieves the extraction of various table structures, not only improving the accuracy of the method but also enhancing its practicality. Attached Figure Description

[0080] The preferred embodiments of the present invention are described below with reference to the accompanying drawings, in which:

[0081] Figure 1 This is a flowchart of the main steps of the table recognition method according to an embodiment of the present invention.

[0082] Figure 2 This is a schematic diagram of the table categories in an embodiment of the present invention.

[0083] Figure 3 This is an embodiment of the present invention. Figure 1 Detailed flowchart of step S104.

[0084] Figure 4 This is an embodiment of the present invention. Figure 1 Detailed flowchart of step S105. Detailed Implementation

[0085] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0086] Those skilled in the art should understand that these embodiments are merely illustrative of the technical principles of the present invention and are not intended to limit the scope of protection of the present invention. Those skilled in the art can make adjustments as needed to adapt to specific application scenarios.

[0087] First read Figure 1 , Figure 1 This is a flowchart illustrating the main steps of the table recognition method according to an embodiment of the present invention. The table recognition method according to an embodiment of the present invention includes:

[0088] Step S101: Obtain the text line information of the text lines in the table image to be recognized;

[0089] Step S102: Obtain the table region in the image of the table to be recognized;

[0090] Step S103: Obtain the foreground image of the table frame and the foreground image of the table text in the image of the table to be recognized;

[0091] Step S104: Extract the table border structure of the table area based on the table border foreground image;

[0092] Step S105: Extract the table text structure of the table area based on the table text foreground image;

[0093] Step S106: Based on the table border structure and table text structure, and combined with the table area, obtain the recognized table.

[0094] In this embodiment of the invention, images of documents or receipts containing tables, to be identified, are acquired using image acquisition devices such as document scanners or high-speed scanners.

[0095] In step S101, the text line information of the text lines in the table image to be identified is obtained, wherein the text line information includes the text line position, the content of a single character, and the position of a single character.

[0096] First, the text line positions in the table image to be recognized are obtained using a text detection model. As an example, multiple adjacent words constitute a text line, and there are one or more text lines. The text line position can be represented by a quadrilateral, and the position of each text line is recorded by the coordinates of the four vertices of the quadrilateral in the table image to be recognized.

[0097] The table image to be recognized is cropped according to the position of the text line to obtain the text line image corresponding to each text line.

[0098] By using a character recognition model to recognize characters in a text line image, the content of each character in the corresponding text line image and the position of each character in the text line image are obtained. Then, based on the text line position in the table image to be recognized, the position of each character in the text line image is converted to its position in the table image using coordinates, resulting in the final position of each character. The position of each character can be represented by the pixel coordinates of its center.

[0099] It should be noted that all coordinates involved in this invention are coordinates in the pixel coordinate system. As an example, the top-left vertex of the image is selected as the origin of the pixel coordinate system, and the pixel spacing in the horizontal / vertical direction is used as the unit distance in the horizontal / vertical direction of the pixel coordinate system. Furthermore, according to application conventions, the horizontal direction will be referred to as the lateral direction and the vertical direction as the longitudinal direction in this specification.

[0100] It should be noted that the method for text detection in images is not limited in this invention. As an example, the CTPN model, SegLink model, etc. can be used, or a text detection model can be built based on the VGG16 network, RPN network, etc.

[0101] The present invention does not limit the method of text recognition in images. As an example, it can be implemented using OCR software, and those skilled in the art can also use other methods.

[0102] In this invention, tables are classified according to the characteristics of their horizontal and vertical borders. The table categories include tables with borders and tables without borders. Tables with borders can be further divided into tables with multiple borders and tables with few borders.

[0103] like Figure 2 (a) shows a multi-bordered table, which has real, identifiable horizontal and vertical borders, and each cell of the table is divided by a solid line.

[0104] like Figure 2 (b) shows a table with few borders. This type of table contains both horizontal and vertical borders, but has fewer cells with both horizontal and vertical borders. Without considering the case where multiple lines of text in a large cell need to be split into multiple cells using a table without borders, a table with few borders is equivalent to a table with many borders.

[0105] like Figure 2(c) shows a frameless table. This type of table does not have complete horizontal and vertical borders. That is, its cells do not contain at least one horizontal and one vertical border at the same time. Its cells are divided according to the arrangement of its content. Generally, a row of information is aligned horizontally at the same height, and a column of information is aligned vertically.

[0106] In step S102, the table region in the table image to be identified is obtained through a table region detection model. The recognition result output by the table region detection model includes the table location and table category of the table region. The table location can be represented by a rectangle, and depending on the table image to be identified, the table category in the recognition result includes at least one of the following: table with multiple borders, table with few borders, and table without borders.

[0107] Accordingly, depending on the table type, the table area in step S102 includes at least one table area with multiple borders, a table area with few borders, and a table area without borders, and the table area with multiple borders and the table area with few borders can be collectively referred to as a table with borders.

[0108] As an example, the table region detection model can use YOLO series models such as YOLO V3, YOLO V4 or YOLO V5, or other neural networks such as Faster R-CNN network or SSD network.

[0109] It should be noted that the table region detection model in step S102 is a model that has already been trained using table sample images. In the table sample images, the table regions can be manually labeled. The labeling should include at least the table category and location. The table category includes at least one of the following: multi-bordered table, few-bordered table, and no-bordered table. The location label can be made using a rectangle, where the rectangle is the smallest bounding rectangle that can contain the table content.

[0110] In this invention, the table region and category are detected based on learning and training, which is easy to optimize. By defining the table region of various table categories, the interference of external table interference on the subsequent extraction of table frame structure and table text structure can be reduced.

[0111] The table line foreground image refers to the image that separates the table lines from the image of the table to be identified, and shows the position of the table lines in the image of the table to be identified. The table line foreground image only shows whether the table lines exist and their positions, and is not related to the actual thickness of the table lines or whether they are broken.

[0112] A table text foreground image refers to an image that isolates the text from the table image to be recognized, representing the position of the text in the table image. The table text foreground image only indicates whether the text exists and its position, and is unrelated to the actual content of the text.

[0113] In step S103, the foreground image of the table frame lines and the foreground image of the table text in the image to be recognized are obtained. The method for obtaining the foreground image of the table frame lines and the foreground image of the table text in the image to be recognized is not limited in this invention. As an example, it can be implemented by referring to the method described in paragraphs 0057 to 0067 of the specification of Chinese Patent Application Publication No. CN112528813A, "Table Recognition Method, Apparatus and Computer-Readable Storage Medium," the specific steps of which include: using a Unet neural network model comprising 4 downsampling convolutional layers and 4 upsampling convolutional layers as the image recognition model, and this image recognition model is trained based on table image samples and the corresponding table line foreground label image and text foreground label image; the table line foreground label image and... The table image samples are all the same size. The label value stored at the position of each pixel in the table line foreground label image depends on whether there is a table line at the corresponding position in the table image sample. The text foreground label image is the same size as the table image samples. The label value stored at the position of each pixel in the text foreground label image depends on whether there is a text line at the corresponding position in the table image sample. For example, if there is a table line at a certain position in the table image sample, the label value stored at the corresponding position in the table line foreground label image is 1, otherwise it is 0. Similarly, if there is a text line at a certain position in the table image sample, the label value stored at the corresponding position in the text foreground label image is 1, otherwise it is 0. During training, the loss value of the image recognition model is calculated using the loss function L shown in Equation (1) based on the table image samples and the corresponding table line foreground label image and text foreground label image.

[0114] (1)

[0115] In equation (1), This indicates the number of image samples in the table. =1, 2, 3, ... ; and These represent the height and width of the table image sample, respectively. =1, 2, 3, ... , =1, 2, 3, ... ; Indicates according to the first The table line foreground label image and the text foreground label image corresponding to the table image sample are determined in the first... Location in a table of image samples The label value at the location; The output of the image recognition model is in the first position. Location in a table of image samples The predicted label value is calculated at the specified location. The gradient of each model parameter in the image recognition model is calculated based on the loss value, and the model parameters are updated using backpropagation based on the gradients to optimize the model and complete the training.

[0116] Next, combine Figure 3 Step S104 is explained. In this embodiment, as... Figure 3 As shown, Figure 3 (a) is an example table in the image of the table to be identified. Figure 3 (b) is the result obtained after step S103. Figure 3 (a) is a schematic diagram of the foreground image of the table frame corresponding to the example table in the image to be identified.

[0117] First, morphological operations are used to obtain the horizontal and vertical outlines of the table frame in the foreground image. Figure 3 (c) is Figure 3 (b) is a schematic diagram of the horizontal outline corresponding to the foreground image of the table frame. Figure 3 (d) is Figure 3 (b) is a schematic diagram of the vertical outline corresponding to the foreground image of the table frame.

[0118] The point where the horizontal and vertical contour lines intersect is called the intersection point, such as... Figure 3 The position of circular point 1 shown in (e).

[0119] The starting and / or ending points (leftmost and / or rightmost) of the horizontal contour line, which do not intersect with the vertical contour line, and / or the starting and / or ending points (topmost and / or bottommost) of the vertical contour line, which do not intersect with the horizontal contour line, are collectively referred to as endpoints, such as... Figure 3 The position of the cross line 2 shown in (e).

[0120] Record the position of each intersection and endpoint, and based on the horizontal and / or vertical contour lines, record the four-way index of each intersection and endpoint, i.e., the connection relationship of the points to the left and / or right of the horizontal contour line adjacent to the point, and / or the connection relationship of the points above or below the vertical contour line adjacent to the point.

[0121] As an example, such as Figure 3 As shown in (e), intersection point p1 has only two-way indices (below and right), intersection point p2 has three-way indices (above, below, and left), intersection point p3 has four-way indices (above, below, left, and right), and endpoint p4 has only a left-side index.

[0122] Based on the positional relationship between the horizontal and vertical contour lines and the table area obtained in step S102, the horizontal and vertical contour lines are assigned to the table areas that overlap with them, thus obtaining the table candidate areas.

[0123] In other words, a table candidate region is a table region that contains horizontal and / or vertical outlines. Next, the table structure within the table candidate region (i.e., the table region) will be extracted as a unit.

[0124] The horizontal contour lines within the candidate area of ​​the table are numbered from left to right to obtain the first horizontal contour line number; the vertical contour lines within the candidate area of ​​the table are numbered from top to bottom to obtain the first vertical contour line number. At this point, each non-connected horizontal and / or vertical contour line segment has a different number.

[0125] Based on the first horizontal contour line number and the first vertical contour line number, the numbers of two or more horizontal contour lines and / or vertical contour lines that are not actually connected in the candidate area of ​​the table, but are approximately on the same straight line after being connected, are merged and updated.

[0126] Preferably, the first horizontal contour line number and the first vertical contour line number can be merged and updated according to the horizontal contour line deviation threshold and the vertical contour line deviation threshold, respectively, to obtain the second horizontal contour line number and the second vertical contour line number.

[0127] As an example, the average of the vertical coordinates of the leftmost and rightmost positions of the horizontal contour line is selected as the vertical reference coordinate of the horizontal contour line. The difference between the vertical reference coordinates of the two horizontal contour lines is calculated and compared with the horizontal contour line deviation threshold. If the difference between the vertical reference coordinates of the two horizontal contour lines is less than or equal to the horizontal contour line deviation threshold, then these two horizontal contour lines can be merged and assigned the same sequence number. After all horizontal contour lines have been checked to see if they can be merged, the sequence numbers are updated again from top to bottom to obtain the sequence number of the second horizontal contour line.

[0128] Using the same method, the average of the lateral coordinates of the top and bottom positions of the vertical contour line is selected as the lateral reference coordinate of that vertical contour line. The difference between the lateral reference coordinates of the two vertical contour lines is calculated and compared with the vertical contour line deviation threshold. If the difference between the lateral reference coordinates of the two vertical contour lines is less than or equal to the vertical contour line deviation threshold, then these two vertical contour lines can be merged and assigned the same sequence number. After all vertical contour lines have been checked to see if they can be merged, the sequence numbers are updated again from left to right to obtain the sequence number of the second vertical contour line.

[0129] As an example, both the horizontal contour line deviation threshold and the vertical contour line deviation threshold can be set to 5 pixels. Those skilled in the art can also set the horizontal contour line deviation threshold and the vertical contour line deviation threshold to other same or different values ​​according to the actual situation.

[0130] Figure 3 (f) is a schematic diagram of the second horizontal contour line number and the second vertical contour line number after merging and updating. Figure 3 (f) It can be seen that the second horizontal contour line number of the two horizontal contour lines is H2, the second horizontal contour line number of the two horizontal contour lines is H3, and the second vertical contour line number of the two vertical contour lines is V6.

[0131] As can be seen from the above methods for numbering, merging, and updating the horizontal and vertical contour line numbers, the first horizontal contour line number is assigned to each horizontal contour line without considering positional relationships, and the initial number of horizontal contour lines is hctr_num; similarly, the first vertical contour line number is assigned to each vertical contour line without considering positional relationships, and the initial number of vertical contour lines is vctr_num.

[0132] After merging and updating the sequence numbers, the number of horizontal contours corresponding to the second horizontal contour line number is sort_hctr_num, and sort_hctr_num <= hctr_num; the number of vertical contours corresponding to the second vertical contour line number is sort_vctr_num, and sort_vctr_num <= vctr_num, as shown below. Figure 3 (f) sort_hctr_num is 9, sort_vctr_num is 7.

[0133] Based on the intersection and endpoints of the horizontal and vertical contour lines, the sequence number of the second horizontal contour line, and the sequence number of the second vertical contour line, a contour point map is obtained, where the height of the contour point map is sort_hctr_num+2 and the width is sort_vctr_num+2.

[0134] It should be noted that the height and width of the contour point plot are increased by 2 each to reserve space for recording the two ends of the contour. For both horizontal and vertical contours, there may be isolated points (such as the endpoints mentioned earlier) where the boundary positions of the current dimension contour line do not intersect with the contour line of another dimension. These isolated points only have the index of the current contour line and not the index of the contour line of the other dimension. Figure 3 In (e), endpoint p4 is used Figure 3 In the contour point diagram represented by the contour line number in (f), there is only the horizontal contour line number H0, but no vertical contour line.

[0135] Based on the contour point map and the four-way index of each point (intersection or endpoint), cells are formed by connecting lines in a top-to-bottom and left-to-right order, and the collection of all cells constitutes the table frame structure.

[0136] In the table frame structure, the row and column start and end outlines of each cell correspond to different second horizontal outline numbers and second vertical outline numbers. Therefore, the second horizontal outline number and second vertical outline number corresponding to each cell can also be called the row and column start and end index of the cell.

[0137] Next, combine Figure 4 The following describes the implementation process of step S105. In step S105, the text content belonging to a certain table area is determined based on the text line position of the text line obtained in step S101 and the table position of the table area obtained in step S102.

[0138] In step S1051, the vertical overlap ratio of the first and second text lines within the table area is calculated. For example, the overlap ratio can be defined as the ratio of the length of the overlapping portion of the two text lines in the vertical direction to the row height occupied by the two text lines. When the overlap ratio is greater than an overlap ratio threshold (e.g., 60%), it is determined that the two text lines belong to the same table row (a table row can contain text lines from multiple cells in the same row). The first and second text lines are any two text lines within the table area.

[0139] The top and bottom coordinates of a table row are determined by the positions of the text lines contained within it. For example, the maximum and minimum vertical values ​​of the rectangles containing the positions of all text lines in the table row can be used as the top and bottom coordinates of the table row.

[0140] By introducing the positional relationship of multiple text lines to assist in determining table rows, the anti-interference ability of table structure extraction can be improved, thus making the method of the present invention very robust.

[0141] Examine all text lines within a table range to obtain all table rows within the table range. Each table row contains one or more text lines.

[0142] In step S1052, the table rows are sorted from top to bottom to obtain the table row numbers, and a text line index is created for the text lines in each table row in order from left to right. The text line data corresponding to the text line index includes: text line position, table row number, etc.

[0143] Based on the text line index, add column splitting candidate points in the order of table row numbers from top to bottom and each row from left to right.

[0144] Candidate split points are divided into two categories: head split points and tail split points. The head split point is denoted as headsplit, and the tail split point is denoted as tailsplit. The head split point is located at the beginning of the string in the same column of the same table row, and the tail split point is located at the end of the string in the same column of the same table row.

[0145] Accordingly, a head cutoff point is generated to the left of the starting character (first word) of the text line, and a tail cutoff point is generated to the right of the ending character (last word) of the text line.

[0146] In some tables, multiple strings belonging to different columns within the same row are too close together, causing the single text line detected in step S101 to actually be composed of multiple strings belonging to different columns joined together. Therefore, preferably, the text line is checked based on the content or position of individual characters in the text line information to see if the text line can be further distinguished as belonging to different columns, and column splitting candidate points are added. These column splitting candidate points can be referred to as the beginning and end of the text line.

[0147] In one embodiment, excluding the start and end characters of the text line, for each other word in the text line, the word distance between the word and the word to its left adjacent to the word is calculated to obtain the first word spacing; the word distance between the word and the word to its right adjacent to the word is calculated to obtain the second word spacing.

[0148] When the ratio of the first character spacing to the second character spacing is greater than the character spacing threshold, a text line end cutoff point and a text line beginning cutoff point are inserted to the left of the character, with the text line beginning cutoff point being close to the character.

[0149] When the ratio of the first character spacing to the second character spacing is less than the reciprocal of the character spacing threshold, a text line end cutoff point and a text line beginning cutoff point are inserted to the right of the character, with the text line end cutoff point being close to the character.

[0150] As an example, let ABC be three consecutive characters in a line of text. Calculate the first character spacing between AB. The second single-word spacing between , and BC The single-character spacing threshold is set to 1.5.

[0151] when At this point, insert a text line ending point and a text line beginning point to the left of B (between A and B), with the text line beginning point close to B.

[0152] when When this occurs, insert a text line middle end tangent point and a text line middle start tangent point between B and C on the right side of A, with the text line middle end tangent point close to B.

[0153] In another embodiment, when the single-word content in a text line transitions from a numeric string to a text string, insert a text line middle end tangent point and a text line middle start tangent point between the numeric string and the text string, where the text line middle end tangent point is close to the literal character.

[0154] As an example, the text line content includes "10,000,00 Bank of China", where "10,000,00" is a numeric string and "Bank of China" is a literal string. Insert a text line middle end tangent point and a text line middle start tangent point to the left of the single word "中", with the text line middle end tangent point close to "中".

[0155] By processing the text line middle end tangent point and the text line middle start tangent point in the text line, the deficiencies of the text line detection model in step S101 can be remedied. For example, the cells containing numbers in a table are usually set to right alignment. When the cell immediately adjacent to the right of this cell contains left-aligned text content; or the text content in two adjacent cells already fills the same row of the cell, especially for a borderless table, sometimes the text detection model will recognize the different cell contents in the same table row in two or more cells as one text line.

[0156] Therefore, the method for assisting in the extraction of the table text structure using the text recognition information of the present invention (i.e., the introduction of single-word content and single-word position information) improves the ability to divide the columns in a borderless table that are relatively close or approximately adhered to each other during the extraction of the table text structure, thereby making the method of the present invention have good generalization, and at the same time further improving the accuracy of table recognition.

[0157] After obtaining the column segmentation candidate points in all the table rows of the table area, in step S1053, sort and number the column segmentation candidate points in the order from top to bottom of the table row and from left to right within the table row, so as to establish a column segmentation candidate point index for each column segmentation candidate point.

[0158] The column segmentation candidate point index of each column segmentation candidate point at least includes the following data: column segmentation candidate point number, column segmentation candidate point position, column segmentation candidate point type, left adjacent segmentation candidate point number, right adjacent segmentation candidate point number.

[0159] It should be noted that the specific rules for numbering the candidate points of column segmentation are not limited in this invention. As an example, all candidate points of column segmentation can be uniformly numbered; or a two-level numbering system can be used, where the first level number is the table row number and the second level number is the sorted number of the candidate points of column segmentation within that table row. Those skilled in the art can also use other suitable numbering methods according to the actual situation.

[0160] Following a top-down order, based on the column splitting candidate point index, virtual splitting points in the same column are searched for each column splitting candidate point. Furthermore, for head splitting points, only other head splitting points are searched, meaning that the virtual splitting points in the same column of a certain head splitting point only contain the head splitting point; for tail splitting points, only other tail splitting points are searched, meaning that the virtual splitting points in the same column of a certain tail splitting point only contain the tail splitting point.

[0161] It should be noted that although the head cut point can be further subdivided into the head cut point at the end of the text line and the head cut point in the middle of the text line, and the tail cut point can be further subdivided into the tail cut point at the end of the text line and the tail cut point in the middle of the text line, when searching for virtual cut points in the same column, only the head cut point or the tail cut point is used to search for cut points of the same type.

[0162] As an example, obtain the horizontal and vertical distances between the first tangent point of the current row and the second tangent point of the next row. When the horizontal distance between the tangent points is less than or equal to a first horizontal distance threshold, and the vertical distance between the tangent points is less than or equal to a first vertical distance threshold, these two tangent points are virtual tangent points in the same column of the same group. Connect the first and second tangent points with a straight line to obtain the column virtual tangent line determined by the first and second tangent points.

[0163] Using the same method, continue searching downwards for all virtual dividing points in the same column that belong to the same group as the two head dividing points mentioned above, and connect each pair of virtual dividing points in the same column with a straight line to obtain the column virtual dividing line corresponding to the virtual dividing points in the same column of that group. Since all virtual dividing points in the same column of this group are head dividing points, this type of column virtual dividing line can be called a head virtual dividing line.

[0164] Obtain the horizontal and vertical distances between the first tail tangent point of the current row and the second tail tangent point of the next row. When the horizontal distance between the tail tangent points is less than or equal to a second horizontal distance threshold, and the vertical distance is less than or equal to a second vertical distance threshold, these two tail tangent points are considered virtual split points in the same column of the same group. Connect the first and second tail tangent points with a straight line to obtain the column virtual split line determined by the first and second tail tangent points.

[0165] Using the same method, continue searching downwards for all virtual dividing points in the same column that belong to the same group as the two tail dividing points mentioned above, and connect each pair of virtual dividing points in the same column with a straight line to obtain the column virtual dividing line corresponding to the virtual dividing points in the same column of that group. Since all virtual dividing points in the same column of this group are tail dividing points, this type of column virtual dividing line can be called a tail virtual dividing line.

[0166] As can be seen from the above, a column virtual dividing line is composed of one or more connected straight segments. Furthermore, a column virtual dividing line can be further subdivided into a head virtual dividing line and a tail virtual dividing line. The head virtual dividing line is formed by connecting the head cutting points and represents the left starting line of the table column; the tail virtual dividing line is formed by connecting the tail cutting points and represents the right ending line of the table column.

[0167] As an example, both the first horizontal distance threshold and the second horizontal distance threshold are set to 12 pixels, and both the first vertical distance threshold and the second vertical distance threshold are set to 4 times the average line height of the text lines. Those skilled in the art may also set the first horizontal distance threshold and the second horizontal distance threshold to other identical or different values, and the first vertical distance threshold and the second vertical distance threshold to other identical or different values, depending on the actual situation.

[0168] Next, in step S1054, the reliability of the virtual column dividing lines is checked to determine the actual table column lines. First, a straight line is fitted based on the candidate column dividing points on the virtual dividing lines to obtain the virtual dividing fitted line.

[0169] Calculate the vertical distance from each candidate point on the virtual dividing line to the virtual dividing line, which is the deviation of the candidate point from the virtual dividing line in terms of its fitting position.

[0170] The first proportion of column splitting candidate points on the virtual splitting line of a column whose fitted position deviation is less than or equal to the fitted position deviation threshold.

[0171] When the first proportion is greater than or equal to the first confidence threshold, the virtual dividing line of the column is determined to be a reliable virtual dividing line, and the virtual dividing line of the column is retained.

[0172] When the first proportion is less than the first confidence threshold, the second proportion of the virtual segmentation line belonging to the beginning or end of the text line in the column is counted among all column segmentation candidate points.

[0173] If the second proportion is greater than or equal to the second confidence threshold, the virtual dividing line of that column is determined to be a reliable virtual dividing line and is retained. If the second proportion is less than the second confidence threshold, the virtual dividing line of that column is determined to be an unreliable virtual dividing line and is deleted.

[0174] As an example, the fitting position deviation threshold is set to 6 pixels, and both the first and second confidence thresholds are set to 60%. Those skilled in the art can also set the values ​​of the fitting position deviation threshold, the first confidence threshold, and the second confidence threshold according to the actual situation, and the first and second confidence thresholds can also be set to different values.

[0175] Perform a confidence check on all column virtual dividing lines within the table area to obtain the retained reliable virtual dividing lines. Sort all reliable virtual dividing lines within the table area from left to right, ensuring that within the same table row, the reliable virtual dividing line containing the column dividing candidate point on the right is to the right of the reliable virtual dividing line containing the column dividing candidate point on the left.

[0176] As mentioned above, the column virtual split line includes the head virtual split line and the trusted tail virtual split line. Accordingly, the trusted virtual split line can be further subdivided into the trusted head virtual split line and the trusted tail virtual split line. The trusted head virtual split line is the trusted virtual split line corresponding to the head virtual split line, and the trusted tail virtual split line is the trusted virtual split line corresponding to the tail virtual split line.

[0177] In step S1055, the consecutively numbered trusted virtual segmentation lines of the header are merged to obtain the table column start line; the consecutively numbered trusted virtual segmentation lines of the tail are merged to obtain the table column end line. After merging all trusted virtual segmentation lines, a set of table column start lines and table column end lines arranged at intervals are obtained.

[0178] Next, merge the adjacent table column end lines and table column start lines to obtain virtual wireless table column lines. When merging the table column end lines and table column start lines, the condition that the table column end line is to the left of the table column start line must be met.

[0179] In step S1056, based on the previously obtained table rows, the top coordinates of the upper table row and the bottom coordinates of the lower table row in two adjacent table rows are known. The average of the top and bottom coordinates is used as the position of the virtual wireless table row line. All table rows are traversed in order from top to bottom or from bottom to top to obtain all virtual wireless table row lines.

[0180] In step S1057, a virtual wireless table is obtained from the column lines and row lines of the virtual wireless table. Based on the position and content of the characters in the text line information obtained in step S101, the characters in the text lines of the table area are placed one by one into the virtual wireless cells of the corresponding virtual wireless table to obtain the table text structure.

[0181] Continue with step S106. Based on the table frame structure and table text structure, and combined with the table area, obtain the recognized table.

[0182] When the table area is a multi-bordered table area, the individual characters in the table text structure (or text lines) are placed into the corresponding cells of the table border structure according to their positions. This process is repeated character by character in the table border structure of the multi-bordered table to obtain a fully recognized table.

[0183] When the table area is a table area with few borders, firstly, based on the table text structure obtained in step S105, establish the correspondence between each virtual wireless table row (i.e., the table row in the virtual wireless table) in the table text structure and the text row obtained in step S101. Then, based on the corresponding position of the virtual wireless table row in the table border structure, add the text row to the table border structure of the table with few borders one by one to obtain the table with complete recognition.

[0184] When the table area is a borderless table area, the table text structure is directly added to the border structure of the borderless table to obtain the recognized table.

[0185] It should be noted that after adding text to a table with borders, it is usually necessary to fine-tune the text position within the table, such as centering the text vertically in each row and left-right aligning the text within a cell. As an example, multi-border tables use the cell outline as the alignment reference, while multi-border tables use the top and bottom center positions of the virtual wireless cell rows as the alignment reference. Other methods for adjusting text position can be adopted depending on the specific circumstances.

[0186] By combining table recognition methods based on table category, table border structure, and table text structure, the table recognition results can be modified and adjusted during the combination process according to the characteristics of different table categories. This allows the method of the present invention to flexibly extract table structures of various categories, and has the characteristics of high practicality.

[0187] It should be noted that after obtaining the original image through the image acquisition device, the original image usually needs to undergo image preprocessing to obtain a table image that meets the input requirements of neural network models such as text detection models and table region detection models. The method of image preprocessing is not limited in this invention; those skilled in the art can use one or more of the following methods, such as rotation, stretching, normalization, and size transformation, depending on the actual situation. The above-mentioned image preprocessing methods are all well-known to those skilled in the art and will not be described in detail in this invention.

[0188] Furthermore, the present invention also provides a document acquisition method. The method involves acquiring images of the document to be processed using devices such as document scanners or high-speed scanners. For example, the document is a file or invoice containing tables and text content.

[0189] According to the table recognition method in the above method embodiment, tables and text lines in the document image to be processed are obtained, wherein the number of tables is one or more.

[0190] For text in the document image that is not in a table, each character is collected in order from top to bottom and from left to right based on the text information of the text line (text line position, single character content, and single character position), and combined with the recognized table to generate the document corresponding to the document image to be processed, thereby realizing the extraction of text information and table structure in the entire document image to be processed.

[0191] Furthermore, the present invention also provides a storage medium that can be configured to store a program for executing the table recognition method and document acquisition method of the above-described method embodiments. This program can be loaded and run by a processor to implement the table recognition method and document acquisition method. For ease of explanation, only the parts related to the embodiments of the present invention are shown; for specific technical details not disclosed, please refer to the method section of the embodiments of the present invention. The storage medium can be a storage device comprising various electronic devices. Optionally, in the embodiments of the present invention, the storage medium is a non-transitory read / write storage medium.

[0192] Those skilled in the art will recognize that the method steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of electronic hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in electronic hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the invention.

[0193] It should be noted that the terms "first," "second," and other ordinal numbers in the specification, claims, and accompanying drawings of this invention are used only to distinguish similar objects and are not intended to describe or indicate a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein.

[0194] Additionally, in the description of this application, the term "A and / or B" means all possible combinations of A and B, such as just A, just B, or A and B.

[0195] The technical solution of the present invention has been described above with reference to the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after such changes or substitutions will all fall within the scope of protection of the present invention.

Claims

1. A table recognition method, characterized in that, The method includes: Obtain text line information from the text lines in the table image to be recognized, wherein the text line information includes text line position, single word content, and single word position; Obtain the table region in the table image to be identified, wherein the table region includes at least one of a table region with multiple borders, a table region with few borders, and a table region without borders; Obtain the foreground image of the table border and the foreground image of the table text in the image of the table to be identified; Based on the foreground image of the table border, extract the table border structure of the table area; Based on the foreground image of the table text, extract the table text structure of the table area; Based on the table border structure and the table text structure, and in combination with the table area, the recognized table is obtained; The method for "extracting the table text structure of the table region based on the table text foreground image" includes: Based on the position of the text line, determine the table row in the table area; Obtain column splitting candidate points in the table row. The column splitting candidate points include head splitting point and tail splitting point, wherein the head splitting point is located at the beginning position of the string in the same table column in the same table row, and the tail splitting point is located at the end position of the string in the same table column in the same table row. Based on the column segmentation candidate points, a column virtual segmentation line is obtained, wherein the column virtual segmentation line includes a head virtual segmentation line and a tail virtual segmentation line; The credibility of the virtual dividing line of the column is judged to obtain the credible virtual dividing line; Based on the trusted virtual dividing line, the virtual wireless table column lines are obtained; The virtual wireless table row lines are obtained by averaging the top coordinates of the uppermost table row and the bottom coordinates of the lowermost table row in two adjacent table rows. The table text structure is obtained based on the virtual wireless table column lines, the virtual wireless table row lines, and the text line information; The head cutting point includes a first head cutting point and a second head cutting point, and the tail cutting point includes a first tail cutting point and a second tail cutting point. The method for "obtaining the column virtual cutting line based on the column cutting candidate points" includes: Obtain the horizontal distance and vertical distance between the first head cutting point and the second head cutting point. When the horizontal distance between the head cutting points is less than or equal to a first horizontal distance threshold and the vertical distance between the head cutting points is less than or equal to a first vertical distance threshold, connect the first head cutting point and the second head cutting point with a straight line to obtain the virtual head cutting line. Obtain the horizontal distance and vertical distance between the first tail cutting point and the second tail cutting point. When the horizontal distance between the tail cutting points is less than or equal to a second horizontal distance threshold and the vertical distance between the tail cutting points is less than or equal to a second vertical distance threshold, connect the first tail cutting point and the second tail cutting point with a straight line to obtain the tail virtual cutting line. The method for "determining the credibility of the virtual dividing line of the column to obtain a credible virtual dividing line" includes: Perform linear fitting on the candidate points of the column on the virtual dividing line to obtain the virtual dividing fitting line; Obtain the fitting position deviation between the candidate points of the column segmentation on the virtual segmentation line and the fitting line of the virtual segmentation; Obtain the first proportion of the column splitting candidate points whose fitting position deviation is less than or equal to the fitting position deviation threshold among all the column splitting candidate points on the column virtual splitting line; When the first ratio is greater than or equal to the first confidence threshold, the column virtual dividing line is the confidence virtual dividing line; The trusted virtual segmentation line includes a trusted header virtual segmentation line and a trusted tail virtual segmentation line. The trusted header virtual segmentation line is the trusted virtual segmentation line corresponding to the header virtual segmentation line, and the trusted tail virtual segmentation line is the trusted virtual segmentation line corresponding to the tail virtual segmentation line. The method for "obtaining virtual wireless table column lines based on the trusted virtual segmentation lines" includes: Sort the trusted virtual dividing lines; The virtual dividing line of the trusted header with consecutive serial numbers is the starting line of the table column; The credible tail virtual dividing line for consecutive merged sequence numbers is the table column termination line; The adjacent end lines and start lines of the table columns are merged to obtain the virtual wireless table column lines, wherein the end lines of the table columns are located to the left of the start lines of the table columns.

2. The table recognition method according to claim 1, characterized in that, Methods for "obtaining text line information from text lines in a table image to be recognized" include: The text row position in the table image to be identified is obtained by using a text detection model; Based on the position of the text line, the text line image is cropped from the table image to be recognized; The text recognition model is used to obtain the content and position of each character in the text line image.

3. The table recognition method according to claim 1, characterized in that, The method for "obtaining the table region in the table image to be identified" includes: The table region is obtained by using a trained table region detection model; The table region detection model is trained using sample images with completed region annotations, wherein the region annotations include table category and location annotations, the table category includes at least one of multi-bordered table, few-bordered table, and no-bordered table, and the location annotation is the bounding rectangle containing the table region; The table region detection model is one of the following: YOLO network, Faster R-CNN network, or SSD network.

4. The table recognition method according to claim 1, characterized in that, The method for "extracting the table border structure of the table region based on the table border foreground image" includes: The horizontal and vertical outlines in the foreground image of the table frame are obtained through morphological operations. The point where the horizontal and vertical outlines intersect is called the intersection point. The starting / ending point of the horizontal outline and the starting / ending point of the vertical outline are collectively referred to as the endpoints. Obtain the four-way connection index of the intersection point and the endpoint in the direction of the horizontal contour line and the direction of the vertical contour line; Based on the horizontal contour line, the vertical contour line, and the table area, a table candidate area is obtained; The horizontal and vertical contour lines within the candidate area of ​​the table are sorted to obtain the first horizontal contour line number and the first vertical contour line number. Based on the horizontal contour line deviation threshold and the vertical contour line deviation threshold, the first horizontal contour line number and the first vertical contour line number are merged and updated respectively to obtain the second horizontal contour line number and the second vertical contour line number. Based on the horizontal contour line, the vertical contour line, the second horizontal contour line number, and the second vertical contour line number, a contour point map is obtained; The points in the contour point map are connected according to the four-way connection index to obtain the table frame structure.

5. The table recognition method according to claim 1, characterized in that, The method for "determining the table row in the table area based on the position of the text line" includes: Obtain the vertical overlap ratio of the first text line and the second text line, where the first text line and the second text line are any two text lines in the table area; When the overlap ratio is greater than the overlap ratio threshold, it is determined that the first text line and the second text line belong to the same table row; Sort the text lines contained in the table rows and create a text line index.

6. The table recognition method according to claim 1, characterized in that, The head cutting point includes the head cutting point at the end of the text line and the head cutting point in the middle of the text line; the tail cutting point includes the tail cutting point at the end of the text line and the tail cutting point in the middle of the text line; the method for "obtaining the column splitting candidate points in the table row" includes: Insert a beginning cutoff point at the left of the starting character of the text line; Insert a tail cutoff point at the end of the text line to the right of the ending character of the text line; The column segmentation candidate points are inserted into the text line based on the content or position of the single character. Sort the column splitting candidate points contained in the table rows and create a column splitting candidate point index; The method of "inserting the column segmentation candidate point in the text line according to the content or position of the single character" includes: Get the first character spacing between the selected character and the character to its left adjacent to the selected character. Get the second character spacing between the selected character and the character adjacent to the right of the selected character. When the ratio of the first character spacing to the second character spacing is greater than the character spacing threshold, a middle tail cutoff point and a middle head cutoff point of the text line are inserted to the left of the selected character. When the ratio of the first character spacing to the second character spacing is less than the reciprocal of the character spacing threshold, insert the middle end cut point and the middle beginning cut point of the text line to the right of the selected character. When the single-word content transitions from a numeric string to a text string, a middle end cutoff point and a middle beginning cutoff point of the text line are inserted sequentially between the numeric string and the text string.

7. The table recognition method according to claim 1, characterized in that, The method of "determining the credibility of the virtual dividing line of the column to obtain a credible virtual dividing line" also includes: When the first proportion is less than the first confidence threshold, obtain the second proportion of the text line end head cut point or the text line end tail cut point among all the column segmentation candidate points on the column virtual segmentation line; When the second ratio is greater than or equal to the second confidence threshold, the column virtual dividing line is the confidence virtual dividing line.

8. The table recognition method according to claim 7, characterized in that, The method for "obtaining the table text structure based on the virtual wireless table column lines, the virtual wireless table row lines, and the text row information" includes: A virtual wireless table is obtained based on the column lines and row lines of the virtual wireless table; based on the single character content and position in the text line information, the characters in the text line are placed one by one into the virtual wireless cells of the virtual wireless table to obtain the table text structure.

9. The table recognition method according to claim 8, characterized in that, The method for "obtaining the recognized table based on the table border structure and the table text structure, and in combination with the table area" includes: When the table area is the multi-border table area, the text line is added based on the table border structure; When the table area is the table area with few borders, obtain the correspondence between each virtual wireless table row and the text row in the virtual wireless table, and fill the text row corresponding to the virtual wireless table row into the table border structure row by row; When the table area is a borderless table area, the table text structure is directly added to the table border structure.

10. A document acquisition method, characterized in that, The method includes: Obtain the image of the document to be processed; According to any one of claims 1 to 9, the table recognition method obtains tables and text lines in the document image to be processed, wherein the number of tables is one or more; The document that has been identified is obtained based on the table and the text lines outside the table area.

11. A storage medium adapted to store multiple lines of program code, characterized in that, The program code is adapted to be loaded and run by a processor to perform the table recognition method of any one of claims 1 to 9, or the document acquisition method of claim 10.

Images