Document table identification method, reconstruction method and device based on textbox topology

Abstract

The invention discloses a textbox topology-based document table recognition method and device and a textbox topology-based document table reconstruction method and device, and relates to the field of image processing. The method comprises the following steps of: obtaining position information of a textbox of a target document image; shielding all textbox areas of the target document image to obtain a mask image; identifying all line segments in the mask image to obtain a line segment set; combining the textboxes which do not have inclusion relations in the textbox set and do not have cross relations with any horizontal line segments or vertical line segments in the line segment set to form block textboxes, and obtaining a block textbox set; determining a global minimum enclosing rectangle of all the block textboxes in the block textbox set to obtain a global boundary, and respectively outwards expanding each block textbox in the horizontal direction and the vertical direction until an expansion stop condition is met to obtain an expanded block textbox; according to the position of each extension block textbox, all the cell areas are determined, and the situation that the false detection rate is large due to character interference is avoided.

Description

Technical Field

[0001] This application relates to the field of image processing, and in particular to a document table recognition method, reconstruction method, and apparatus based on text box topology. Background Technology

[0002] Recognizing tables in scanned and photographed documents is widely used in many scenarios (such as document digitization, medical form structuring, logistics document recognition, data analysis, data integration and sharing, and automating table processing to improve work efficiency), and has always been a key focus in the field of image processing.

[0003] Existing table recognition technologies include rule-based table recognition technologies. The inventors of this application have discovered that rule-based table recognition technologies have the following limitations:

[0004] Rule-based table recognition methods use edge detection and Hough transform to find lines, and finally generate cells based on the intersection points. They rely on clear table lines, and text interference can lead to a high false detection rate for table lines and cells. Summary of the Invention

[0005] The purpose of this application is to provide a document table recognition method, reconstruction method, and apparatus based on text box topology, so as to solve the problem that existing rule-based table recognition methods cannot detect table lines and cells due to text interference, resulting in a high false detection rate.

[0006] To achieve the above objectives, this application provides the following solution:

[0007] In a first aspect, this application provides a document table recognition method based on text box topology, characterized in that the document table recognition method based on text box topology includes:

[0008] Obtain the position information of the text boxes in the target document image;

[0009] Based on the position information of each text box, all text box areas of the target document image are masked to obtain a mask image;

[0010] All line segments in the mask image are identified using computer vision detection methods to obtain a set of line segments;

[0011] The text boxes that do not contain each other in the text box set and do not intersect with any horizontal or vertical line segment in the line segment set are merged to form a block text box set.

[0012] Determine the global minimum bounding rectangle of all block text boxes in the block text box set to obtain the global boundary, and expand each block text box outward in the horizontal and vertical directions respectively until the expansion stopping condition is met to obtain the expanded block text box;

[0013] Determine the range of all cells based on the position of each extended block text box.

[0014] Optionally, the document table recognition method based on text box topology further includes:

[0015] Before obtaining the mask image, for all text box regions in the target document image, the tilt angle of each side of each text box is calculated based on the four corner coordinates of each text box, and a corner coordinate matrix of size 4×N and a tilt angle matrix of size N×5 are constructed; where N represents the number of text boxes, 5 represents the number of tilt angles for each text box, and the tilt angle of each text box includes the tilt angle of each side of the text box and the average tilt angle of the text box;

[0016] The abnormal text boxes are identified based on the tilt angle matrix, and the corner coordinates of the abnormal text boxes are removed from the corner coordinate matrix to obtain the effective text box matrix.

[0017] Based on the effective text box matrix, the target document image is subjected to gridded geometric correction to obtain the corrected image;

[0018] The masking of all text box areas in the target document image specifically includes:

[0019] Mask all text box areas in the corrected image.

[0020] Optionally, the abnormal text box is determined by following these steps:

[0021] Find the text boxes adjacent to each text box from all text boxes, and construct an N×N cross relationship matrix;

[0022] If in the cross-relationship matrix, any text box B i B is satisfied if the following conditions are met. i If it is an abnormal text box, otherwise, B i For a basic normal text box:

[0023] B i There are two adjacent text boxes, one on the left and one on the right.

[0024] Not in the neighboring dip angle interval Inside, and

[0025] in, For B i The angle of inclination, For B i The angle of the text box adjacent to the left. For B i The angle of the adjacent text box to the right. for and The average value, where T is the preset angle difference threshold;

[0026] If any initially normal text box B i B is satisfied if the following conditions are met. i For the final normal text box, otherwise, B i For the exception text box:

[0027]

[0028] Wherein, δ represents the expansion range of the preset neighborhood angle interval.

[0029] Optionally, the document table recognition method based on text box topology further includes:

[0030] Within the rectangular area containing all text boxes, text boxes that do not have an inclusion or intersection relationship with any identified horizontal or vertical line segment are merged to obtain a block text box. Before this process, line segments that overlap with the text box and isolated line segments around it are removed.

[0031] Optionally, determining all cell ranges based on the positions of each expanded block text box specifically includes:

[0032] If the left edge of the extended block text box reaches the left edge of the global boundary and the right edge reaches the right edge of the global boundary, then the extended block text box is a full text area, and the area above, below, or between the full text area is a table-containing area.

[0033] If there is an extended block text box within the table area and the upper edge of the extended block text box is adjacent to the full text area above it, and / or the lower edge of the extended block text box is adjacent to the full text area below it, then the extended block text box within the table area is an embedded text area, and the areas within the table area that are adjacent to the embedded text area on the left and right are embedded table areas.

[0034] If there is no embedded text area within the table area, then the table area is a full table area; wherein, the embedded table area or the full table area is the determined area of ​​all cells.

[0035] Secondly, this application provides a document table reconstruction method based on text box topology, including:

[0036] The document table recognition method described above is used to determine all cell regions of the target document image;

[0037] Based on the cell position relationships of all cell regions, the table in the target document image is reconstructed; wherein, all cell regions include embedded table regions and full table regions.

[0038] Optionally, reconstructing the table in the target document image based on the cell position relationships of all cell regions specifically includes:

[0039] Create a table border around the outer edge of all the cell ranges;

[0040] Draw lines along each edge of each extended text box and extend them outward from both ends until they intersect the table border. If a line segment passes through an extended text box, cut off the part of the line segment that passes through the extended text box and keep the rest of the line segment.

[0041] If there are no extended text boxes between all parallel line segments with the same starting position, and the distance between the parallel line segments with the same starting position is less than the height of the text box, then the parallel line segments with the same starting position are merged into one line segment to complete the cell reconstruction of the table.

[0042] The text in the expanded block text boxes within each reconstructed cell is merged into the cell text.

[0043] Optionally, the document table reconstruction method based on text box topology further includes:

[0044] If any extended block text box contains a diagonal line, the cell containing the extended block text box is divided into multiple sub-cell areas according to the diagonal line, and all text boxes within the sub-cell area are merged into the sub-cell text.

[0045] Thirdly, this application provides a document table recognition device based on text box topology, comprising:

[0046] The character recognition module is used to obtain the position information of the text boxes in the target document image;

[0047] The mask acquisition module is used to mask all text box areas of the target document image based on the position information of each text box, thereby obtaining a mask image.

[0048] The line segment recognition module is used to identify all line segments in the mask image using computer vision detection methods to obtain a set of line segments.

[0049] The block text box acquisition module is used to merge text boxes in the text box set that do not have an inclusion relationship and do not intersect with any horizontal or vertical line segment in the line segment set to form a block text box set.

[0050] The expansion module is used to determine the global minimum bounding rectangle of all block text boxes in the block text box set, obtain the global boundary, and expand each block text box outward in the horizontal and vertical directions respectively until the expansion stopping condition is met, thus obtaining the expanded block text box.

[0051] The table region recognition module is used to determine all cell regions based on the position of each extended block text box.

[0052] Fourthly, this application provides a document table reconstruction apparatus based on text box topology, including the document table recognition apparatus and table reconstruction module based on text box topology as described above, wherein:

[0053] The table reconstruction module is used to reconstruct the table in the target document image based on the cell position relationship of all cell regions; wherein, all cell regions include embedded table regions and full table regions.

[0054] This application provides a document table recognition, reconstruction method, and apparatus based on text box topology. First, by acquiring the position information of text boxes in the target document image, all text box areas in the target document image are masked based on the position information of each text box. This reduces text interference in the table and minimizes the false detection rate of table lines and cells caused by text interference when relying on clear table lines for table recognition. Then, a computer vision detection method is used to identify all line segments in the mask image, obtaining a line segment set, thus achieving accurate identification of line segments in the table and improving the recognition accuracy of document tables. Next, text boxes that do not have an inclusion relationship and do not intersect with any horizontal or vertical line segment in the line segment set are merged to form block text boxes. After obtaining the block text box set, the global minimum bounding rectangle of all block text boxes in the block text box set is determined to obtain the global boundary. Each block text box is then expanded outwards in both the horizontal and vertical directions until the expansion stops, resulting in an expanded block text box. Based on the position of each expanded block text box, all cell areas are determined, achieving automatic and accurate identification of table areas. Attached Figure Description

[0055] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0056] Figure 1 A flowchart illustrating a document table recognition method based on text box topology, provided as an embodiment of this application;

[0057] Figure 2 A schematic diagram of a table-containing area provided in an embodiment of this application;

[0058] Figure 3 A schematic diagram illustrating a full text area, an embedded text area, and an embedded table area provided in an embodiment of this application;

[0059] Figure 4 A flowchart illustrating a document table reconstruction method based on text box topology, provided for another embodiment of this application;

[0060] Figure 5 A schematic diagram of the functional modules of a document table recognition device based on text box topology provided in an embodiment of this application;

[0061] Figure 6 A schematic diagram of the functional modules of a document table reconstruction device based on text box topology provided in an embodiment of this application;

[0062] Figure 7 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0063] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0064] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0065] In one exemplary embodiment, such as Figure 1 As shown, a document table recognition method based on text box topology is provided. This method is executed by a computer device and includes the following steps 101 to 106. Wherein:

[0066] Step 101: Obtain the position information of the text box in the target document image.

[0067] In this embodiment, the target document image is a document image obtained by scanning, photographing, or other methods of a document containing tables. The text boxes in the target document image refer to text boxes obtained through general OCR (Optical Character Recognition) technology, visual video, or manually annotated text boxes. The position information of the text boxes includes the coordinates of the four corner points of the text box. Step 101 stores the position information of all text boxes in a matrix to obtain a text box matrix.

[0068] Step 102: Based on the position information of each text box, mask all text box areas of the target document image to obtain a mask image.

[0069] In this embodiment, masking all text box areas involves setting the pixels of all text box areas to 0 (black), while retaining the original grayscale values ​​of other areas, thus obtaining a mask image. Masking all text box areas reduces interference from text on line segment recognition, ensuring the accuracy of line segment recognition.

[0070] Step 103: Identify all line segments in the mask image using computer vision detection methods to obtain a set of line segments.

[0071] In this embodiment, line segments include horizontal line segments, vertical line segments, diagonal lines, and miscellaneous lines. All line segments in the mask image can be identified using computer vision detection methods. In the line segment set, each line segment is represented by the coordinates of its start and end points. Computer vision detection methods include Hough Transform, Line Segment Detector (LSD), and Canny edge detection combined with Hough Transform.

[0072] Step 104: Merge text boxes in the text box set that do not have an inclusion relationship and do not intersect with any horizontal or vertical line segment in the line segment set to form a block text box set.

[0073] In this embodiment of the application, the text box set is B = {B1, B2, ..., B}. N The set of line segments L is L = {L1, L2, ..., L}. M For any horizontal or vertical line segment L in the set of line segments L, k and any two text boxes B in the text box set B i and B j If the following formula is satisfied, then it is considered to be related to any horizontal or vertical line segment L in the line segment set L. k There is no overlap:

[0074]

[0075] st(L k With B i Or B j Intersecting)∨(L k Spanning BoundingBox(B i ∪B j ));

[0076] Where ∨ represents the OR operation, BoundingBox(B i ∪B j ) represents text box B i and B j The axis is aligned to the bounding rectangle, BoundingBox(B i ∪B j The coordinates of the top left corner of ) are (min(B i .x min B j .x min ),min(B i .y min B j .y min The coordinates of the lower right corner are (max(B)). i .x max B j .x max ),min(B i .y max B j .y mma )).

[0077] If any two text boxes B in the text box set B are... i and B j Text box B is considered valid if the following conditions are met. i and B j No overlap (B) i B j ):

[0078]

[0079] Among them, IoU(B i B j () indicates text box B i and B j The intersection-union ratio, ^ represents the AND operation, Area(Intersection(B i B j )) indicates text box B i and B j The area of ​​intersection.

[0080] This article's frame B i and Bj Merge is allowed if and only if the following merge condition is met: MergeCondition(B i B j ,L):

[0081]

[0082] Step 105: Determine the global minimum bounding rectangle of all block text boxes in the block text box set to obtain the global boundary, and expand each block text box outward in the horizontal and vertical directions respectively until the expansion stopping condition is met to obtain the expanded block text box.

[0083] In this embodiment of the application, the global minimum bounding rectangle of all block text boxes in the block text box set is determined, that is, the (x) of all block text boxes is taken. min y min (x) is used as the coordinates of the top-left corner of the global minimum bounding rectangle, and the coordinates of the top-left corner of all block text boxes are taken. max y max The coordinates of the bottom right corner of the smallest global bounding rectangle.

[0084] Expansion stopping conditions include horizontal expansion stopping conditions and vertical expansion stopping conditions.

[0085] Horizontal expansion stopping conditions include left-side expansion stopping conditions and right-side expansion stopping conditions. Left-side expansion stopping conditions occur when, during expansion to the left, the nearest vertical line segment (a line segment in the set with an angle close to 90°), the left boundary of the global border, or the right vertical edge of another block text box. Right-side expansion stopping conditions occur when, during expansion to the right, the nearest vertical line segment, the right boundary of the global border, or the left vertical edge of another block text box.

[0086] Vertical expansion stopping conditions include upward expansion stopping conditions and downward expansion stopping conditions. The upward expansion stopping condition is when, during the upward expansion process, the nearest horizontal line segment (a line segment in the set with an angle close to 0°), the upper boundary of the global boundary, or the lower horizontal edge of another block text box. The downward expansion stopping condition is when, during the downward expansion process, the nearest horizontal line segment, the lower boundary of the global boundary, or the upper horizontal edge of another block text box.

[0087] An expanded block text box is a text box that expands outward from each block text box in both the horizontal and vertical directions.

[0088] Step 106: Determine all cell ranges based on the positions of each expanded block text box.

[0089] In this embodiment of the application, if the left edge of the extended block text box reaches the left boundary of the global boundary and the right edge reaches the right boundary of the global boundary, then the extended block text box is a full text area, and the area above, below or between the full text area is a table-containing area.

[0090] If an extended text box exists within a table area and its upper edge is adjacent to the full text area above it, and / or its lower edge is adjacent to the full text area below it, then the extended text box is an embedded text area. Correspondingly, the areas within the table area that are adjacent to the embedded text area on the left and right are embedded table areas.

[0091] If there is no embedded text area within a table area, then that area is the entire table area.

[0092] The resulting embedded table area or full table area is the determined range of all cells.

[0093] For example, Figure 2 The red box indicates the area containing the table. Figure 3 The green boxes above and below the red box represent the entire text area. Figure 3 The green boxes to the left and right of the red box indicate the embedded text areas. Figure 3 The red box indicates the embedded table area. Figure 3 There are no green boxes to the left and right of the red box in the middle, that is... Figure 3 The red box in the middle represents the entire table area.

[0094] By implementing steps 101 to 106 above, firstly, the position information of the text boxes in the target document image is obtained. Based on the position information of each text box, all text box areas in the target document image are masked to reduce text interference in the table and avoid the problem of high false detection rate of table lines and cells due to text interference when relying on clear table lines for table recognition. Then, all line segments in the mask image are identified by computer vision detection methods to obtain a set of line segments, thereby achieving accurate recognition of line segments in the table and improving the recognition accuracy of document tables. Next, text boxes that do not have an inclusion relationship and do not intersect with any horizontal or vertical line segment in the line segment set are merged to form block text boxes. After obtaining the set of block text boxes, the global minimum bounding rectangle of all block text boxes in the block text box set is determined to obtain the global boundary. Each block text box is then expanded outward in both the horizontal and vertical directions until the expansion stops, resulting in an expanded block text box. Based on the position of each expanded block text box, all cell areas are determined, thereby achieving automatic and accurate recognition of table areas.

[0095] In another exemplary embodiment of this application, the above-described document table recognition method based on text box topology further includes steps 201 to 203. Wherein:

[0096] Step 201: Before obtaining the mask image by masking all text box regions in the target document image, calculate the tilt angle of each side of each text box based on the four corner coordinates of each text box and construct a corner coordinate matrix of size 4×N and a tilt angle matrix of size N×5; where N represents the number of text boxes, and 5 represents the number of tilt angle information for each text box. The tilt angle information for each text box includes the tilt angle of each side of the text box and the average tilt angle of the text box.

[0097] In this embodiment of the application, the coordinate matrix of the four corners of the k-th text box is set as follows:

[0098]

[0099] Where (x1,y1), (x2,y2), (x3,y3), and (x4,y4) are the coordinates of the four corner points of the k-th text box arranged clockwise or counterclockwise.

[0100] Then the four edge vectors of the k-th text box are defined as follows:

[0101]

[0102] Among them, e1, e2, e3, and e4 are the four edge vectors of the k-th text box arranged in clockwise or counterclockwise order.

[0103] Then the angle of inclination of each side of the k-th text box is:

[0104]

[0105] Where, θ i Let Δy be the angle of inclination of the i-th side of the text box. i Let Δx be the absolute value of the difference between the y-axis coordinates of the starting and ending points of the i-th edge vector. i It is the absolute value of the difference between the x-axis coordinates of the starting and ending points of the i-th edge vector.

[0106] Step 202: Determine the abnormal text boxes based on the tilt angle matrix, and remove the corner coordinates of the abnormal text boxes from the corner coordinate matrix to obtain the effective text box matrix.

[0107] In this embodiment of the application, in the horizontal or vertical direction, two adjacent text boxes should meet the smoothness condition, that is, the tilt angle of the text box and the text boxes adjacent to it on the left and right or the text boxes adjacent to it on the top and bottom should be equal, decreasing or increasing. If the smoothness condition is not met, it indicates that the text box is abnormal and should be removed to avoid inaccurate calculation of local deformation and affect the subsequent recognition result of the table.

[0108] Step 203: Based on the effective text box matrix, perform gridded geometric correction on the target document image to obtain the corrected image.

[0109] In this embodiment, the effective text box matrix can more effectively correct the target document image, simplifying the merging and expansion process of text boxes. This embodiment is the first to use the geometric features of text boxes for image correction and image adjustment, effectively correcting overall or partial rotation, bending, and perspective problems generated during shooting or scanning, and achieving effective processing of tables with complex geometric deformations.

[0110] When processing the rotation correction of target document images, the inventors of this application discovered that the rotation of the scanned document resulted in irregular black borders on some edges, causing misidentification of the largest rectangle (assuming that the corners of the largest rectangle could maximize the correction effect and minimize error), necessitating the selection of the second largest rectangle. However, to ensure the effectiveness of the second largest rectangle, it is necessary to obtain subsequent smaller rectangles to confirm its validity, i.e., confirm that the second largest rectangle belongs to a cell or combination of cells within the table range. However, the positions of these subsequent smaller rectangles are not fixed under different sample conditions. If the sample has table distortion, this process becomes very unstable, meaning that the cell or combination of cells used as the correction reference is itself unreliable and needs correction. Therefore, to reduce text interference, a strategy of identifying and masking text boxes in the table is adopted. The numerous text boxes distributed throughout the table have an orientation consistent with the curvature of the table lines, meaning that the text box within each cell can correct its own cell, forming a region correction. The text boxes within the entire table form a calibration matrix. Therefore, it is proposed to use the text boxes to form a global tilt matrix to reflect the deformation state of different positions in the table. Then, the global tilt matrix can be used to perform more accurate calibration through regional dynamic affine transformation (mesh processing), with a calibration accuracy of ±5px, which is three times higher than the ±15px calibration accuracy of using a single table rectangle.

[0111] Considering that there are abnormal text boxes in the text matrix due to deviations in the direction judgment of a small number of characters or symbols, in order to filter out these abnormal text boxes, the smoothness (consistency) is judged by the tilt angle with the adjacent text boxes, and finally a global tilt angle matrix of text boxes is formed for the gridding correction of the table.

[0112] Accordingly, in step 102 above, masking all text box areas of the target document image specifically includes:

[0113] Mask all text box areas in the corrected image.

[0114] In another exemplary embodiment of this application, step 202 described above determines the abnormal text box according to steps 301 to 303. Wherein:

[0115] Step 301: Find the text boxes adjacent to each text box from all text boxes and construct an N×N cross relationship matrix.

[0116] In this embodiment of the application, finding the text box adjacent to each text box from all text boxes includes the following steps: 3011 to 3013. Wherein:

[0117] Step 3011: For any text box, with its top-left corner coordinates as (x1, y1), top-right corner coordinates as (x2, y2), bottom-right corner coordinates as (x3, y3), and bottom-left corner coordinates as (x4, y4), define the midpoints of its four sides as follows:

[0118] Midpoint at the top Midpoint on the right Midpoint below Midpoint on the left

[0119] Step 3012, iterate through all text box pairs (B i B j ), calculate B i The midpoint of each edge and B j Distance to the midpoint of each edge;

[0120] Step 3013, if B i M top With B j M bottom If the distance is the smallest and less than the preset distance threshold, then B j For B i The text box above B is labeled 'a'. i M right With B j M left If the distance is the smallest and less than the preset distance threshold, then B j For B i The text box adjacent to the right of B is labeled b. If B i M bottom With B j M top If the distance is the smallest and less than the preset distance threshold, then B j For Bi The text box adjacent to B below is labeled c. If B i M left With B j M right If the distance is the smallest and less than the preset distance threshold, then B j For B i The text box to the left of the text box is labeled d.

[0121] In this embodiment, a, b, c, and d are not specifically limited and can be set according to actual needs. For example, a = 1, b = 2, c = 3, and d = 4 can be set.

[0122] Construct an N×N cross relationship Rel[i][j] according to the following formula:

[0123]

[0124] In this embodiment of the application, if k = a, B j It is B i The text box above B; if k = b, B j It is B i The text box adjacent to the right of B; if k = c, B j It is B i The text box adjacent to B below; if k = d, B j It is B i The text box adjacent to the left of the text box.

[0125] Step 302, if any text box B in the cross-relationship matrix... i B is satisfied if the following conditions are met. i If it is an abnormal text box, otherwise, B i For a basic normal text box:

[0126] B i There are two adjacent text boxes, one on the left and one on the right.

[0127] Not in the neighborhood dip angle interval Inside, and

[0128] in, For B i The angle of inclination, For B i The angle of the text box adjacent to the left. For B i The angle of the adjacent text box to the right. for and The average value of the angle difference is T, where T is a preset angle difference threshold. The tilt angle of the text box is the average of the tilt angles of any two opposite sides (two horizontal sides or two vertical sides).

[0129] Step 303, if any initially normal text box B i B is satisfied if the following conditions are met. i For the final normal text box, otherwise, B i For the exception text box:

[0130]

[0131] Wherein, δ represents the expansion range of the preset neighborhood angle interval.

[0132] In this embodiment of the application, the setting of δ allows text box B to... i The tilt angle has a certain degree of flexibility within the neighborhood tilt angle range to avoid misjudging text box B due to fluctuations in the neighborhood tilt angle itself. i abnormal.

[0133] In another exemplary embodiment of this application, step 203 specifically includes:

[0134] Using the following formula, the coordinates (x, y) of each corner point of each text box in the target document image are mapped to the correction coordinates (x', y') using the deformation field parameter matrix W and the affine transformation matrix A, respectively, to obtain the corrected image:

[0135]

[0136] Where M is the total number of normal text boxes, and φ(r) is the radial basis function of the thin plate spline, φ(r) = r 2 logr, r = ||(x, y) - p k ||,||(x,y)-p k || represents the distance from point (x, y) to control point p. k Euclidean distance, p k Let w be the coordinates of the k-th original control point. The original control points include the four corner points of N text boxes, and the total number of original control points is 4N. k The element in the k-th row or k-th column of the deformation field parameter matrix W (w xk ,w yk ), w x k represents the x-coordinate weight of the k-th original control point, w yk This represents the weight of the y-coordinate of the k-th original control point.

[0137] In this embodiment, W and A are calculated using the following formula:

[0138]

[0139] Where λ is the principal smoothing coefficient (the larger the value, the smoother the deformation), γ is the affine regularization coefficient to prevent A from overfitting, L is the Laplacian matrix, K is the sparsity kernel matrix, C is the constraint matrix, I3 is the 3×3 identity matrix, and P′ vec This represents the one-dimensional vector formed by expanding the corrected ideal coordinates of each original control point. Multiplying by A essentially adds an L2 regularization term (i.e., weight decay) to the parameters of A to prevent A from overfitting noise or outlier data.

[0140] The sparsified kernel matrix K is calculated according to the following formula:

[0141]

[0142] Among them, K ij p is the value of the element in the i-th row and j-th column of the kernel matrix K. i This represents the coordinates of the i-th original control point. p i The set of k nearest neighbors, i.e., the set of p i The k nearest original control points, p j express The coordinates of the j-th original control point. Sparsening the kernel matrix reduces computation (otherwise, the kernel matrix would be a dense matrix with extremely high computational complexity), and only preserves the correlation between locally adjacent original control points, which is consistent with the characteristic of local deformation dominating in document correction (the deformation of text boxes is usually local, and the influence of distant original control points can be ignored).

[0143] The constraint matrix C is calculated according to the following formula:

[0144]

[0145] Among them, (x i' ,y i' ) represents the coordinates of the i'th original control point, 1 <i'<4M。

[0146] The Laplace matrix L (4M×4M) is calculated according to the following formula:

[0147]

[0148] Among them, L ij Let w be the value of the element in the i-th row and j-th column of the Laplace matrix L. ij w represents the weight between the original control point i and its nearest original control point j. ik Let i be the original control point and the set of its k nearest neighbors. The weights between the k-th original control points are used to measure the association strength between the i-th original control point and its k-th neighbor control point. This association strength is usually calculated based on factors such as the kernel matrix, and can be obtained using thin-plate spline kernel functions, Gaussian kernel functions, etc., based on factors like the distance between two control points. It reflects the influence of adjacent control points on each other's deformation constraints in deformation modeling scenarios such as document correction. For example, for using a thin-plate spline kernel, w... ik The sparse kernel matrix K can be directly taken from the beginning. ik Or its normalized form.

[0149] In another exemplary embodiment of this application, step 102 specifically includes:

[0150] Use the following formula to mask all text box areas of the target document image:

[0151] (M☉I)(x,y)=M(x,y)·I(x,y):

[0152]

[0153] Where (M⊙I)(x,y) represents the grayscale value of pixel (x,y) in the mask image, matrix element I(x,y) represents the grayscale value of pixel (x,y) in the target document image matrix I (grayscale format), matrix element M(x,y) represents the value of the element in the mask matrix M corresponding to pixel (x,y), ⊙ represents element-wise multiplication (Hadamaed product), 0≤x≤W, 0≤Y≤H, W is the width of the target document image, and H is the height of the target document image.

[0154] In this embodiment of the application, the effect of masking is to set the pixel values ​​of all text boxes to 0 (black), while preserving the original grayscale values ​​of other areas.

[0155] In another exemplary embodiment of this application, the above-described document table recognition method based on text box topology further includes:

[0156] Step 401, prior to step 104, remove line segments that overlap with the text box and isolated line segments around it.

[0157] In this embodiment of the application, a line segment isolated on all four sides must meet any of the following conditions:

[0158] 1. Connecting image boundaries: The distance between the endpoints of the line segment and the image edge is no greater than a preset threshold number of pixels;

[0159] 2. No adjacent line segments: There are no other line segment endpoints within the preset r-neighborhood.

[0160] This application does not impose specific limitations on the pixel count threshold and the neighborhood size r, which can be set according to actual needs. For example, the pixel count threshold can be set to 5. For example, the neighborhood size r can be set to 20 pixels.

[0161] In another exemplary embodiment of this application, step 401 above, removing line segments that overlap with the text box, includes:

[0162] Given a set of line segments (each row [x1, y1, x2, y2]) and a collection of text boxes Retain line segments that satisfy the following conditions:

[0163]

[0164] Among them, Intersection(l,B) i ) indicates that line segment l and text box B are intersected. i Intersection detection, This indicates that line segment l and text box B are intersecting. i Non-intersecting, L filtered This indicates the condition that a line segment set L retains. The line segment.

[0165] In another exemplary embodiment of this application, step 401 above, removing isolated line segments around the perimeter, includes:

[0166] Remove isolated line segments from all four sides using the following formula:

[0167]

[0168] Among them, L final Indicates the removal of L filtered The set of non-isolated line segments obtained after considering the isolated line segments around the center. Indicates the logical NOT operation. L filtered \{l} represents L filtered The subset of line segments obtained by removing line segment l from the middle. IsIsolated(l,L) filtered ) indicates that line segment l is in the set of line segments L. filteredThe function checks if a line segment is isolated; it returns True if the line segment is isolated, and False if it is not. `TouchesBorder(l)` determines if line segment `l` connects to the boundary of the target document image. It returns True if the distance between the endpoints of line segment `l` and the edge of the target document image is less than a set distance threshold, indicating that line segment `l` connects to the boundary of the target document image. Its width W (horizontal pixels) and height H (vertical pixels) determine the boundary of the target document image. `d(endpoints(l), endpoints(l′))` calculates the endpoints `endpoints(l)` of line segment `l` and the line segment set `L`. filtered This function calculates the distance (e.g., Euclidean distance) between the endpoints (l′) of other line segments l and l′. `endpoints()` retrieves the coordinates of the two endpoints of line segments l and l′. `r` represents the neighborhood threshold (e.g., 20 pixels) used to determine if there are no adjacent line segments. If the minimum distance between the endpoints of line segments l and l′ is greater than `r`, then there are no other line segment endpoints within the neighborhood of `r`.

[0169] In another exemplary embodiment of this application, step 104 specifically includes steps 501 to 503. Wherein:

[0170] Step 501, define a conflict graph G = (V, E), where each vertex V corresponds to a text box, and the edges...

[0171] In the embodiments of this application, Represents the logical NOT operation. Text box B i and B j If the merging conditions are not met, then text box B will be merged in Figure G. i and B j Connecting the corresponding vertices yields edge E.

[0172] Step 502, find the complement graph of the conflict graph G. The maximal clique in, namely:

[0173]

[0174] Here, Blocks represents the set of all maximal cliques. Indicates supplementary image The largest cluster.

[0175] In this embodiment of the application, supplementary diagrams are provided. The meaning of the edge in the middle is the opposite of the meaning of the edge in the conflict graph G. (Supplementary graph) In a maximal clique, the vertices corresponding to two text boxes that can be merged are connected to form an edge. Each maximal clique corresponds to a set of mergeable text boxes, because any two text boxes within a maximal clique satisfy the merging condition.

[0176] If supplementary image If any pair of vertices C satisfies the following conditions, then the pair of vertices C forms the complement graph. The largest cluster:

[0177] There is an edge between any two vertices in C, which means that there is no edge in the original conflict graph G, and the MergeCondition is true.

[0178] It is impossible to add any more vertices to C without breaking the property that any two vertices in C have an edge.

[0179] Step 503: For each maximal clique C contained in the text box set, calculate the minimum bounding rectangle B according to the following formula. block This results in a block text box:

[0180]

[0181] Here, ∪ represents the union, forming the smallest area that covers all text boxes, but it is not necessarily the smallest rectangle, so a minimum bounding matrix operation MinBoundingBox is added outside.

[0182] In this embodiment, the coordinates of the four corner points of the smallest bounding rectangle are calculated.

[0183] Many table recognition technologies often fail to effectively recognize merged cells, treating them as multiple rows or columns and disrupting the original semantic meaning. This hinders semantic recognition of the text content in subsequent cells. However, the embodiments of this application can accurately merge the text in some merged cells.

[0184] In one exemplary embodiment, such as Figure 4 As shown, a document table reconstruction method based on text box topology is provided. This method is executed by a computer device and, in addition to steps 101 to 106 described above, also includes:

[0185] Step 601: Reconstruct the table in the target document image based on the cell position relationship of all cell regions; wherein, all cell regions include embedded table regions and full table regions.

[0186] When extracting table lines, it was still found that many erroneous line segments were generated due to image quality and extraction parameter settings. This application embodiment reconstructs the table in the target document image based on the positional relationship between the line segments and the extended block text box. It can filter out erroneous line segments and merge duplicate line segments based on the positional relationship between the extended block text box and the line segments, thus ensuring the accuracy of the reconstructed table.

[0187] In another exemplary embodiment of this application, step 601 described above includes steps (1) to (4). Wherein:

[0188] Step (1) Create table borders at the outer borders of all cell ranges.

[0189] In this embodiment of the application, a table outline is created at the outer frame of all cell areas, that is, four intersecting line segments (top, bottom, left, and right) are created at the outer frame of all cell areas as the table outline.

[0190] Step (2): Draw lines along each edge of each extended text box and extend them outward from both ends until they intersect with the table frame. If a line segment passes through an extended text box, cut off the part of the line segment that passes through the extended text box and keep the rest of the line segment.

[0191] Step (3): If there are no extended block text boxes between all line segments that start at the same position and are parallel to each other, and the distance between the line segments that start at the same position and are parallel to each other is less than the height or width of the smallest unit text box, then the line segments that start at the same position and are parallel to each other are merged into one line segment, and the cell reconstruction of the table is completed.

[0192] In this embodiment, the text box sizes are not consistent due to font size differences. The smallest unit text box refers to the text box with the smallest height or width. Taking the smallest unit text box is to prevent the borders of cells with potentially empty text from being merged.

[0193] Step (4) merges the text in the expanded block text boxes of each reconstructed cell into cell text.

[0194] In another exemplary embodiment of this application, the above-described document table reconstruction method further includes:

[0195] If any extended block text box contains a diagonal line, the cell containing that extended block text box is divided into multiple sub-cell areas according to the diagonal line, and all text boxes within the sub-cell area are merged into the sub-cell text.

[0196] In this embodiment of the application, if any line segment in the set of line segments obtained in step 103 intersects any corner point of the cell at one end and intersects the edge of the cell at the other end, and is neither a horizontal line segment nor a vertical line segment, then it is considered to be a diagonal line.

[0197] Rule-based table recognition methods often make significant errors in identifying merged cells, cells divided by diagonal lines, and the attribution of text within cells. In contrast, this application can accurately identify merged cells, cells divided by diagonal lines, and the attribution of text within cells.

[0198] This application prioritizes using text boxes from the target document image to construct the tilt matrix, overcoming two technical biases of traditional methods:

[0199] Bias 1: Table line detection should precede text recognition;

[0200] In this embodiment, a text box is used to obtain table cells, and then the table lines are optimized.

[0201] Bias 2: Text areas should be completely excluded;

[0202] This application first performs image gridding correction using recognized text boxes, then identifies table lines to form the initial table outline, and finally expands the text boxes to determine the cells and refines the table lines to complete the reconstruction of the entire table structure and cells. This changes the original mode of recognizing tables and text separately to a mode of bidirectional collaborative processing of tables and text, reflecting an advanced approach that prioritizes structure and uses text as a supplement.

[0203] The advantages of using OCR results to optimize table detection are as follows:

[0204] An effective text box matrix constructed based on text boxes can more effectively correct target document images. Traditional table recognition techniques require setting separate recognition models or parameters for complex table lines, such as double lines, multiple lines, mixed thick lines, and merged cells; otherwise, misjudgments will occur. However, in this embodiment, step 105 restores the size and position of the cells first, allowing for unified processing of complex table lines and eliminating the need to consider the processing difficulty of line shape recognition. It can also recognize diagonal lines in cells, enabling the reconstruction of sub-cells.

[0205] Based on the position of each extended block text box, table areas and non-table areas can be accurately separated. Unlike previous table recognition methods, it does not require manual determination of table areas.

[0206] The text within a cell can be merged after the table is divided, reducing the difficulty of recognizing text within separate cells. Previous patented methods processed tables and text blocks separately, making it difficult to effectively handle conflicts (text block processing without table constraints can result in cross-cell text connections or text recognition errors).

[0207] For more complex target document images, feedback mechanisms can be used to further optimize the table recognition results. Because curved text lacks sufficient alignment, only single-character recognition is possible. However, after correction and position alignment, the contextual relationships of the text can be established, allowing for more accurate recognition through contextual semantics.

[0208] Machine learning-based table recognition techniques use classifiers to determine whether pixels belong to table lines, and post-processing employs Non-Maximum Suppression (NMS). However, these techniques have limitations, requiring extensive labeled data for training and exhibiting poor generalization to tilted documents. This application provides a more economical implementation method that is lighter and more interpretable than deep learning solutions in document scanning and imaging scenarios. This application achieves performance close to that of deep learning methods but eliminates the significant costs associated with sample data collection, labeling, and training. In practical applications, it can utilize lighter, low-resource environments (without GPUs). By employing an effective text box matrix, gridded geometric correction is applied to the target document image, improving the recognition and reconstruction of tilted documents and addressing the poor generalization problem of machine learning-based table recognition techniques for tilted documents.

[0209] End-to-end deep learning-based table recognition technology directly recognizes table content, but its limitation is that it cannot obtain the physical table line coordinates and requires high GPU computing power (>4GB video memory). Since it cannot recognize the table line coordinates, it cannot accurately obtain the table structure by relying solely on text blocks. However, this application can accurately obtain the table structure.

[0210] The technical solution of this application has many technical advantages, as shown in Table 1 below.

[0211] Table 1. Comparison of the advantages of the technical solution in this application with traditional table recognition technology.

[0212] Technical indicators This application proposal Traditional rule-based methods Deep learning-based methods Deformation handling capability Strong (supports various complex deformations) Weak (only supports simple rotations) Medium (depends on training data) Borderless table recognition support Not supported Partial support Computing resource requirements (CPU only) Low High (GPU required) Training data requirements unnecessary unnecessary A large amount of labeled data is needed. Processing speed Fast (Optimization of traditional algorithms) quick Slow (model inference time) Difficulty of technology updates Low (parameter adjustment) High (Rules need to be modified) High (requires retraining) Explainability high high Low-level (black box model)

[0213] Based on the same inventive concept, this application also provides a document table recognition device based on text box topology for implementing the document table recognition method based on text box topology described above. The solution provided by this device is similar to the solution described in the above method. Therefore, the specific limitations of one or more document table recognition device embodiments based on text box topology provided below can be found in the limitations of the document table recognition method based on text box topology described above, and will not be repeated here.

[0214] In one exemplary embodiment, such as Figure 5 As shown, a document table recognition device 70 based on text box topology is provided, including:

[0215] The character recognition module 701 is used to obtain the position information of the text box in the target document image;

[0216] The mask acquisition module 702 is used to mask all text box areas of the target document image based on the position information of each text box to obtain a mask image;

[0217] The line segment recognition module 703 is used to identify all line segments in the mask image using a computer vision detection method to obtain a set of line segments.

[0218] The block text box acquisition module 704 is used to merge text boxes in the text box set that do not have an inclusion relationship and do not have an intersection relationship with any horizontal or vertical line segment in the line segment set to form a block text box, thereby obtaining a block text box set.

[0219] The expansion module 705 is used to determine the global minimum bounding rectangle of all block text boxes in the block text box set, obtain the global boundary, and expand each block text box outward in the horizontal and vertical directions respectively until the expansion stopping condition is met, thus obtaining the expanded block text box.

[0220] The table region recognition module 706 is used to determine all cell regions based on the position of each extended block text box.

[0221] In this embodiment of the application, the following descriptions are provided: obtaining the position information of text boxes in the target document image; masking all text box areas of the target document image based on the position information of each text box to obtain a mask image; identifying all line segments in the mask image using computer vision detection methods to obtain a set of line segments; merging text boxes in the text box set that do not have an inclusion relationship and do not intersect with any horizontal or vertical line segment in the line segment set to form a block text box set; determining the global minimum bounding rectangle of all block text boxes in the block text box set to obtain the global boundary; and expanding each block text box outward in the horizontal and vertical directions until the expansion stops to obtain an expanded block text box. For details, please refer to the descriptions in the above method embodiments, which will not be repeated here.

[0222] In another exemplary embodiment of this application, the above-described document table recognition device 70 based on text box topology further includes:

[0223] The image correction module is used to calculate the tilt angle of each side of each text box based on the four corner coordinates of each text box before obtaining the mask image by masking all text box areas in the target document image, and to construct a corner coordinate matrix of size 4×N and a tilt angle matrix of size N×5; where N represents the number of text boxes, 5 represents the number of tilt angle information for each text box, and the tilt angle information for each text box includes the tilt angle of each side of the text box and the average tilt angle of the text box;

[0224] The abnormal text boxes are identified based on the tilt angle matrix, and the corner coordinates of the abnormal text boxes are removed from the corner coordinate matrix to obtain the effective text box matrix.

[0225] Based on the effective text box matrix, the target document image is subjected to gridded geometric correction to obtain the corrected image.

[0226] In another exemplary embodiment of this application, the document table recognition device 70 based on text box topology described above further includes:

[0227] The line segment removal module is used to remove overlapping line segments and isolated line segments from text boxes before they form a block text box, provided that the text boxes do not have an inclusion relationship in the merged text box set and do not intersect with any horizontal or vertical line segments in the line segment set.

[0228] In this embodiment of the application, the specific steps for removing line segments that overlap with the text box and isolated line segments around it are detailed in the above method embodiments, and will not be repeated here.

[0229] Based on the same inventive concept, this application also provides a document table reconstruction apparatus based on text box topology for implementing the document table reconstruction method based on text box topology described above. The solution provided by this apparatus is similar to the implementation described in the above method. Therefore, the specific limitations of one or more embodiments of the document table reconstruction apparatus based on text box topology provided below can be found in the limitations of the document table reconstruction method based on text box topology described above, and will not be repeated here.

[0230] In one exemplary embodiment, such as Figure 6 As shown, a document table reconstruction apparatus 80 based on text box topology is provided, which, in addition to the aforementioned document table recognition apparatus 70 based on text box topology, also includes:

[0231] The table reconstruction module 801 is used to reconstruct the table in the target document image based on the cell position relationship of all cell regions; wherein, all cell regions include embedded table regions and full table regions.

[0232] In this embodiment, the following descriptions are provided: obtaining the position information of text boxes in the target document image; masking all text box areas of the target document image based on the position information of each text box to obtain a mask image; identifying all line segments in the mask image using computer vision detection methods to obtain a set of line segments; merging text boxes in the text box set that do not have an inclusion relationship and do not intersect with any horizontal or vertical line segment in the line segment set to form a block text box set; determining the global minimum bounding rectangle of all block text boxes in the block text box set to obtain the global boundary; and expanding each block text box outward in the horizontal and vertical directions until the expansion stops to obtain an expanded block text box; and reconstructing the table in the target document image based on the cell position relationships of all cell areas. These details are not repeated here.

[0233] In another exemplary embodiment of this application, the above-described document table reconstruction apparatus 80 based on text box topology further includes:

[0234] The sub-cell reconstruction module is used to: if any extended block text box contains a diagonal line, divide the cell containing the extended block text box into multiple sub-cell areas according to the diagonal line, and merge all text boxes in the sub-cell area into sub-cell text.

[0235] In this embodiment of the application, if any line segment in the aforementioned set of line segments intersects with any corner point of the cell at one end and with the edge of the cell at the other end, and is neither a horizontal line segment nor a vertical line segment, then it is considered to be a diagonal line.

[0236] In one exemplary embodiment, a computer device is provided, which may be a server or a terminal, and its internal structure diagram may be as follows. Figure 7As shown, the computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores document table recognition and reconstruction data. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When executed by the processor, the computer program implements a document table recognition method and / or a document table reconstruction method.

[0237] Those skilled in the art will understand that Figure 7 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0238] In one exemplary embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0239] In one exemplary embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0240] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0241] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0242] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0243] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0244] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0245] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A document table recognition method based on text box topology, characterized in that, The document table recognition method based on text box topology includes: Obtain the position information of the text boxes in the target document image; Based on the position information of each text box, all text box areas of the target document image are masked to obtain a mask image; All line segments in the mask image are identified using computer vision detection methods to obtain a set of line segments; The text boxes that do not contain each other in the text box set and do not intersect with any horizontal or vertical line segment in the line segment set are merged to form a block text box set. Determine the global minimum bounding rectangle of all block text boxes in the block text box set to obtain the global boundary, and expand each block text box outward in the horizontal and vertical directions respectively until the expansion stopping condition is met to obtain the expanded block text box; Determine the range of all cells based on the position of each extended block text box.

2. The document table recognition method based on text box topology according to claim 1, characterized in that, Also includes: Before obtaining the mask image, for all text box regions in the target document image, the tilt angle of each side of each text box is calculated based on the four corner coordinates of each text box, and a corner coordinate matrix of size 4×N and a tilt angle matrix of size N×5 are constructed; where N represents the number of text boxes, 5 represents the number of tilt angles for each text box, and the tilt angle of each text box includes the tilt angle of each side of the text box and the average tilt angle of the text box; The abnormal text boxes are identified based on the tilt angle matrix, and the corner coordinates of the abnormal text boxes are removed from the corner coordinate matrix to obtain the effective text box matrix. Based on the effective text box matrix, the target document image is subjected to gridded geometric correction to obtain the corrected image; The masking of all text box areas in the target document image specifically includes: Mask all text box areas in the corrected image.

3. The document table recognition method based on text box topology according to claim 2, characterized in that, To identify the abnormal text box, follow these steps: Find the text boxes adjacent to each text box from all text boxes, and construct an N×N cross relationship matrix; If in the cross-relationship matrix, any text box B i B is satisfied if the following conditions are met. i If it is an abnormal text box, otherwise, B i For a basic normal text box: B i There are two adjacent text boxes, one on the left and one on the right. Not in the neighboring dip angle interval Inside, and in, For B i The angle of inclination, For B i The angle of the text box adjacent to the left. For B i The angle of the adjacent text box to the right. for and The average value, where T is the preset angle difference threshold; If any initially normal text box B i B is satisfied if the following conditions are met. i For the final normal text box, otherwise, B i For error text boxes: Wherein, δ represents the expansion range of the preset neighborhood angle interval.

4. The document table recognition method based on text box topology according to claim 1, characterized in that, Also includes: Within the rectangular area containing all text boxes, text boxes that do not have an inclusion or intersection relationship with any identified horizontal or vertical line segment are merged to obtain a block text box. Before this process, line segments that overlap with the text box and isolated line segments around it are removed.

5. The document table recognition method based on text box topology according to claim 1, characterized in that, The step of determining all cell ranges based on the position of each extended block text box specifically includes: If the left edge of the extended block text box reaches the left edge of the global boundary and the right edge reaches the right edge of the global boundary, then the extended block text box is a full text area, and the area above, below, or between the full text area is a table-containing area. If there is an extended block text box within the table area and the upper edge of the extended block text box is adjacent to the full text area above it, and / or the lower edge of the extended block text box is adjacent to the full text area below it, then the extended block text box within the table area is an embedded text area, and the areas within the table area that are adjacent to the embedded text area on the left and right are embedded table areas. If there is no embedded text area within the table area, then the table area is a full table area; wherein, the embedded table area or the full table area is the determined area of ​​all cells.

6. A document table reconstruction method based on text box topology, characterized in that, The document table reconstruction method based on text box topology includes: The document table recognition method according to any one of claims 1 to 5 is used to determine all cell regions of the target document image; Based on the cell position relationships of all cell regions, the table in the target document image is reconstructed; wherein, all cell regions include embedded table regions and full table regions.

7. The document table reconstruction method based on text box topology according to claim 6, characterized in that, The step of reconstructing the table in the target document image based on the cell position relationships of all cell regions specifically includes: Create a table border around the outer edge of all the cell ranges; Draw lines along each edge of each extended text box and extend them outward from both ends until they intersect the table border. If a line segment passes through an extended text box, cut off the part of the line segment that passes through the extended text box and keep the rest of the line segment. If there are no extended text boxes between all parallel line segments with the same starting position, and the distance between the parallel line segments with the same starting position is less than the height of the text box, then the parallel line segments with the same starting position are merged into one line segment to complete the cell reconstruction of the table. The text in the expanded block text boxes within each reconstructed cell is merged into the cell text.

8. The document table reconstruction method based on text box topology according to claim 7, characterized in that, Also includes: If any extended block text box contains a diagonal line, the cell containing the extended block text box is divided into multiple sub-cell areas according to the diagonal line, and all text boxes within the sub-cell area are merged into the sub-cell text.

9. A document table recognition device based on text box topology, characterized in that, The document table recognition device includes: The character recognition module is used to obtain the position information of the text boxes in the target document image; The mask acquisition module is used to mask all text box areas of the target document image based on the position information of each text box to obtain a mask image. The line segment recognition module is used to identify all line segments in the mask image using computer vision detection methods to obtain a set of line segments. The block text box acquisition module is used to merge text boxes in the text box set that do not have an inclusion relationship and do not intersect with any horizontal or vertical line segment in the line segment set to form a block text box set. The expansion module is used to determine the global minimum bounding rectangle of all block text boxes in the block text box set, obtain the global boundary, and expand each block text box outward in the horizontal and vertical directions respectively until the expansion stopping condition is met, thus obtaining the expanded block text box. The table region recognition module is used to determine all cell regions based on the position of each extended block text box.

10. A document table reconstruction device based on text box topology, characterized in that, The document table reconstruction device includes the document table recognition device based on text box topology and the table reconstruction module as described in claim 9, wherein: The table reconstruction module is used to reconstruct the table in the target document image based on the cell position relationship of all cell regions; wherein, all cell regions include embedded table regions and full table regions.

Images