A table recognition method and device, electronic equipment and storage medium

Abstract

The application provides a table recognition method and device, electronic equipment and storage medium, wherein the table recognition method comprises: acquiring a to-be-recognized image; extracting a straight line segment endpoint in the to-be-recognized image; for each straight line segment endpoint, acquiring a to-be-fitted point of each straight line segment endpoint according to a preset to-be-fitted point screening rule; fitting the straight line segment endpoint and the to-be-fitted point of the straight line segment endpoint to acquire a first candidate cell corner point; merging the first candidate cell corner point according to a candidate corner point merging rule to acquire a second candidate cell corner point; and reconstructing a table in the to-be-recognized image based on the second candidate cell corner point. Through fitting of the candidate cell corner point, the extraction accuracy of the candidate cell corner point is improved; meanwhile, the fitted candidate cell corner point is also subjected to merging processing, and the extraction accuracy of the candidate cell corner point is further improved, so that the recognition effect of the above table recognition method is improved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and more specifically, to a table recognition method, apparatus, electronic device, and storage medium. Background Technology

[0002] Existing technologies for table recognition generally categorize them into traditional and non-traditional methods. Traditional table detection and recognition methods include, for example, using Hough transform to detect straight lines in an image and then using the intersections of these lines to determine rectangles, thereby achieving table recognition. Non-traditional table detection methods include, for example, using deep learning methods for table recognition.

[0003] When using the traditional table recognition method described above, too many parameters need to be adjusted, resulting in high time complexity for line detection and a low line detection rate. This makes it difficult to accurately reconstruct the table in the image, leading to poor table recognition performance.

[0004] When using the above-mentioned non-traditional table recognition methods for table recognition, the recognition accuracy depends on the training data. However, in actual use, the quantity and quality of the training data are often not guaranteed, resulting in poor recognition performance of the above-mentioned non-traditional table recognition methods. Summary of the Invention

[0005] The purpose of this application is to provide a table recognition method, apparatus, electronic device, and storage medium to improve the table recognition effect.

[0006] In a first aspect, embodiments of this application provide a table recognition method, comprising: acquiring an image to be recognized; extracting endpoints of line segments in the image to be recognized; for each endpoint of the line segment, obtaining a fitting point for each endpoint of the line segment according to a preset fitting point filtering rule; fitting the endpoints of the line segments and the fitting points of the endpoints of the line segments to obtain first candidate cell corner points; merging the first candidate cell corner points according to a candidate corner point merging rule to obtain second candidate cell corner points; and reconstructing a table in the image to be recognized based on the second candidate cell corner points.

[0007] In the implementation of the above scheme, the extraction accuracy of candidate cell corner points is improved by fitting the candidate cell corner points; at the same time, the fitted candidate cell corner points are merged to further improve the extraction accuracy of candidate cell corner points, thereby improving the recognition effect of the above table recognition method.

[0008] In one implementation of the first aspect, the step of obtaining the fitting point for each line segment endpoint according to a preset fitting point filtering rule includes: obtaining line segment endpoints within a first distance threshold from the current line segment endpoint; determining the line segment where the current line segment endpoint is located, and the line segment where the line segment endpoint within the first distance threshold of the current line segment endpoint is located; if the angle between the line segment where the current line segment endpoint is located and the line segment where the line segment endpoint within the first distance threshold of the current line segment endpoint is located satisfies a preset angle threshold, then the line segment endpoint within the first distance threshold of the current line segment endpoint is determined as the fitting point for the current line segment endpoint.

[0009] In the implementation of the above scheme, candidate cell corners are obtained by fitting points near the endpoints of the line segments that meet the requirements of cell corners, thereby improving the extraction accuracy of cell corners and also improving the table recognition effect of the above table recognition method.

[0010] In one implementation of the first aspect, the step of merging the first candidate cell corners according to the candidate corner merging rule to obtain the second candidate cell corner includes: for each first candidate cell corner, obtaining first candidate cell corners within a second distance threshold from the current first candidate cell corner; merging the current first candidate cell corner with the second candidate cell corners within the second distance threshold from the current first candidate cell corner; performing repeated iterations until the iteration exit condition is met to obtain the second candidate cell corner.

[0011] In the implementation of the above scheme, by merging the corner points of the first candidate cell, only the corner point of the second candidate cell exists near each corner point of the second candidate cell. This eliminates the need to judge the validity of the corner points of the candidate cells when reconstructing the table, thus improving the table recognition efficiency of the above table recognition method. At the same time, since only the corner point of the second candidate cell exists near each corner point of the second candidate cell, the reconstructed table will not have redundant table lines, further improving the table recognition effect of the above table recognition method.

[0012] In one implementation of the first aspect, reconstructing the table in the image to be identified based on the second candidate cell corner points includes:

[0013] For each second candidate cell corner point, the current second candidate cell corner point is determined as a certain type of cell corner point. Based on the preset corner point type judgment rule, other types of corner points of the cells containing the current second candidate cell corner point are determined, and the reconstruction of the cells containing the second candidate cell corner point is completed. After the reconstruction of all the cells containing the second candidate cell corner points is completed, the table reconstruction is completed.

[0014] In the implementation of the above scheme, the four corner points of the cell are determined by determining the cell corner point type. The four corner points are used to reconstruct the cell, and then the entire table is reconstructed. The reconstruction efficiency is high, which improves the recognition efficiency of the above table recognition method.

[0015] In one implementation of the first aspect, before reconstructing the table in the image to be identified based on the second candidate cell corner points, the method further includes: for each second candidate cell corner point, determining the number of second candidate cell corner points in the same row as the current second candidate cell corner point; for each second candidate cell corner point, determining the number of second candidate cell corner points in the same column as the current second candidate cell corner point; and filtering the second candidate cell corner points according to the second candidate cell corner point filtering rules.

[0016] In the implementation of the above scheme, the number of corner points of the second candidate cells in the same row or column is used to filter the second candidate cells, removing the second candidate cell corner points that are theoretically impossible to be cell corner points. This makes the subsequent reconstruction of the table based on the corner points of the second candidate cells more efficient and effective.

[0017] In one implementation of the first aspect, before determining the number of second candidate cell corner points that are in the same row as the current second candidate cell corner point, the method further includes: extracting the tilt angle of the line segments in the image to be identified; and determining the tilt angle θ of the image to be identified based on the tilt angle of the line segments.

[0018] The step of determining the number of second candidate cell corners that are in the same row as the current second candidate cell corner includes: for each second candidate cell corner, obtaining a straight line A1 with a slope of tanθ that passes through the current second candidate cell corner; if the distance between a second candidate cell corner and the straight line A1 meets a third distance threshold, then the second candidate cell corner is determined as a second candidate cell corner that is in the same row as the current second candidate cell corner; and counting the number of second candidate cell corners that are in the same row as the current second candidate cell corner.

[0019] Determining the number of second candidate cell corners that are in the same column as the current second candidate cell corner includes: for each second candidate cell corner, obtaining the slope passing through the current second candidate cell corner as... If the distance between a corner point of a second candidate cell and the line A2 satisfies the fourth distance threshold, then the corner point of the second candidate cell is determined as the corner point of the second candidate cell in the same column as the current corner point of the second candidate cell; count the number of corner points of the second candidate cell in the same column as the current corner point of the second candidate cell.

[0020] In the implementation of the above scheme, the tilt angle of the image to be identified, as well as the third and fourth distance thresholds, are used to filter the second candidate cell corners that are in the same row and column as the current second candidate cell corners. This allows for the counting of the number of second candidate cell corners in the same row and column, improving the error tolerance of filtering the second candidate cell corners and thus improving the effect of reconstructing the table using the second candidate cell corners.

[0021] In one implementation of the first aspect, before extracting the endpoints of the line segments in the image to be identified, the method further includes: preprocessing the image to be identified; the preprocessing includes grayscale processing and normalization processing.

[0022] In the above scheme, by preprocessing the image to be recognized, the subsequent steps can extract the required data from the image more quickly, thereby improving the recognition efficiency of the above table recognition method.

[0023] Secondly, embodiments of this application provide a table recognition device, comprising: a to-be-recognized image acquisition module for acquiring an image to be recognized; a line segment endpoint extraction module for extracting line segment endpoints from the image to be recognized; a to-be-fitted point acquisition module for acquiring a to-be-fitted point for each line segment endpoint according to a preset to-be-fitted point filtering rule; a first candidate cell corner point acquisition module for fitting the line segment endpoints and the to-be-fitted point of the line segment to acquire a first candidate cell corner point; a second candidate cell corner point acquisition module for merging the first candidate cell corner points according to a candidate corner point merging rule to acquire a second candidate cell corner point; and a table reconstruction module for reconstructing a table in the image to be recognized based on the second candidate cell corner points.

[0024] Thirdly, embodiments of this application provide a computer-readable storage medium storing computer program instructions, which, when read and executed by a processor, perform the method provided in the first aspect or any possible implementation thereof.

[0025] Fourthly, embodiments of this application provide an electronic device, including: a memory and a processor, wherein the memory stores computer program instructions, and the computer program instructions are read and executed by the processor to perform the method provided in the first aspect or any possible implementation of the first aspect. Attached Figure Description

[0026] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0027] Figure 1 A flowchart illustrating the table recognition method provided in this application embodiment;

[0028] Figure 2 This is a schematic diagram of the structure of the table recognition device provided in the embodiments of this application;

[0029] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0030] The technical solutions of the embodiments of this application will now be described with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solutions of this application, and are therefore merely examples and should not be used to limit the scope of protection of this application.

[0031] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the application; the terms “comprising” and “having”, and any variations thereof, in the specification, claims, and foregoing description of the drawings are intended to cover non-exclusive inclusion.

[0032] In the description of the embodiments of this application, technical terms such as "first" and "second" are used only to distinguish different objects and should not be construed as indicating or implying relative importance or implicitly specifying the number, specific order, or primary and secondary relationship of the indicated technical features. In the description of the embodiments of this application, "multiple" means two or more, unless otherwise explicitly defined.

[0033] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0034] Please see Figure 1 This application provides a table recognition method, including:

[0035] Step S110: Obtain the image to be recognized.

[0036] It is understood that the image to be identified in the embodiments of this application is an image containing a table, and the purpose of the above-mentioned table recognition method is to identify the table in the image to be identified.

[0037] It should be noted that the image to be identified obtained in step S110 can be an image to be identified directly captured by the device performing the above table recognition method, or it can be an image to be identified sent by other devices to the device performing the above table recognition method.

[0038] As an optional implementation of the above-described table recognition method, after acquiring the image to be recognized, the method further includes: preprocessing the image to be recognized. This implementation may include, for example, performing grayscale conversion and normalization on the image to be recognized. For example, grayscale conversion can be performed using the OpenCV color space conversion function `cv2.cvtColor()`. Normalization can be performed using OpenCV's `cv2.resize` function.

[0039] In the above scheme, by preprocessing the image to be recognized, the subsequent steps can extract the required data from the image more quickly, thereby improving the recognition efficiency of the above table recognition method.

[0040] Step S120: Extract the endpoints of the line segments in the image to be identified.

[0041] It can be understood that each complete cell is composed of four line segments, and each line segment is composed of two endpoints and a line connecting these two endpoints. Therefore, after identifying the line segments in the image to be identified in step S120, the line segments can be stored by storing their endpoints. For example, the coordinates of the start and end points of the line segments can be stored in a two-dimensional list {[(x0,y0),(x1,y1)],{[(x2,y2),(x3,y3)],…}, where (x0,y0) and (x2,y2) represent the start points of the line segments, and (x1,y1) and (x3,y3) represent the end points. For easier subsequent calculations, all the endpoints of the line segments can also be stored in a one-dimensional list {(x0,y0),(x1,y1),…}.

[0042] Step S120 can employ a line detection algorithm to detect line segments in the image to be recognized and extract the endpoint data of the line segments. For example, this implementation may use the LSD line detection algorithm, FLD line detection algorithm, EDlines line detection algorithm, LSWMS line detection algorithm, CannyLines line detection algorithm, MCMLSD line detection algorithm, or LSM line detection algorithm to detect line segments in the image to be recognized and extract the endpoint coordinate data of the line segments.

[0043] Step S120 involves using a line detection algorithm to detect line segments in the image to be recognized. The length of each detected line is calculated, and line segments shorter than a preset length threshold are filtered out. The reason for this filtering is that the line detection algorithm might misidentify text in the table as a line. Filtering line segments shorter than the preset length threshold reduces unnecessary computation time and speeds up table recognition. It should be noted that the length of a line segment can be directly calculated from the coordinates of its endpoints.

[0044] Step S130: For each line segment endpoint, obtain the fitting point for each line segment endpoint according to the preset fitting point filtering rules.

[0045] Since there may be unfiltered or noise points near the actual cell corners, the endpoints of line segments cannot be directly used as cell corners. Therefore, the embodiments of this application use a fitting method to reduce the influence of noise points.

[0046] As an optional implementation of the above table recognition method, step S130 includes: obtaining the endpoint of the line segment within a first distance threshold from the endpoint of the current line segment; determining the line segment where the endpoint of the current line segment is located, and the line segment where the endpoint of the line segment within the first distance threshold from the endpoint of the current line segment is located; if the angle between the line segment where the endpoint of the current line segment is located and the line segment where the endpoint of the line segment within the first distance threshold from the endpoint of the current line segment meets a preset angle threshold, then the endpoint of the line segment within the first distance threshold from the endpoint of the current line segment is determined as the fitting point of the endpoint of the current line segment. For example, this implementation involves: using the current line segment endpoint as the center and a first distance threshold as the search radius, searching for line segment endpoints within the first distance threshold of the current line segment endpoint; determining the line segment where the current line segment endpoint is located, and the line segment where the line segment endpoints within the first distance threshold of the current line segment endpoint are located; determining the angle between the line segment where the current line segment endpoint is located and the line segment where the line segment endpoints within the first distance threshold of the current line segment endpoint are located; if the angle between the line segment where the current line segment endpoint is located and the line segment where the line segment endpoints within the first distance threshold of the current line segment endpoint are located satisfies a preset angle threshold, then the line segment endpoint within the first distance threshold of the current line segment endpoint is determined as the fitting point of the current line segment endpoint.

[0047] Based on the characteristic that at least two perpendicular line segments exist at the corner of a cell, this application's embodiments define a perpendicular line segment endpoint as follows: if the angle between the line segment containing the endpoint of the current line segment (within a first distance threshold) and the line segment containing the current line segment endpoint is 90° or close to 90°, then the line segment containing the endpoint of the current line segment (within a first distance threshold) and the line segment containing the current line segment endpoint are considered perpendicular. It is understood that the preset angle threshold does not have to be a specific angle value; it can be represented by the sine or cosine of the angle. For example, if the preset angle threshold is set to a cosine value less than 0.08, then if the cosine of the angle between the line segment containing the current line segment endpoint and the line segment containing the endpoint of the current line segment (within a first distance threshold) is less than 0.08, then the endpoint of the line segment (within a first distance threshold) can be determined as the fitting point for the current line segment endpoint.

[0048] It should be noted that, taking line a as [(x0,y0),(x1,y1)] and line b as [(x2,y2),(x3,y3)] as an example, the method for calculating the cosine value between lines a and b is as follows:

[0049]

[0050] In the implementation of the above scheme, candidate cell corners are obtained by fitting points near the endpoints of the line segments that meet the requirements of cell corners, thereby improving the extraction accuracy of cell corners and also improving the table recognition effect of the above table recognition method.

[0051] Step S140: Fit the endpoints of the line segment and the points to be fitted on the line segment to obtain the first candidate cell corner points;

[0052] The fitting of a line segment endpoints and the points to be fitted to those endpoints can be done directly using the mean or by using a weighted average. These two methods are described in detail below:

[0053] The first implementation method directly uses the mean for fitting. For example, this implementation method includes:

[0054] If the set of points to be fitted at the endpoint (x, y) of the current line segment is {X1, X2, ..., X...} n}, then the first candidate cell corner point (x) is fitted using the mean method. new ,y new )for:

[0055]

[0056]

[0057] Among them, X i [0] represents X in the set of points to be fitted. i x-coordinate; X i [1] represents X in the set of points to be fitted. i The ordinate.

[0058] The second implementation method uses a weighted average for fitting. For example, this implementation method includes:

[0059] If the set of points to be fitted at the endpoint (x, y) of the current line segment is {X1, X2, ..., X...} n}, then the first candidate cell corner point (x) is fitted using a weighted average method. new ,y new )for:

[0060]

[0061]

[0062]

[0063] in, X is the midpoint of the set of points to be fitted. iThe weight of point X can be determined as follows: i The distance from the endpoint of the current line segment is used to determine this, or it can be determined by: using point X... i The value is determined by the deviation between the angle between the current line segment and the line segment containing the endpoint of the current line segment and 90°. The specific selection method can be determined according to the actual situation.

[0064] Step S150: According to the candidate corner merging rules, merge the corners of the first candidate cell to obtain the corners of the second candidate cell.

[0065] Since the first candidate cell corner points obtained after fitting in steps S130 and S140 may still contain misdetected points or noise points, it is necessary to merge the first candidate cell corner points obtained in step S140.

[0066] As an optional implementation of the above table recognition method, step S150 includes: for each first candidate cell corner point, obtaining first candidate cell corner points within a second distance threshold from the current first candidate cell corner point; merging the current first candidate cell corner point with second candidate cell corner points within the second distance threshold from the current first candidate cell corner point; repeating the iteration until the iteration exit condition is met, and obtaining the second candidate cell corner point. For example, this implementation involves: using the current first candidate cell corner point as the center and the second distance threshold as the search radius, searching for first candidate cell corner points within the second distance threshold from the current first candidate cell corner point; merging the current first candidate cell corner point with second candidate cell corner points within the second distance threshold from the current first candidate cell corner point using a mean merging method; repeating the above steps until the iteration exit condition is met, and obtaining the second candidate cell corner point.

[0067] The above-mentioned method of averaging is used for example:

[0068] If the set of points to be merged for the current first candidate cell corner point (x, y) is {Y1, Y2, ..., Y...} n}, then the second candidate cell corner point (x) is obtained by averaging. sum ,y sum )for:

[0069]

[0070]

[0071] Among them, Y i [0] represents Y in the set of points to be merged. i The x-coordinate; Y i [1] represents Y in the set of points to be merged.i The ordinate.

[0072] The above iteration exit condition is: the corner point of the second candidate cell (x sum ,y sum The value of ) no longer changes, or the preset maximum number of iterations is reached.

[0073] In the implementation of the above scheme, by merging the corner points of the first candidate cell, only the corner point of the second candidate cell exists near each corner point of the second candidate cell. This eliminates the need to judge the validity of the corner points of the candidate cells when reconstructing the table, thus improving the table recognition efficiency of the above table recognition method. At the same time, since only the corner point of the second candidate cell exists near each corner point of the second candidate cell, the reconstructed table will not have redundant table lines, further improving the table recognition effect of the above table recognition method.

[0074] As an optional implementation of the above table recognition method, before step S160, the method further includes: for each second candidate cell corner point, determining the number of second candidate cell corner points in the same row as the current second candidate cell corner point; for each second candidate cell corner point, determining the number of second candidate cell corner points in the same column as the current second candidate cell corner point; and filtering the second candidate cell corner points according to the second candidate cell corner point filtering rules. For example, for each second candidate cell corner point, after determining the number of second candidate cell corner points in the same row and column as each second candidate cell corner point, determining the maximum number X of second candidate cell corner points in the same row. max And the maximum number Y of the second candidate cell corner points in the same column max When filtering for the corner points of the second candidate cells, obtain the number X of the second candidate cell corner points that are in the same row as the corner points of the second candidate cells. num And the number Y of second candidate cell corner points that are in the same column as the corner point of the second candidate cell. num If X num and Y num If the following formula is not satisfied, the corner point of the second candidate cell is considered a false positive:

[0075]

[0076] Where α is a proportionality parameter.

[0077] It's understandable that the purpose of the second candidate corner filtering rule is to use the number of second candidate points in the same row and column required to construct the table as a constraint. If the predetermined number of points does not exist in the row or column where a second candidate cell corner point is located, then the second candidate cell corner point is considered impossible to be a cell corner point. Based on experience, in X... max and Y max A certain proportion of the second candidate cell corners can be used as the final cell corners to reconstruct the table. Therefore, in this embodiment, a proportion parameter α is set to constrain the second candidate cell corners.

[0078] It should be noted that the aforementioned proportional parameter α is related to the density of the cells. When the cells are sparse, the proportional parameter α takes a larger value, and when the cells are dense, the proportional parameter α takes a smaller value. The specific value can be set according to the actual application.

[0079] In the implementation of the above scheme, the number of corner points of the second candidate cells in the same row or column is used to filter the second candidate cells, removing the second candidate cell corner points that are theoretically impossible to be cell corner points. This makes the subsequent reconstruction of the table based on the corner points of the second candidate cells more efficient and effective.

[0080] As an optional implementation of the above table recognition method, the method for determining the number of second candidate cell corner points that are in the same row and column as the current second candidate cell corner point is as follows: obtain the tilt angle of the image to be recognized, and obtain the number of second candidate cell corner points that are in the same row and column as the current second candidate cell corner point based on the tilt angle of the image to be recognized. For example, this implementation method includes:

[0081] Extract the tilt angle of the line segments in the image to be identified; based on the tilt angle of the line segments, determine the tilt angle θ of the image to be identified;

[0082] For each corner point of the second candidate cell, obtain the points that have passed through the current second candidate cell. A1 is a line with a slope of tanθ at its corner points:

[0083]

[0084] If a second candidate cell corner point If the distance between the second candidate cell corner and the line A1 satisfies the third distance threshold, i.e., satisfies the following formula, then the second candidate cell corner is determined as the second candidate cell corner in the same row as the current second candidate cell corner.

[0085]

[0086] Where ε3 is the third distance threshold.

[0087] Count the number of second candidate cell corner points that are in the same row as the current second candidate cell corner point;

[0088] For each corner point of the second candidate cell, obtain the corner points that pass through the current second candidate cell. The slope is Line A2:

[0089]

[0090] If a second candidate cell corner point If the distance between the second candidate cell corner point and the line A2 satisfies the fourth distance threshold, that is, satisfies the following formula, then the second candidate cell corner point is determined as the second candidate cell corner point in the same column as the current second candidate cell corner point;

[0091]

[0092] Where ε4 is the fourth distance threshold.

[0093] Count the number of second candidate cell corner points that are in the same column as the current second candidate cell corner point.

[0094] The method described above for determining the tilt angle θ of the image to be recognized based on the tilt angle of a straight line segment is as follows: the range of tilt angles of the straight line segment is determined as follows. The range is divided into several equal parts. In this embodiment, it is divided into 480 parts, with each part having an angle of [missing information]. After obtaining the tilt angle of each line segment, the tilt angle of each line segment is divided into corresponding intervals. Finally, the number of line segments contained in each angle interval is counted, and the median value of the angle interval with the largest number of line segments is taken as the tilt angle of the image to be identified.

[0095] It should be noted that when counting the number of line segments contained in each angle interval, the weight of the line segments can also be added. The method for calculating the weight of the line segments is as follows:

[0096]

[0097] Where length is a constant, usually taken as the shortest length value in the line segment.

[0098] In the implementation of the above scheme, the tilt angle of the image to be identified, as well as the third and fourth distance thresholds, are used to filter the second candidate cell corners that are in the same row and column as the current second candidate cell corners. This allows for the counting of the number of second candidate cell corners in the same row and column, improving the error tolerance of filtering the second candidate cell corners and thus improving the effect of reconstructing the table using the second candidate cell corners.

[0099] Step S160: Reconstruct the table in the image to be identified based on the corner points of the second candidate cells.

[0100] As an optional implementation of the above table recognition method, step S160 includes: for each second candidate cell corner point, determining the current second candidate cell corner point as a certain type of cell corner point, determining other types of corner points of the cell containing the current second candidate cell corner point based on a preset corner point type judgment rule, and completing the reconstruction of the cells containing the second candidate cell corner points; after completing the reconstruction of all cells containing the second candidate cell corner points, the table reconstruction is completed. An example of this implementation is:

[0101] The current second candidate cell corner is determined as the top-left corner of the cell. A list {L,U,R,D} is constructed to represent the type of the point, where L, U, R, and D represent the left, up, right, and down directions of the second candidate cell corner, respectively. The values ​​of L, U, R, and D are 0 or 1. If the value is 0, it proves that the second candidate cell corner does not have a line segment in this direction. If the value is 1, it proves that the second candidate cell corner has a line segment in this direction.

[0102] To determine if the current second candidate cell corner meets the criteria for being the top-left corner of a cell, the criteria are: the values ​​of elements R and D in the list {L,U,R,D} of the current second candidate cell corners are both 1, meaning there are line segments to the right and down of the current second candidate cell corner. If the current second candidate cell corner meets the criteria, then the top-right and bottom-left corners of the cell with the current second candidate cell corner as its top-left corner are determined sequentially. For example, to determine the top-right corner: among the second candidate cell corners in the same row as the current second candidate cell corner, select those whose values ​​of L and D in the list {L,U,R,D} are both 1. Then, among these, select the second candidate cell corner closest to the current second candidate cell corner as the top-right corner. Similarly, the bottom-left corner can be determined.

[0103] Finally, determine the bottom right corner of the cell, for example:

[0104] Among the second candidate cell corners in the same row as the lower left corner, filter the second candidate cell corners that satisfy the condition that the value of element L and element U in the list {L,U,R,D} is 1. Then, select the second candidate cell corner that is closest to the lower left corner as the first candidate lower right corner from the second candidate cell corners that satisfy the condition that the value of element L and element U in the list {L,U,R,D} is 1.

[0105] Among the second candidate cell corners that are in the same column as the top right corner, filter the second candidate cell corners that satisfy the condition that the value of element L and element U in the list {L,U,R,D} is 1. Then, select the second candidate cell corner that is closest to the top right corner from the second candidate cell corners that satisfy the condition that the value of element L and element U in the list {L,U,R,D} is 1 as the second candidate bottom right corner.

[0106] If the first candidate bottom right corner point and the second candidate bottom right corner point are the same point, then that point is determined as the bottom right corner point.

[0107] If the bottom right corner of the first candidate cell and the bottom right corner of the second candidate cell are not the same point, the bottom right corner of the cell containing the current second candidate cell is determined as follows:

[0108] If the top-left corner of the cell is (x1, y1), the bottom-left corner is (x2, y2), and the top-right corner is (x3, y3), and the bottom-right corner to be found is (x4, y4), the method is as follows:

[0109]

[0110] The table reconstruction is complete after all cells containing the corner points of the second candidate cell have been reconstructed.

[0111] In the implementation of the above scheme, the four corner points of the cell are determined by determining the cell corner point type. The four corner points are used to reconstruct the cell, and then the entire table is reconstructed. The reconstruction efficiency is high, which improves the recognition efficiency of the above table recognition method.

[0112] Please see Figure 2 Based on the same inventive concept, this application also provides a table recognition device 200, comprising:

[0113] Image acquisition module 210 is used to acquire the image to be recognized;

[0114] The line segment endpoint extraction module 220 is used to extract the endpoints of line segments in the image to be identified;

[0115] The module 230 for obtaining the fitting point is used to obtain the fitting point of each endpoint of the line segment according to the preset fitting point filtering rules.

[0116] The first candidate cell corner point acquisition module 240 is used to fit the endpoints of the line segment and the points to be fitted to the endpoints of the line segment to obtain the first candidate cell corner points;

[0117] The second candidate cell corner point acquisition module 250 is used to merge the corner points of the first candidate cell according to the candidate corner point merging rules to obtain the second candidate cell corner point;

[0118] The table reconstruction module 260 is used to reconstruct the table in the image to be identified based on the corner points of the second candidate cells.

[0119] As an optional implementation of the above-mentioned table recognition device, the fitting point acquisition module 230 specifically performs the following steps: acquiring the endpoint of a straight line segment within a first distance threshold from the endpoint of the current straight line segment; determining the straight line segment where the endpoint of the current straight line segment is located, and the straight line segment where the endpoint of the straight line segment within the first distance threshold from the endpoint of the current straight line segment is located; if the angle between the straight line segment where the endpoint of the current straight line segment is located and the straight line segment where the endpoint of the straight line segment within the first distance threshold from the endpoint of the current straight line segment satisfies a preset angle threshold, then the endpoint of the straight line segment within the first distance threshold from the endpoint of the current straight line segment is determined as the fitting point of the endpoint of the current straight line segment.

[0120] As an optional implementation of the above-mentioned table recognition device, the second candidate cell corner acquisition module 250 specifically performs the following steps: for each first candidate cell corner, acquire first candidate cell corners within a second distance threshold from the current first candidate cell corner; merge the current first candidate cell corner with the second candidate cell corners within the second distance threshold from the current first candidate cell corner; perform repeated iterations until the iteration exit condition is met, and acquire the second candidate cell corner.

[0121] As an optional implementation of the above-mentioned table recognition device, the table reconstruction module 260 specifically performs the following steps: for each second candidate cell corner point, the current second candidate cell corner point is determined as a certain type of cell corner point, and other types of corner points of the cell containing the current second candidate cell corner point are determined based on the preset corner point type judgment rule, thereby completing the reconstruction of the cell containing the second candidate cell corner point; after completing the reconstruction of all the cells containing the second candidate cell corner points, the table reconstruction is completed.

[0122] As an optional implementation of the above-mentioned table recognition device, the table recognition device 200 further includes: a second candidate corner filtering module, used to determine, for each second candidate cell corner, the number of second candidate cell corners in the same row as the current second candidate cell corner; to determine, for each second candidate cell corner, the number of second candidate cell corners in the same column as the current second candidate cell corner; and to filter the second candidate cell corners according to the second candidate corner filtering rules.

[0123] As an optional implementation of the above-mentioned table recognition device, the second candidate corner filtering module further includes, before determining the number of second candidate cell corners in the same row as the current second candidate cell corner, extracting the tilt angle of the straight line segments in the image to be recognized; and determining the tilt angle θ of the image to be recognized based on the tilt angle of the straight line segments.

[0124] As an optional implementation of the above-mentioned table recognition device, the second candidate corner filtering module determines the number of second candidate cell corners in the same row as the current second candidate cell corner, including: for each second candidate cell corner, obtaining a straight line A1 with a slope of tanθ passing through the current second candidate cell corner; if the distance between a second candidate cell corner and the straight line A1 meets a third distance threshold, then the second candidate cell corner is determined as a second candidate cell corner in the same row as the current second candidate cell corner; and counting the number of second candidate cell corners in the same row as the current second candidate cell corner.

[0125] As an optional implementation of the aforementioned table recognition device, the second candidate corner filtering module determines the number of second candidate cell corners that are in the same column as the current second candidate cell corner, including:

[0126] For each corner point of the second candidate cell, obtain the slope passing through the current corner point of the second candidate cell as... If the distance between a corner point of a second candidate cell and the line A2 satisfies the fourth distance threshold, then the corner point of the second candidate cell is determined as the corner point of the second candidate cell in the same column as the current corner point of the second candidate cell; count the number of corner points of the second candidate cell in the same column as the current corner point of the second candidate cell.

[0127] As an optional embodiment of the above-mentioned table recognition device, the table recognition device 200 further includes: a preprocessing module for preprocessing the image to be recognized.

[0128] As an optional implementation of the above-mentioned table recognition device, the preprocessing module includes: a grayscale processing unit for performing grayscale processing on the image to be recognized; and a normalization processing unit for performing normalization processing on the image to be recognized.

[0129] Figure 3 This is a schematic diagram of an electronic device provided in an embodiment of this application. (Refer to...) Figure 3 The electronic device 300 includes a processor 310, a memory 320, and a communication interface 330. These components are interconnected and communicate with each other via a communication bus 340 and / or other forms of connection mechanism (not shown).

[0130] The memory 320 includes one or more (only one is shown in the figure), which may be, but is not limited to, Random Access Memory (RAM), Read Only Memory (ROM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), etc. The processor 310 and other possible components may access the memory 320 to read and / or write data therein.

[0131] Processor 310 includes one or more (only one is shown in the figure), which can be an integrated circuit chip with signal processing capabilities. The processor 310 described above can be a general-purpose processor, including a central processing unit (CPU), a microcontroller unit (MCU), a network processor (NP), or other conventional processors; it can also be a special-purpose processor, including a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0132] Communication interface 330 includes one or more (only one is shown in the figure) and can be used to communicate directly or indirectly with other devices to exchange data. For example, communication interface 330 can be an Ethernet interface; it can be a mobile communication network interface, such as an interface for 3G, 4G, or 5G networks; or it can be other types of interfaces with data transmission and reception functions.

[0133] One or more computer program instructions may be stored in the memory 320, and the processor 310 may read and run these computer program instructions to implement the table recognition method provided in the embodiments of this application and other desired functions.

[0134] Understandable. Figure 3 The structure shown is for illustrative purposes only; the electronic device 300 may also include components that are more advanced than those shown. Figure 3 The more or fewer components shown, or having the same Figure 3 The different configurations shown. Figure 3 The components shown can be implemented using hardware, software, or a combination thereof. For example, electronic device 300 can be a single server (or other device with computing power), a combination of multiple servers, a cluster of a large number of servers, etc., and can be either a physical device or a virtual device.

[0135] This application also provides a computer-readable storage medium storing computer program instructions. These computer program instructions are read and executed by a computer's processor to perform the table recognition method provided in this application. For example, the computer-readable storage medium can be implemented as follows: Figure 3 The memory 320 in the electronic device 300.

[0136] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0137] Furthermore, the units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0138] Furthermore, the functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0139] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A table recognition method, characterized in that, include: Acquire the image to be recognized; Extract the endpoints of the line segments in the image to be identified; For each endpoint of the line segment, the fitting point of each endpoint of the line segment is obtained according to the preset fitting point filtering rules; Fit the endpoints of the line segment and the points to be fitted on the line segment to obtain the first candidate cell corner points; According to the candidate corner merging rules, the corners of the first candidate cell are merged to obtain the corners of the second candidate cell; Based on the corner points of the second candidate cells, the table in the image to be identified is reconstructed; Before reconstructing the table in the image to be identified based on the second candidate cell corner points, the method further includes: for each second candidate cell corner point, determining the number of second candidate cell corner points in the same row as the current second candidate cell corner point; for each second candidate cell corner point, determining the number of second candidate cell corner points in the same column as the current second candidate cell corner point; and filtering the second candidate cell corner points according to the second candidate cell corner point filtering rules. Before determining the number of second candidate cell corner points in the same row as the current second candidate cell corner point, the method further includes: extracting the tilt angle of line segments in the image to be identified; and determining the tilt angle of the image to be identified based on the tilt angle of the line segments. ; Determining the number of second candidate cell corners that are in the same row as the current second candidate cell corner includes: for each second candidate cell corner, obtaining the slope passing through the current second candidate cell corner as... straight line If a corner point of a second candidate cell is perpendicular to the line... If the distance between them meets the third distance threshold, then the second candidate cell corner point is determined as the second candidate cell corner point in the same row as the current second candidate cell corner point; count the number of second candidate cell corner points in the same row as the current second candidate cell corner point; Determining the number of second candidate cell corners that are in the same column as the current second candidate cell corner includes: for each second candidate cell corner, obtaining the slope passing through the current second candidate cell corner as... straight line If a corner point of a second candidate cell is perpendicular to the line... If the distance between them meets the fourth distance threshold, then the corner point of the second candidate cell is determined as the corner point of the second candidate cell in the same column as the current corner point of the second candidate cell; count the number of corner points of the second candidate cell in the same column as the current corner point of the second candidate cell.

2. The table recognition method according to claim 1, characterized in that, The step of obtaining the fitting point for each endpoint of the line segment according to a preset fitting point filtering rule includes: Get the endpoints of the line segments within the first distance threshold from the current line segment endpoint; Determine the line segment containing the endpoint of the current line segment, and the line segment containing the endpoint of the line segment within a first distance threshold from the endpoint of the current line segment; If the angle between the line segment containing the endpoint of the current line segment and the line segment containing the endpoint of the line segment within a first distance threshold from the endpoint of the current line segment satisfies a preset angle threshold, then the endpoint of the line segment within a first distance threshold from the endpoint of the current line segment is determined as the fitting point of the endpoint of the current line segment.

3. The table recognition method according to claim 1, characterized in that, The step of merging the corner points of the first candidate cell according to the candidate corner point merging rules to obtain the corner points of the second candidate cell includes: For each of the first candidate cell corner points, obtain the first candidate cell corner points within the second distance threshold from the current first candidate cell corner point; Merge the current first candidate cell corner point with the second candidate cell corner point within the second distance threshold from the current first candidate cell corner point; Repeat the iteration until the iteration exit condition is met, and obtain the corner point of the second candidate cell.

4. The table recognition method according to claim 1, characterized in that, The step of reconstructing the table in the image to be identified based on the second candidate cell corner points includes: For each second candidate cell corner point, the current second candidate cell corner point is determined as a certain type of cell corner point. Based on the preset corner point type judgment rule, other types of corner points of the cells containing the current second candidate cell corner point are determined, and the reconstruction of the cells containing the second candidate cell corner point is completed. After the reconstruction of all the cells containing the second candidate cell corner points is completed, the table reconstruction is completed.

5. The table recognition method according to claim 1, characterized in that, Before extracting the endpoints of the line segments in the image to be identified, the method further includes: The image to be identified is preprocessed; The preprocessing includes grayscale conversion and normalization.

6. A form recognition device, characterized in that, include: The image acquisition module is used to acquire the image to be recognized. The line segment endpoint extraction module is used to extract the endpoints of line segments in the image to be identified; The module for obtaining the points to be fitted is used to obtain the points to be fitted for each endpoint of the line segment according to a preset filtering rule for the points to be fitted. The first candidate cell corner point acquisition module is used to fit the endpoints of the line segment and the points to be fitted of the line segment to obtain the first candidate cell corner points; The second candidate cell corner point acquisition module is used to merge the corner points of the first candidate cell according to the candidate corner point merging rules to obtain the second candidate cell corner point; The table reconstruction module is used to reconstruct the table in the image to be identified based on the corner points of the second candidate cells; The device further includes: The second candidate corner filtering module is used to determine, for each second candidate cell corner, the number of second candidate cell corners in the same row as the current second candidate cell corner; to determine, for each second candidate cell corner, the number of second candidate cell corners in the same column as the current second candidate cell corner; and to filter the second candidate cell corners according to the second candidate corner filtering rules. Before determining the number of second candidate cell corners in the same row as the current second candidate cell corner, the second candidate corner filtering module further includes: extracting the tilt angle of line segments in the image to be identified; and determining the tilt angle of the image to be identified based on the tilt angle of the line segments. ; The second candidate corner filtering module determines the number of second candidate cell corners in the same row as the current second candidate cell corner, including: for each second candidate cell corner, obtaining the slope passing through the current second candidate cell corner. straight line If a corner point of a second candidate cell is perpendicular to the line... If the distance between them meets the third distance threshold, then the second candidate cell corner point is determined as the second candidate cell corner point in the same row as the current second candidate cell corner point; count the number of second candidate cell corner points in the same row as the current second candidate cell corner point; The second candidate corner filtering module determines the number of second candidate cell corners in the same column as the current second candidate cell corner, including: for each second candidate cell corner, obtaining the slope passing through the current second candidate cell corner. straight line If a corner point of a second candidate cell is perpendicular to the line... If the distance between them meets the fourth distance threshold, then the corner point of the second candidate cell is determined as the corner point of the second candidate cell in the same column as the current corner point of the second candidate cell; count the number of corner points of the second candidate cell in the same column as the current corner point of the second candidate cell.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer program instructions, which, when read and executed by a processor, perform the method as described in any one of claims 1 to 5.

8. An electronic device, characterized in that, include: A memory and a processor, wherein the memory stores computer program instructions, which are read and executed by the processor to perform the method as described in any one of claims 1 to 5.

Images