Table image data analysis method, device and storage medium

Abstract

The present invention discloses a table image data parsing method, device and storage medium, the method comprising: obtaining a table image, performing noise reduction and tilt correction processing on the table image; calculating the position information of each cell in the table image and cropping the cell; identifying the character content in each cell and storing the character content in association with the cell position information. The table image data parsing method of the present invention is implemented to obtain a table image, split the table image into single independent cell image slices, perform text recognition on the cell image slices, and save the cell position information and text information accordingly. Through the steps described in this method, the disadvantage of traditional image recognition that the entire image is recognized as a whole while ignoring the role of the table border line, resulting in the loss of the table's structured information, is solved. Thereby achieving the effect of improving the degree of structuring and reducing data loss when parsing images with tables.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to a table image data parsing method, device and storage medium. Background Art

[0002] In the internet, industrial production, and big data sectors, it's often necessary to identify and parse chart data in order to extract, store, and utilize the data within it. Existing image text recognition technologies, when recognizing images containing tables, only recognize the entire image as a whole, ignoring the role of table borders. This results in lost structural information, leading to parsed data errors or even data loss.

[0003] For example, when merging cells or containing multiple rows of data in the same cell, ignoring the table lines may cause the cell data to be misaligned with other cell data; when the data in adjacent cells are very close to each other, the problem of continuous data reading may occur. Summary of the Invention

[0004] The present invention aims to provide a method, device and storage medium for parsing table image data. The method improves the accuracy of image parsing by processing table information in image files.

[0005] In view of this, the first aspect of the present application provides a table image data parsing method, the method comprising:

[0006] Obtain a table image, and perform noise reduction and tilt correction processing on the table image;

[0007] Calculate the position information of each cell in the table image and crop the cells;

[0008] The character content in each cell is identified and stored in association with the cell position information.

[0009] Optionally, in combination with the first aspect, in a possible implementation, the step of performing noise reduction and tilt correction on the table image specifically includes:

[0010] Binarization, expansion, and corrosion operations are performed on the table image to remove text and interference information in the table image;

[0011] Obtain the horizontal lines and vertical lines in the table image to obtain a horizontal line graph, a vertical line graph and an original image respectively;

[0012] Find the average inclination angle of the horizontal or vertical lines in a horizontal or vertical line graph;

[0013] The horizontal line graph, the vertical line graph and the original image are tilt-corrected according to the average tilt angle.

[0014] Optionally, in combination with the first aspect, in a possible implementation, the step of calculating the position information of each cell in the table image and cropping the cell includes:

[0015] Calculate the pixel points with the same coordinates in the horizontal and vertical lines, and obtain all the intersection points of the horizontal and vertical lines;

[0016] Keep the coordinates of all intersection points in order as an intersection array;

[0017] Save the coordinates of all pixel points in the horizontal and vertical line graphs as a table border array.

[0018] Optionally, in combination with the first aspect, in a possible implementation, the step of maintaining the coordinates of all intersections in order as an intersection array specifically includes:

[0019] Obtaining coordinate information of all intersections, dividing all intersections with the same vertical coordinate into a row, and dividing all intersections into multiple rows;

[0020] The intersection points in each row are sorted in ascending order according to the horizontal coordinate, and the intersection points in each row are sorted in ascending order according to the vertical coordinate, to form the intersection point array.

[0021] Optionally, in combination with the first aspect, in a possible implementation, the step of calculating the position information of each cell in the table image and cropping the cell specifically includes:

[0022] Calculate the upper left corner vertex, upper right corner vertex, lower right corner vertex and lower left corner vertex of the cell from the intersection array and the table border array;

[0023] The cell is cropped according to the coordinates of the upper left vertex, the upper right vertex, the lower right vertex, and the lower left vertex.

[0024] Optionally, in combination with the first aspect, in a possible implementation, the step of calculating the upper left corner vertex of the cell from the intersection array and the table border array specifically includes:

[0025] Take the coordinate information of the mth point in the nth row intersection array and record it as point A(X m , Y n );

[0026] Get the coordinate information of the m+1th point in the nth row intersection array (X m+1 , Y n );

[0027] Calculate the center point coordinates of the mth point in the nth row and the m+1th point in the nth row ((X m +X m+1 ) / 2,Yn );

[0028] Determine whether the center point coordinates ((X m +X m+1 ) / 2, Yn);

[0029] If it exists, point A is considered to be the upper left vertex;

[0030] If it does not exist, discard point A, take the m+1th point in the intersection array of the nth row as point A and repeat the above steps.

[0031] Optionally, in combination with the first aspect, in a possible implementation, the step of calculating the upper right corner vertex of the cell from the intersection array and the table border array specifically includes:

[0032] Get the coordinates of the upper left vertex A (X m , Y n );

[0033] Take the coordinate information of the m+1th point in the intersection array of the nth row and record it as point B(X m+1 , Y n );

[0034] Get the coordinate information of the m+1th point in the n+1th row intersection array (X m+1 , Y n+1 );

[0035] Calculate the center point coordinates (X m+1 ,(Y n +Y n+1 ) / 2);

[0036] Determine whether the center point coordinates (X m+1 ,(Y n +Y n+1 ) / 2);

[0037] If it exists, point B is considered to be the upper right vertex;

[0038] If it does not exist, discard point B, take the m+2th point in the intersection array of the nth row as point B and repeat the above steps.

[0039] Optionally, in combination with the first aspect, in a possible implementation, the step of calculating the vertex of the lower right corner of the cell from the intersection array and the table border array specifically includes:

[0040] Get the coordinates of the upper left vertex A (X m , Y n ) and the coordinates of the upper right vertex B (Xm+1 , Y n );

[0041] Take the point in the intersection array of row n+1 that has the same or similar horizontal coordinate as point B and record it as point C (X m+1 , Y n+1 );

[0042] Take the point in the intersection array of row n+1 that has the same or similar horizontal coordinate as point A and record it as point D(X m , Y n+1 );

[0043] Calculate the center point coordinates of point C and point D ((X m +X m+1 ) / 2,Y n+1 );

[0044] Determine whether the center point coordinates ((X m +X m+1 ) / 2,Y n+1 );

[0045] If it exists, point C is considered to be the lower right vertex;

[0046] If it does not exist, discard point C, take the point in the intersection array of the n+2th row that has the same or similar horizontal coordinate as point B as point C, and repeat the above steps.

[0047] Optionally, in combination with the first aspect, in a possible implementation, the step of calculating the lower left corner vertex of the cell from the intersection array and the table border array specifically includes:

[0048] Get the coordinates of the upper left vertex A (X m , Y n ), coordinates of the upper right corner vertex B (X m+1 , Y n ) and the coordinates of the lower right vertex C (X m+1 , Y n+1 );

[0049] The coordinates of the lower left corner vertex D (X) are calculated based on the coordinates of the upper left corner vertex A, the upper right corner vertex B and the lower right corner vertex C. m , Y n+1 );

[0050] Calculate the center coordinates of point A and point D (X m ,(Y n +Y n+1 ) / 2);

[0051] Determine whether the center point coordinates (X m ,(Yn +Y n+1 ) / 2);

[0052] If it exists, point D is considered to be the lower left vertex;

[0053] If it does not exist, the cell determined by points A, B, C, and D is considered to lack a border line, and points A, B, C, and D are discarded, and the above steps of calculating the upper left corner vertex, upper right corner vertex, lower right corner vertex, and lower left corner vertex of the cell are repeated.

[0054] Optionally, in combination with the first aspect, in a possible implementation, the step of calculating the position information of each cell in the table image and cropping the cell further includes:

[0055] Get the coordinates of the upper left corner vertex A of the previous cell (X m , Y n ), coordinates of the upper right corner vertex B (X m+1 , Y n ), the coordinates of the lower right corner vertex C (X m+1 , Y n+1 ), coordinates of the lower left vertex D (X m , Y n+1 );

[0056] The lower right vertex or upper right vertex of the previous cell is used as the upper left vertex of the next cell, and the upper right vertex of the next cell, the lower right vertex of the next cell, and the lower left vertex of the next cell are calculated.

[0057] Optionally, in combination with the first aspect, in a possible implementation, the step of identifying character content in each cell and storing the character content in association with cell position information specifically includes:

[0058] Name each cell to form an ID name;

[0059] Identify the character content of each cell;

[0060] Get the coordinate position information of the cell;

[0061] The ID name, character content and coordinate position information of each cell are associated and stored.

[0062] A second aspect of the present application provides a table image data parsing device, the device comprising:

[0063] An image processing module is used to obtain a table image and perform noise reduction and tilt correction on the table image;

[0064] The calculation module is used to calculate the position information of each cell in the table image and crop the cells;

[0065] The recognition module recognizes the character content in each cell and stores the character content in association with the cell position information.

[0066] The third aspect of the present application provides a non-volatile computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are executed by one or more processors, the one or more processors can execute the above-mentioned table image data parsing method.

[0067] The present invention discloses a table image data parsing method, device and storage medium. The table image data parsing method, device and storage medium of the present invention are implemented to obtain a table image, perform noise reduction and tilt correction processing on the table image, calculate the position information of each cell in the table image and crop the cell, and calculate the position information of each cell in the table image and crop the cell. The table image is split into single independent cell image slices, and then text recognition is performed on the cell image slices, and the cell position information and text information are saved accordingly. Through the steps described in this method, the disadvantage of traditional image recognition that the entire image is recognized as a whole while ignoring the role of the table border line, resulting in the loss of the structured information of the table, is solved. Thereby achieving the effect of improving the degree of structuring and reducing data loss when parsing images with tables. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] The present invention will be further described below with reference to the accompanying drawings and embodiments, in which:

[0069] Figure 1 A schematic diagram of a flow chart of a table image data parsing method in the present invention;

[0070] Figure 2 Schematic diagram of a sub-flow of step S1 in a table image data parsing method of the present invention;

[0071] Figure 3 Schematic diagram of a sub-flow of step S2 in a table image data parsing method of the present invention;

[0072] Figure 4 Schematic diagram of a sub-flow of step S22 in a table image data parsing method of the present invention;

[0073] Figure 5 Schematic diagram of a sub-flow of step S2 in a table image data parsing method of the present invention;

[0074] Figure 6Schematic diagram of a sub-flow of step S241 in a table image data parsing method of the present invention;

[0075] Figure 7 This is a schematic diagram of the first calculation of the upper left corner vertex of a cell in a table image data parsing method of the present invention;

[0076] Figure 8 A schematic diagram of the second calculation of the upper left corner vertex of a cell in a table image data parsing method of the present invention;

[0077] Figure 9 Schematic diagram of a sub-flow of step S242 in a table image data parsing method of the present invention;

[0078] Figure 10 A schematic diagram of the first calculation of the upper right corner vertex of a cell in a table image data parsing method of the present invention;

[0079] Figure 11 A schematic diagram of the second calculation of the upper right corner vertex of a cell in a table image data parsing method of the present invention;

[0080] Figure 12 Schematic diagram of a sub-flow of step S243 in a table image data parsing method of the present invention;

[0081] Figure 13 A schematic diagram of the first calculation of the lower right corner vertex of a cell in a table image data parsing method of the present invention;

[0082] Figure 14 A schematic diagram of the second calculation of the lower right corner vertex of a cell in a table image data parsing method of the present invention;

[0083] Figure 15 Schematic diagram of a sub-flow of step S244 in a table image data parsing method of the present invention;

[0084] Figure 16 This is a schematic diagram of the first calculation of the lower left corner vertex of a cell in a table image data parsing method of the present invention;

[0085] Figure 17 A schematic diagram of the second calculation of the lower left corner vertex of a cell in a table image data parsing method of the present invention;

[0086] Figure 18 Schematic diagram of a sub-flow of step S2 in a table image data parsing method of the present invention;

[0087] Figure 19 Schematic diagram of a sub-flow of step S3 in a table image data parsing method of the present invention;

[0088] Figure 20 A schematic diagram of a table image to be parsed in a table image data parsing method of the present invention;

[0089] Figure 21 A schematic diagram of a horizontal line graph in a table image data parsing method according to the present invention;

[0090] Figure 22 A schematic diagram of a vertical line graph in a table image data parsing method of the present invention;

[0091] Figure 23 A schematic diagram of a cell in a table image data parsing method according to the present invention;

[0092] Figure 24 A schematic diagram of all cells in a table image data parsing method according to the present invention;

[0093] Figure 25 A schematic diagram of calculating the upper left corner vertex of a merged cell in a table image data parsing method of the present invention;

[0094] Figure 26 A schematic diagram of calculating the upper right corner vertex of a merged cell in a table image data parsing method of the present invention;

[0095] Figure 27 A schematic diagram of calculating the lower right corner vertex of a merged cell in a table image data parsing method of the present invention;

[0096] Figure 28 A schematic diagram of calculating the lower right corner vertex of a merged cell in a table image data parsing method of the present invention;

[0097] Figure 29 A schematic diagram of calculating the lower left corner vertex of a merged cell in a table image data parsing method of the present invention;

[0098] Figure 30 It is a structural diagram of a table image data parsing device in the present invention. DETAILED DESCRIPTION

[0099] The technical solutions in the embodiments of the present application will be described below in conjunction with the accompanying drawings. In this application, "at least one" refers to one or more, and "plurality" refers to two or more. "And / or" describes the association relationship between associated objects, indicating that three relationships can exist. For example, "A and / or B" can mean: A exists alone, A and B exist simultaneously, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the associated objects are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, "at least one of a, b, or c" can mean: a, b, c, a and b, a and c, b and c, or a, b, and c, where a, b, and c can be single or plural. It is worth noting that "at least one" can also be interpreted as "one or more items."

[0100] It should be noted that, in this application, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described in this application as "exemplary" or "for example" should not be construed as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0101] The first, second, etc. descriptions appearing in the embodiments of this application are only for illustration and distinction of the description objects. There is no order, nor does it indicate any special limitation on the number of devices in the embodiments of this application, and cannot constitute any limitation on the embodiments of this application.

[0102] First, let’s briefly introduce the technical terms involved in this application:

[0103] Binarization: Image binarization is the process of setting the grayscale value of the pixels on the image to 0 or 255, which is the process of making the entire image appear in obvious black and white.

[0104] Dilation and Erosion: Dilation expands the highlights in an image, resulting in a larger highlight area than the original. Erosion erodes the highlights in the original image, resulting in a smaller highlight area than the original. Dilation and erosion are common image morphological operations, primarily used to remove noise, segment independent image elements, and connect adjacent elements within an image.

[0105] For details, see Figure 1 , the table image data parsing method includes:

[0106] S1. Obtain a table image, and perform noise reduction and tilt correction processing on the table image.

[0107] Specifically, a table image to be parsed is obtained, and the table image is binarized to make the table image black and white to eliminate the influence of color.

[0108] Perform dilation and erosion operations on the table image to remove all text and other interference items, leaving only the horizontal and vertical lines in the table image. This results in a horizontal line graph and a vertical line graph. The horizontal line graph is used to calculate the average tilt angle of the horizontal lines, while the vertical line graph is used to calculate the average tilt angle of the vertical lines. The original image, the horizontal line graph, and the vertical line graph are all rotated by the corresponding degrees to perform tilt correction, as described below.

[0109] S2. Calculate the position information of each cell in the table image and crop the cell.

[0110] Calculate the pixels with the same coordinates in the horizontal and vertical lines, and obtain the intersection of the horizontal and vertical lines. Intersections with equal vertical coordinates are considered to be a row, and the intersections are considered equal if the difference is less than 1% of the image height. Group all intersections by row, and sort the coordinates of each intersection in each group from smallest to largest horizontal coordinate, and save them in array form as an intersection array. If X represents the horizontal coordinate of the intersection and Y represents the vertical coordinate of the intersection, the intersection array data structure is as follows:

[0111] Line 0: [(X0, Y0), (X1, Y0), (X2, Y0) ... (X m , Y0)……]

[0112] …

[0113] Line n: [(X0, Y n ),……(X m , Y n )……]

[0114] …

[0115] The coordinates of all pixels in the horizontal and vertical line graphs are stored in an array to form a table border array. The table border array data structure is as follows:

[0116] [(X1, Y1), (X2, Y2), (X3, Y3)......(X i , Y i )]

[0117] And calculate the upper left corner vertex, upper right corner vertex, lower right corner vertex and lower left corner vertex of each cell according to the above intersection array and table border array, and crop each cell, which will be described in detail later.

[0118] S3. Identify the character content in each cell and associate the character content with the cell position information for storage.

[0119] Specifically, the content of each cell is identified and stored as structured data, that is, the image and text recognition is performed on each cell image slice obtained above in sequence, and the identified text content and cell position information are stored in a database.

[0120] The table image data parsing method, device and storage medium of the present invention are implemented to obtain a table image, perform noise reduction and tilt correction processing on the table image, calculate the position information of each cell in the table image and crop the cell, calculate the position information of each cell in the table image and crop the cell. The table image is split into single independent cell image slices, and then text recognition is performed on the cell image slices, and the cell position information and text information are saved accordingly. Through the steps described in this method, the disadvantage of traditional image recognition that the entire image is recognized as a whole while ignoring the role of the table border line, resulting in the loss of the structured information of the table, is solved. Thereby achieving the effect of improving the degree of structuring and reducing data loss when parsing images with tables.

[0121] See Figure 2 , the step S1 specifically includes:

[0122] S11, performing binarization processing, expansion and corrosion operations on the table image to remove text and interference information in the table image;

[0123] S12, obtaining the horizontal lines and vertical lines in the table image, and obtaining a horizontal line graph, a vertical line graph and an original image respectively;

[0124] S13. Calculate the average inclination angle of the horizontal or vertical lines in the horizontal or vertical line graph;

[0125] S14. Perform tilt correction on the horizontal line graph, the vertical line graph, and the original image according to the average tilt angle.

[0126] After obtaining the table image to be parsed, the table image is binarized to convert it into black and white to eliminate the influence of color. The table image is then dilated and eroded to remove all text and other interference items, leaving only the horizontal and vertical lines in the table image, thus obtaining the horizontal and vertical line graphs.

[0127] Use the horizontal line graph to calculate the average tilt angle of the horizontal lines, or use the vertical line graph to calculate the average tilt angle of the vertical lines, rotate the original image, the horizontal line graph, and the vertical line graph as a whole by a corresponding degree, and perform tilt correction on the horizontal line graph, the vertical line graph, and the original image.

[0128] See Figure 3 , the step S2 specifically includes:

[0129] S21, calculating the pixel points with the same coordinates in the horizontal and vertical lines, and obtaining all the intersection points of the horizontal and vertical lines;

[0130] S22, storing the coordinates of all intersection points in order as an intersection point array;

[0131] S23. Save the coordinates of all pixel points in the horizontal line graph and the vertical line graph as a table border array.

[0132] See Figure 4 , the step S22 specifically includes:

[0133] S221, obtaining coordinate information of all intersections, treating all intersections with the same vertical coordinate as a row, and dividing all intersections into multiple rows;

[0134] S222 , sorting the intersection points in each row from small to large according to the horizontal coordinate, and sorting each row from small to large according to the vertical coordinate, to form the intersection point array.

[0135] Specifically, we calculate the pixels with the same coordinates in the horizontal and vertical lines, and obtain the intersection of the horizontal and vertical lines. The intersections with the same vertical coordinates are considered to be a row, and the intersections are considered equal if the difference is less than 1% of the image height. All intersections are grouped by row, and the coordinates of each intersection in each group are sorted from small to large by horizontal coordinate, and saved in array form as an intersection array. If X represents the horizontal coordinate of the intersection and Y represents the vertical coordinate of the intersection, the intersection array data structure is as follows:

[0136] Line 0: [(X0, Y0), (X1, Y0), (X2, Y0) ... (X m , Y0)……]

[0137] …

[0138] Line n: [(X0, Y n ),……(X m , Y n )……]

[0139] …

[0140] The coordinates of all pixels in the horizontal and vertical line graphs are stored in an array to form a table border array. The table border array data structure is as follows:

[0141] [(X1, Y1), (X2, Y2), (X3, Y3)......(X i , Y i )].

[0142] See Figure 5 , the step S2 further comprises:

[0143] S24, calculating the upper left corner vertex, upper right corner vertex, lower right corner vertex and lower left corner vertex of the cell from the intersection array and the table border array;

[0144] S25. Cut the cell according to the coordinates of the upper left corner vertex, the upper right corner vertex, the lower right corner vertex, and the lower left corner vertex.

[0145] Wherein, step S24 specifically includes:

[0146] S241, calculate the upper left corner vertex of the cell; S242, calculate the upper right corner vertex of the cell; S243, calculate the lower right corner vertex of the cell; S244, calculate the lower left corner vertex of the cell.

[0147] See Figure 6 , the step S241 further includes:

[0148] S2411, take the coordinate information of the mth point in the nth row intersection array, record it as point A(X m , Y n );

[0149] S2412, obtain the coordinate information of the m+1th point in the nth row intersection array (X m+1 , Y n );

[0150] S2413. Calculate the center point coordinates (X m +X m+1 ) / 2,Y n );

[0151] S2414: Determine whether the center point coordinates (X m +X m+1 ) / 2, Yn);

[0152] S2415. If it exists, point A is considered to be the upper left vertex;

[0153] S2416. If it does not exist, discard point A, take the m+1th point in the intersection point array of the nth row as point A, and repeat the above steps.

[0154] Specifically, take the mth (m starts from 0 and increases) point in the nth (n starts from 0 and increases) intersection array, and record it as point A(X m , Y n ), verify whether point A is the upper left corner vertex of the cell.

[0155] Determine whether there is a border line between point A and the m+1th point. Determine whether the line exists by determining whether the center point of the line exists, that is, determine whether there is a line with coordinates ((X m +X m+1 ) / 2, Y1), if any, see Figure 7 , then point A is considered to be the upper left corner vertex of the cell, and step S2421 is continued; if it does not exist, see Figure 8 , then discard point A(X m , Y n ), m is incremented by 1 and step S2411 is executed again. At this time, m=m+1 in step S2411.

[0156] See Figure 9 , the step S242 further includes:

[0157] S2421, get the coordinates of the upper left corner vertex A (X m , Y n );

[0158] S2422, take the coordinate information of the m+1th point in the intersection array of the nth row, record it as point B (X m+1 , Y n );

[0159] S2423, obtain the coordinate information of the m+1th point in the n+1th row intersection array (X m+1 , Y n+1 );

[0160] S2424. Calculate the center point coordinates (X m+1 ,(Y n +Y n+1 ) / 2);

[0161] S2425, determine whether the center point coordinates (X m+1 ,(Y n +Y n+1 ) / 2);

[0162] S2426. If it exists, point B is considered to be the upper right vertex;

[0163] S2427. If it does not exist, discard point B, take the m+2th point in the intersection point array of the nth row as point B, and repeat the above steps.

[0164] Specifically, take the m+1th point in the nth row of the intersection array and record it as point B(X m+1 , Y n ) to verify whether the point is the upper right corner of the cell.

[0165] Assuming that point B is the top right corner of the cell, there is a border line in the original image that passes through point B and the center point of the vertical line in row n+1 (that is, there is a border line between the top right corner vertex B and the bottom right corner vertex). The vertical coordinate of row n+1 can be obtained by using the data in the intersection array in row n+1. n+1, that is, to determine whether there is a coordinate (X m+1 ,(Y n +Y n+1 ) / 2). If present, see Figure 10 , then point B is considered to be the upper right corner vertex of the cell, and step S2431 is continued; if it does not exist, see Figure 11 , then discard point B(X m+1 , Y n ), take the m+2th point in the intersection array of the nth row and make the same judgment until the upper right corner vertex of the cell is found or there are no other points in the current row.

[0166] It can be understood that step S242 is a further calculation based on the determination of the upper left corner vertex A. This step is performed after step S2415.

[0167] See Figure 12 , the step S243 further includes:

[0168] S2431, get the coordinates of the upper left vertex A (X m , Y n ) and the coordinates of the upper right vertex B (X m+1 , Y n );

[0169] S2432. Take the point in the intersection point array of row n+1 that has the same or similar horizontal coordinate as point B and record it as point C (X m+1 , Y n+1 );

[0170] S2433. Take the point in the intersection point array of row n+1 that has the same or similar horizontal coordinate as point A and record it as point D (X m , Y n+1 );

[0171] S2434, calculate the center point coordinates of point C and point D ((X m +X m+1 ) / 2,Y n+1 );

[0172] S2435: Determine whether the center point coordinates ((X m +X m+1 ) / 2,Y n+1 );

[0173] S2436. If it exists, then point C is considered to be the lower right vertex;

[0174] S2437. If it does not exist, discard point C, take the point in the intersection point array of the n+2th row that has the same or similar horizontal coordinate as point B as point C, and repeat the above steps.

[0175] Specifically, if we find the upper right vertex B(X m+1 , Y n ), then find the horizontal coordinate X of the top right corner of the cell in the intersection array of row n+1 m+1 The point that is approximately equal is denoted as point C(X m+1 , Y n+1 ), verify whether point C is the lower right corner vertex of the cell.

[0176] Assuming that point C is the vertex of the lower right corner of the cell, the vertex of the lower left corner of the cell can be obtained when the coordinates of the three vertices are known, and it is recorded as point D(X m , Y n+1 ), then there is a border line between points C and D in the original image. The existence of the line is determined by judging whether there is a center point of the line segment, that is, judging whether there is a point with coordinates ((X m +X m+1 ) / 2,Y n+1 ) point. If present, see Figure 13 , then the point is considered to be the lower right corner vertex of the cell, and step S2441 is continued; if it does not exist, see Figure 14 , then discard the point and continue searching towards row n+2 until a point that meets the requirements is found or there are no points that meet the requirements in all the rows below.

[0177] It can be understood that step S243 is a further calculation based on the determination of the upper left vertex A and the upper right vertex B. This step is performed after step S2426.

[0178] See Figure 15 , the step S244 further includes:

[0179] S2441, get the coordinates of the upper left corner vertex A (X m , Y n ), coordinates of the upper right corner vertex B (X m+1 , Y n ) and the coordinates of the lower right vertex C (X m+1 , Y n+1 );

[0180] S2442, calculate the coordinates of the lower left corner vertex D (X) according to the coordinates of the upper left corner vertex A, the coordinates of the upper right corner vertex B and the coordinates of the lower right corner vertex C. m , Y n+1 );

[0181] S2443, calculate the center point coordinates (X m ,(Y n +Y n+1 ) / 2);

[0182] S2444, determine whether the center point coordinates (X m ,(Y n +Y n+1 ) / 2);

[0183] S2445. If it exists, then point D is considered to be the lower left vertex;

[0184] S2446. If it does not exist, it is considered that the cell determined by point A, point B, point C, and point D lacks a border line, and point A, point B, point C, and point D are discarded, and the above steps of calculating the upper left corner vertex, upper right corner vertex, lower right corner vertex, and lower left corner vertex of the cell are repeated.

[0185] Specifically, according to the rules of rectangle, given three points, the coordinates of the fourth point can be obtained as D(X m , Y n+1 ), verify whether point D is the lower left corner vertex of the cell.

[0186] Assuming that point D is the lower left corner vertex of the cell, there is a border line between the upper left corner vertex A and the lower left corner vertex D. The existence of the line is determined by judging whether there is a center point of the line segment, that is, whether there is a point with coordinates (X m ,(Y n +Y n+1 ) / 2). If present, see Figure 16 , then point D is the lower left corner vertex of the cell, and continue to step S245; if it does not exist, see Figure 17 , the cell determined by point A, point B, point C, and point D lacks a border line, all four points are discarded, m is incremented by 1, and step S2441 is executed again.

[0187] It can be understood that step S244 is a further calculation based on the determination of the upper left vertex A, the upper right vertex B, and the lower right vertex C. This step is performed after step S2436.

[0188] See Figure 18 , the step S2 further comprises:

[0189] S245, get the coordinates of the upper left corner vertex A of the previous cell (X m , Y n ), coordinates of the upper right corner vertex B (X m+1 , Y n ), the coordinates of the lower right corner vertex C (X m+1 , Y n+1 ), coordinates of the lower left vertex D (X m , Y n+1 );

[0190] S246. Use the lower right vertex or the upper right vertex of the previous cell as the upper left vertex of the next cell, and calculate the upper right vertex of the next cell, the lower right vertex of the next cell, and the lower left vertex of the next cell.

[0191] See Figure 19 , the step S3 specifically includes:

[0192] S31, naming each cell to form an ID name;

[0193] S32, identifying the character content of each cell;

[0194] S33, obtaining the coordinate position information of the cell;

[0195] S34. Associate and store the ID name, character content, and coordinate position information of each cell.

[0196] Specifically, the content of each cell is identified and stored as structured data. The cell image slices obtained in the above steps are sequentially subjected to image text recognition, and the recognized text content and cell location information are stored in the database. The data structure is shown in Table 1.

[0197]

[0198] Table 1 Data structure for storing text content and cell location information of each cell

[0199] The data structure includes the cell ID, cell content, row number, column number, the horizontal coordinate of the top left corner of the cell, the vertical coordinate of the top left corner of the cell, the horizontal coordinate of the bottom right corner of the cell, and the vertical coordinate of the bottom right corner of the cell.

[0200] The row number determination rule is: the row number of the cell is the row where the upper left vertex is located and the row where the lower right vertex is located minus one. If the row number of the upper left vertex is not equal to the row number of the lower right vertex minus one, the cells are merged and the row range is stored.

[0201] The rule for determining the column number is: sort the horizontal coordinates of all cell vertices from small to large, merge the approximately equal coordinates into an array, compare the horizontal coordinates of the upper left vertex and the lower right vertex of the cell with the coordinates in the array, the array sequence number of the approximately equal coordinates is the column sequence number of the point, the column number of the upper left vertex and the column number of the lower right vertex minus one is the cell column number, if the column number of the upper left vertex is not equal to the column number of the lower right vertex minus one, then merge the cells and store the column range.

[0202] Please refer to Figure 20 In a specific embodiment, the table image to be parsed is a project funding table.

[0203] The project fund raising table picture is binarized, and then expanded and eroded to obtain horizontal and vertical line graphs. The horizontal line graph is as follows: Figure 21 As shown, the vertical line graph is Figure 22 shown.

[0204] Furthermore, we find the points in the horizontal and vertical graphs with the same coordinates, which are the intersection points of the horizontal and vertical lines. In this example, the image is 1229 pixels high, so the difference in vertical coordinates is less than 12, which is considered to be approximately equal. The intersection points with approximately equal vertical coordinates are considered to be a row, and the coordinates of the points in each group are sorted from smallest to largest by horizontal coordinate. The resulting intersection array is as follows:

[0205] Row 0: [(45, 38), (419, 38), (793, 38), (1171, 38)]

[0206] Row 1: [(45, 86), (419, 86), (793, 86), (1019, 86), (1171, 86)]

[0207] Row 2: [(793, 164), (1019, 164), (1171, 164)]

[0208] Row 3: [(419, 242), (793, 242), (1019, 242), (1171, 242)]

[0209] Row 4: [(45, 320), (419, 320), (793, 320), (1171, 320)]

[0210] Row 5: [(45, 368), (793, 368), (1171, 368)]

[0211] All the pixels in the horizontal and vertical line graphs are stored in the table border array, which will be used in the subsequent process to determine whether there is a border line somewhere. Due to the large amount of data, it is not listed here.

[0212] Next, we take the zeroth point in row 0, labeled A (45, 38), and verify whether point A is the top-left corner of the cell. We then determine whether there is a border line between point A and the first point in row 0 (419, 38). We determine whether the line exists by checking the center of the line. By searching the table border array and finding the point (232, 38), we can determine that point A is the top-left corner of the cell.

[0213] Take the first point in row 0, denoted as point B (419, 38), and determine whether it can be used as the top-right vertex of the cell. Assuming point B is the top-right vertex, there should be a vertical border line passing through point B (419, 38). Verify the existence of this border line by verifying whether the center point of this vertical line exists in the border array. Based on the data, the average vertical coordinate of the first row is 86. Searching the table border array reveals that point (419, 62) exists, so point B (419, 38) can be determined to be the top-right vertex.

[0214] In the first row, we search for a point whose horizontal coordinate is approximately equal to 419. We find point (419, 86) and record it as point C (419, 86). We then determine whether point C (419, 86) can be used as the lower right corner vertex of the cell. Assuming that point C (419, 86) is the lower right corner vertex, based on the three known points, we can infer that the coordinates of the lower left corner vertex are (45, 86), recorded as point D (45, 86). Therefore, there should be a border line between points C (419, 86) and D (45, 86). We verify whether the border line exists by verifying whether the center point of the perpendicular line exists in the border array. We search the table border array and find that point (232, 86) exists. We then determine that point C (419, 86) is the lower right corner vertex.

[0215] Based on the known coordinates of the three points, the coordinates of the fourth point can be obtained as (45, 86), recorded as point D (45, 86). Assuming that point D is the lower left corner vertex of the cell, there is a border line between the upper left vertex A (45, 38) and the lower left vertex D (45, 86). The existence of the line is determined by judging whether there is the center point of the line segment. The table border array is searched and the point (45, 62) is found. Then the point D (45, 86) is judged to be the lower left corner vertex.

[0216] Finally, the original image is cut according to the coordinates of point A (45, 38), point B (419, 38), point C (419, 86), and point D (45, 86), and the cell image slices can be obtained, such as Figure 23 shown.

[0217] Use point B as the upper left corner vertex of the next cell and start the next cycle.

[0218] Through the above steps, all cell image slices can be obtained by looping. All cell images can be cut as follows: Figure 24 shown.

[0219] In this embodiment, the table parsing of the vertically merged cells of "a certain infrastructure project" is further illustrated.

[0220] like Figure 25As shown, take the 0th point in the first row and record it as A(45, 86) to verify whether point A is the top left corner of the cell. Determine whether there is a border line between point A and the first point in the first row (419, 86). Determine whether the line exists by checking whether the center point of the line exists. Search the table border array and find the point (232, 86). It can be determined that point A is the top left corner of the cell.

[0221] Further, such as Figure 26 As shown, take the first point in the first row, record it as point B (419, 86), and determine whether point B (419, 86) can be used as the upper right corner vertex of the cell. Assuming point B is the upper right corner vertex, there should be a vertical border line passing through point B (419, 86). Verify whether the border line actually exists by verifying whether the center point of the vertical line exists in the border array. According to the data, the average value of the vertical coordinate in the second row is 164. Searching the table border array finds that point (419, 125) exists, and it can be determined that point B (419, 86) is the upper right corner vertex.

[0222] Further, such as Figure 27 As shown, in the second row, a point with a horizontal coordinate approximately equal to 419 is searched. No matching point is found in the second row, so the search continues in the third row. In the third row, a point with a horizontal coordinate approximately equal to 419 is found (419, 242), which is recorded as point C (419, 242). The search then determines whether point C (419, 242) can be used as the lower right corner vertex of the cell. Assuming point C is the lower right corner vertex, based on the three known points, the coordinates of the lower left corner vertex can be deduced to be (45, 242), recorded as point D (45, 242). A border line should exist between points C (419, 242) and D (45, 242). The existence of the border line is verified by verifying whether the center point of the border line exists in the border array. The table border array is searched, but point (232, 242) cannot be found. Point C (419, 242) is determined not to be the lower right corner vertex, so the search is discarded and the search continues in the next row.

[0223] Further, such as Figure 28 As shown, in the 4th row, find a point whose horizontal coordinate is approximately equal to 419, find point (419, 320), record it as point C (419, 320), and determine whether point C (419, 320) can be used as the lower right corner vertex of the cell. Assuming that point C (419, 320) is the lower right corner vertex, based on the known 3 points, the coordinates of the lower left corner vertex can be deduced to be (45, 320), record it as point D (45, 320). Then there should be a border line between point C (419, 320) and point D (45, 320). Verify whether the border line exists by verifying whether the center point of the border line exists in the border array. Search the table border array and find that point (232, 320) exists. Determine that point C (419, 320) is the lower right corner vertex.

[0224] Further, such as Figure 29 As shown, according to the rules of a rectangle, given three points, the coordinates of the fourth point can be obtained as (45, 320), recorded as point D (45, 320). Assuming that point D is the lower left corner vertex of the cell, there is a border line between the upper left corner vertex A (45, 86) and the lower left corner vertex D (45, 320). The existence of the line is determined by judging whether the center point of the line segment exists. The table border array is searched and the point (45, 203) is found. Then the point D (45, 320) is determined to be the lower left corner vertex.

[0225] Finally, the original image is cropped according to the coordinates of point A (45, 86), point B (419, 86), point C (419, 320), and point D (45, 320) to obtain cell image slices.

[0226] The cell image slices obtained in the above steps are sequentially subjected to image text recognition, and the recognized text content and cell location information are stored in the database. The data storage is shown in Table 2.

[0227]

[0228] Table 2 Data structure for storing text content and cell location information of each cell

[0229] This application also provides a table image data parsing device, see Figure 30 , the device comprises:

[0230] The image processing module 301 is used to obtain a table image and perform noise reduction and tilt correction on the table image;

[0231] A calculation module 302 is used to calculate the position information of each cell in the table image and crop the cell;

[0232] The recognition module 303 recognizes the character content in each cell and associates the character content with the cell position information for storage.

[0233] In another embodiment of the table image data parsing device, the table image data parsing device may have relatively large differences due to different configurations or performances, and may include one or more processors (central processing units, CPU) (for example, one or more processors) and memories, and one or more storage media for storing applications or data (for example, one or more mass storage devices). The memories and storage media may be temporary storage or permanent storage. The program stored in the storage medium may include one or more modules, and each module may include a series of instruction operations in the table image data parsing device. Furthermore, the processor may be configured to communicate with the storage medium and execute a series of instruction operations in the storage medium on the table image data parsing device.

[0234] The table image data parsing device may further include one or more power supplies, one or more wired or wireless network interfaces, one or more input and output interfaces, and / or one or more operating systems, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, etc. Those skilled in the art will appreciate that the table image data parsing device does not limit the data processing device and may include more or fewer components than shown, or may combine certain components, or arrange the components differently.

[0235] The present application also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. Instructions are stored in the computer-readable storage medium, and when the instructions are executed on a computer, the computer executes the steps of the table image data parsing method.

[0236] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0237] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, and other media that can store program code.

[0238] In the embodiments provided herein, it should be understood that the disclosed methods may be implemented in other ways without departing from the spirit and scope of the present application. The present embodiments are merely illustrative examples and should not be construed as limiting. The specific details provided herein should not limit the purpose of the present application. For example, some features may be omitted or not implemented.

[0239] The technical means disclosed in the present application are not limited to those disclosed in the above-mentioned embodiments, but also include technical solutions composed of any combination of the above-mentioned technical features. It should be noted that those skilled in the art can make several improvements and modifications without departing from the principles of the present application, and such improvements and modifications are also considered to be within the scope of protection of the present application.

[0240] The above is a detailed introduction to a table image data parsing method, device and storage medium provided in the embodiments of the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for those skilled in the art, according to the ideas of the present application, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as a limitation on the present application. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A table image data parsing method, characterized in that: The method comprises: Obtain a table image, and perform noise reduction and tilt correction processing on the table image; Calculate the position information of each cell in the table image and crop the cells; Identifying character content in each cell and storing the character content in association with cell position information; The step of performing noise reduction and tilt correction processing on the table image specifically includes: Binarization, expansion, and corrosion operations are performed on the table image to remove text and interference information in the table image; Obtain the horizontal lines and vertical lines in the table image to obtain a horizontal line graph, a vertical line graph and an original image respectively; Find the average inclination angle of the horizontal or vertical lines in a horizontal or vertical line graph; Performing tilt correction on the horizontal line graph, the vertical line graph and the original image according to the average tilt angle; The steps of calculating the position information of each cell in the table image and cropping the cell include: Calculate the pixel points with the same coordinates in the horizontal and vertical lines, and obtain all the intersection points of the horizontal and vertical lines; Keep the coordinates of all intersection points in order as an intersection array; Save the coordinates of all pixels in the horizontal and vertical line graphs as a table border array; The step of keeping the coordinates of all intersection points in order as an intersection point array specifically includes: Obtaining coordinate information of all intersections, dividing all intersections with the same vertical coordinate into a row, and dividing all intersections into multiple rows; Sort the intersection points in each row from small to large according to the horizontal coordinate, and sort each row from small to large according to the vertical coordinate to form the intersection point array; The steps of calculating the position information of each cell in the table image and cropping the cell specifically include: Calculate the upper left corner vertex, upper right corner vertex, lower right corner vertex and lower left corner vertex of the cell from the intersection array and the table border array; Cutting the cell according to the coordinates of the upper left corner vertex, the upper right corner vertex, the lower right corner vertex and the lower left corner vertex; The step of calculating the upper left corner vertex of the cell from the intersection array and the table border array specifically includes: Take the coordinate information of the mth point in the nth row intersection array and record it as point A(X m , Y n ); Get the coordinate information of the m+1th point in the nth row intersection array (X m+1 , Y n ); Calculate the center point coordinates of the mth point in the nth row and the m+1th point in the nth row ((X m +X m+1 ) / 2,Y n ); Determine whether the center point coordinates ((X m +X m+1 ) / 2, Yn); If it exists, point A is considered to be the upper left vertex; If it does not exist, discard point A, take the m+1th point in the intersection array of the nth row as point A and repeat the above steps.

2. The table image data parsing method according to claim 1, characterized in that: The step of calculating the upper right corner vertex of the cell from the intersection array and the table border array specifically includes: Get the coordinates of the upper left vertex A (X m , Y n ); Take the coordinate information of the m+1th point in the intersection array of the nth row and record it as point B(X m+1 , Y n ); Get the coordinate information of the m+1th point in the n+1th row intersection array (X m+1 , Y n+1 ); Calculate the center point coordinates (X m+1 ,(Y n +Y n+1 ) / 2); Determine whether the center point coordinates (X m+1 ,(Y n +Y n+1 ) / 2); If it exists, point B is considered to be the upper right vertex; If it does not exist, discard point B, take the m+2th point in the intersection array of the nth row as point B and repeat the above steps.

3. The table image data parsing method according to claim 2, characterized in that: The step of calculating the vertex of the lower right corner of the cell from the intersection array and the table border array specifically includes: Get the coordinates of the upper left vertex A (X m , Y n ) and the coordinates of the upper right vertex B (X m+1 , Y n ); Take the point in the intersection array of row n+1 that has the same or similar horizontal coordinate as point B and record it as point C (X m+1 , Y n+1 ); Take the point in the intersection array of row n+1 that has the same or similar horizontal coordinate as point A and record it as point D(X m , Y n+1 ); Calculate the center point coordinates of point C and point D ((X m +X m+1 ) / 2,Y n+1 ); Determine whether the center point coordinates ((X m +X m+1 ) / 2,Y n+1 ); If it exists, point C is considered to be the lower right vertex; If it does not exist, discard point C, take the point in the intersection array of the n+2th row that has the same or similar horizontal coordinate as point B as point C, and repeat the above steps.

4. The table image data parsing method according to claim 3, characterized in that: The step of calculating the lower left corner vertex of the cell from the intersection array and the table border array specifically includes: Get the coordinates of the upper left vertex A (X m , Y n ), coordinates of the upper right corner vertex B (X m+1 , Y n ) and the coordinates of the lower right vertex C (X m+1 , Y n+1 ); The coordinates of the lower left corner vertex D (X) are calculated based on the coordinates of the upper left corner vertex A, the upper right corner vertex B and the lower right corner vertex C. m , Y n+1 ); Calculate the center coordinates of point A and point D (X m ,(Y n +Y n+1 ) / 2); Determine whether the center point coordinates (X m ,(Y n +Y n+1 ) / 2); If it exists, point D is considered to be the lower left vertex; If it does not exist, the cell determined by points A, B, C, and D is considered to lack a border line, and points A, B, C, and D are discarded, and the above steps of calculating the upper left corner vertex, upper right corner vertex, lower right corner vertex, and lower left corner vertex of the cell are repeated.

5. The table image data parsing method according to claim 4, characterized in that: The step of calculating the position information of each cell in the table image and cropping the cell further includes: Get the coordinates of the upper left corner vertex A of the previous cell (X m , Y n ), coordinates of the upper right corner vertex B (X m+1 , Y n ), the coordinates of the lower right corner vertex C (X m+1 , Y n+1 ), coordinates of the lower left vertex D (X m , Y n+1 ); The lower right vertex or upper right vertex of the previous cell is used as the upper left vertex of the next cell, and the upper right vertex of the next cell, the lower right vertex of the next cell, and the lower left vertex of the next cell are calculated.

6. The table image data parsing method according to claim 1, characterized in that: The step of identifying the character content in each cell and storing the character content in association with the cell position information specifically includes: Name each cell to form an ID name; Identify the character content of each cell; Get the coordinate position information of the cell; The ID name, character content and coordinate position information of each cell are associated and stored.

7. A table image data parsing device, characterized in that: The device executes the table image data parsing method according to any one of claims 1 to 6, comprising: An image processing module is used to obtain a table image and perform noise reduction and tilt correction on the table image; The calculation module is used to calculate the position information of each cell in the table image and crop the cells; The recognition module recognizes the character content in each cell and stores the character content in association with the cell position information.

8. A non-volatile computer-readable storage medium, characterized in that: The non-volatile computer-readable storage medium stores computer-executable instructions, which, when executed by one or more processors, enable the one or more processors to execute the table image data parsing method according to any one of claims 1 to 6.

Images