A method for identifying table structure of wireless frame table image

Abstract

The invention discloses a table structure recognition method for a wireless frame table image, and the method comprises the following steps: acquiring a wireless frame table image; processing the image and converting it into a grayscale image; accumulating the pixel values ​​of pixel points in a single row of the grayscale image to obtain a row projection of the grayscale image; accumulating the pixel values ​​of pixel points in a single column of the grayscale image to obtain a column projection of the grayscale image; scanning the row projection to obtain row peaks and row troughs; scanning the column projection to obtain column peaks and column troughs; calculating the position of table lines according to the row troughs and column troughs to obtain a table structure; being able to effectively detect and recognize the wireless frame table image to generate a table structure containing frame lines, which is of great significance for the later document image layout analysis and recognition work; and improving the efficiency and accuracy of table recognition.

Description

Technical Field

[0001] The present invention relates to the technical field of image table recognition, and in particular to a method for recognizing a table structure of a wireless frame table image. Background Art

[0002] With the rapid development of computer technology, the use of electronic documents is becoming more and more popular. In addition, due to the problems of paper documents being easily damaged and difficult to preserve, it is often necessary to convert paper documents into digital form and store them in computers. For example, paper documents can be converted into digital images by taking photos or scanning, and then image processing and optical character recognition (OCR) and other technologies can be used to recognize the text in the digital image for further editing or other applications. Among these paper documents, there is a special type of document - table documents, such as bank statements, value-added tax invoices, company reports, etc., which are widely used in our daily life and work. Document table lines divide the entire document into different areas, which is an important basis for document layout analysis and identification information structured processing. Therefore, the research on the detection and removal of document image table lines is of great significance.

[0003] However, in many cases nowadays, although some table document materials are generated based on tables, the table borders are not generated. When these document materials need to be converted into electronic documents, they cannot be accurately identified and the table cannot be generated for expression. That is, because there are no borders, it is impossible to identify and generate accurate electronic tables. Summary of the invention

[0004] In view of the problem that a table generated by a table image without a wireless frame cannot be recognized in the current field of table recognition technology, the present invention provides a table structure recognition method for a table image without a wireless frame, which calculates and generates a table structure by obtaining row and column troughs through pixel row and column projections.

[0005] To achieve the above object, the embodiments of the present invention adopt the following technical solutions:

[0006] A method for identifying a table structure of a wireless frame table image, the method comprising the following steps:

[0007] Get the wireless frame table image;

[0008] Process the image and convert it into a grayscale image;

[0009] Accumulate the pixel values ​​of the pixels in a single row of the grayscale image to obtain the row projection of the grayscale image;

[0010] Accumulate the pixel values ​​of the pixels in a single column of the grayscale image to obtain the column projection of the grayscale image;

[0011] Scan the line projection to obtain the traveling wave peak and traveling wave trough;

[0012] Scan the column projection to obtain column wave peaks and column wave troughs;

[0013] The table structure is obtained by calculating the table line positions according to the row troughs and column troughs.

[0014] According to one aspect of the present invention, the step of calculating the table line positions according to the row troughs and the column troughs to obtain the table structure comprises the following steps:

[0015] Draw horizontal lines through the midpoints of the traveling troughs, and draw vertical lines through the midpoints of the column troughs; the collection of all horizontal lines forms a horizontal line set, and the collection of all vertical lines forms a vertical line set;

[0016] Each horizontal line intersects with each vertical line to get an intersection point, and a point set is set to include all intersection points;

[0017] The intersection of the first line segment of the horizontal line set and the first line segment of the vertical line set is taken as the upper left corner, the intersection of the first line segment of the horizontal line set and the last line segment of the vertical line set is taken as the upper right corner, the intersection of the last line segment of the horizontal line set and the first line segment of the vertical line set is taken as the lower left corner, and the intersection of the last line segment of the horizontal line set and the last line segment of the vertical line set is taken as the lower right corner to generate a table structure.

[0018] According to one aspect of the present invention, scanning the row projection to obtain the traveling wave peak and the traveling wave trough includes: scanning the row projection row by row, and according to the size of the row projection value, taking the continuous interval in which the row projection value is less than a first threshold as the traveling wave trough, and the other areas except the traveling wave trough as the traveling wave peak.

[0019] According to one aspect of the present invention, scanning the column projections to obtain column peaks and column troughs includes: scanning the column projections column by column, and according to the size of the column projection values, taking the continuous intervals in which the column projection values ​​are less than a second threshold as the column troughs, and the other areas except the column troughs as the column peaks.

[0020] According to one aspect of the present invention, the method includes: after the step of scanning the row projection to obtain the traveling wave peak and the traveling wave trough, scanning the traveling wave peak and the traveling wave trough again to obtain the secondary traveling wave peak and the secondary traveling wave trough respectively, and using the secondary traveling wave trough as the basis for calculating the table line position to obtain the table structure.

[0021] According to one aspect of the present invention, scanning the Hang wave peak and the traveling wave trough to obtain the secondary Hang wave peak and the secondary traveling wave trough respectively comprises the following steps:

[0022] On the basis of the traveling wave peaks, the traveling wave peak with the smallest range is taken as the traveling wave peak unit;

[0023] Scan a traveling wave peak starting from the traveling wave peak unit with fixed pixel points above and below as scanning frames;

[0024] If the average gray value of the traveling wave peak in the scanning frame is less than the average gray value of this traveling wave peak, it is considered that the traveling wave peak in the scanning frame belongs to the secondary traveling wave valley part, and a secondary traveling wave peak is obtained;

[0025] Take the end point of the secondary traveling wave peak as the seven scanning points and use the scanning frame to scan row by row. When the average grayscale value of the traveling wave peak in the scanning frame after scanning the secondary traveling wave trough is greater than the average grayscale value of this traveling wave peak, it is considered that the scanning of a secondary traveling wave trough is completed and a secondary traveling wave trough is obtained.

[0026] According to one aspect of the present invention, the method includes: after the step of scanning the column projection to obtain the column wave peaks and column wave troughs is executed, the column wave peaks and column wave troughs are scanned again to obtain the secondary column wave peaks and secondary column wave troughs respectively, and the secondary column wave troughs are used as the basis for calculating the table line position to obtain the table structure.

[0027] According to one aspect of the present invention, scanning the column wave peaks and column wave troughs to obtain the secondary column wave peaks and secondary column wave troughs respectively comprises the following steps:

[0028] On the basis of obtaining the wave valleys and wave peaks, a wave peak is scanned from left to right to find the maximum value point set and the minimum value point set in the wave peak;

[0029] Calculate the slopes of the maximum point and its nearby points and the minimum point and its nearby points on both sides respectively, and get the left slope set and the right slope set;

[0030] The part with the largest slope on both sides obtained with a maximum point as the center is a secondary wave peak. Except for the detected secondary wave peak, all other parts of a wave peak are secondary wave troughs.

[0031] According to one aspect of the present invention, the method further comprises the steps of: performing straight line detection on the grayscale image, and correcting the table structure according to the detected straight lines.

[0032] According to one aspect of the present invention: the recognized table structure is input into a document with a suffix of json in a certain format. The advantages of the implementation of the present invention: it can effectively detect and recognize the frameless table image to generate a table structure with frame lines, which is of great significance for the subsequent document image layout analysis and recognition work; it improves the efficiency and accuracy of table recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0034] Figure 1 This is a flow chart of the method for identifying the table structure of a wireless frame table image according to Embodiment 1 and Embodiment 2 of the present invention;

[0035] Figure 2 This is a flow chart of a method for identifying a table structure of a wireless frame table image according to a third embodiment of the present invention;

[0036] Figure 3 is a grayscale image of the wireless frame table image column of the present invention, in which the blue circle is the position correctly detected as a primary trough, and the red circle is the position not detected as a primary trough;

[0037] Figure 4 It is a grayscale image of the wireless frame table image column of the present invention, in which the red frame represents the range actually detected as the peak after the first scan;

[0038] Figure 5 This is a grayscale image of the wireless frame table image column described in the present invention. The red dots in the image represent the maximum points obtained by the first peak scan, and the blue dots represent the minimum points obtained by the first peak scan. DETAILED DESCRIPTION

[0039] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0040] Embodiment 1

[0041] like Figure 1 , Figure 3 , Figure 4 and Figure 5 As shown, a method for identifying a table structure of a wireless frame table image comprises the following steps:

[0042] Step S1: Obtain a wireless frame table image;

[0043] In this step, the wireless frame table image can be obtained by scanning or taking a photo, and the image can be in RGB or any image format. In this embodiment, the wireless frame table image Io is obtained, and the image height is H and the image width is W.

[0044] Step S2: Process the image and convert it into a grayscale image;

[0045] First, the wireless frame table image Io is binarized into a grayscale image Ig. Specifically, the image binarization algorithm is used to convert the obtained wireless frame table image Io from the original RGB image into a grayscale image Ig, and the grayscale value of the pixel point in the i-th row and j-th column in Ig is represented by Pix(i,j).

[0046] Step S3: Accumulate the pixel values ​​of the pixel points in a single row of the grayscale image to obtain a row projection of the grayscale image;

[0047] Step S31: Accumulate the pixel values ​​of the pixels in the i-th row of Ig to obtain the row projection of the row of Ig, denoted as Hp i , which can be expressed as the following formula:

[0048]

[0049] Pix(i, j) is the grayscale value of the pixel in the i-th row and j-th column in Ig.

[0050] Step S32: Repeat step S31 for each row of Ig to obtain the row projection of each row of Ig, which is called the row projection of the grayscale image, abbreviated as row projection, denoted as Hp. The set Hp can be expressed as:

[0051] Hp={Hp i |i=0,1,...(H-1)}

[0052] Step S4: Accumulate the pixel values ​​of the pixel points in a single column of the grayscale image to obtain the column projection of the grayscale image;

[0053] Step S41: Accumulate the pixel values ​​of the pixels in the jth column of Ig to obtain the column projection of the column of Ig, denoted as Vp i , which can be expressed as the following formula:

[0054]

[0055] Step S42: Repeat step S41 for each column of Ig to obtain the column projection of each column of Ig, which is called the column projection of the grayscale image, abbreviated as column projection, denoted as Vp, and the set Vp can be expressed as;

[0056] Vp={Vp j |j=0, 1, ...(W-1)}

[0057] In practical applications, step S3 and step S4 may be executed simultaneously or in any order.

[0058] Step S5: Scan the line projection to obtain the traveling wave peak and the traveling wave trough;

[0059] The step S5 of scanning the line projection to obtain the traveling wave peak and the traveling wave trough comprises: scanning the line projection line by line, and according to the size of the line projection value, taking the continuous interval of the line projection value less than the first threshold as the traveling wave trough, and taking the other areas except the traveling wave trough as the traveling wave peak.

[0060] In this embodiment, it specifically includes: scanning the line projection line by line, and according to the size of the line projection value, the continuous interval where the line projection value is less than the first threshold FHTh is called the line trough area, referred to as the line trough, recorded as Htrough. The other areas except the line trough are called the line peak area, referred to as the line peak, recorded as Hpeak. The set Htrough and the set Hpeak can be expressed as:

[0061] Htrough={(Hp1 i , Hp2 i )|i=0, 1,..., (H-1)∩Hp1 i <Hp2 i <FHTh}

[0062] Hpeak=Hp-Htrough.

[0063] Step S6: Scan the column projection to obtain column wave peaks and column wave troughs;

[0064] The step S6 of scanning the column projections to obtain column peaks and column troughs includes: scanning the column projections column by column, and according to the size of the column projection values, taking the continuous intervals in which the column projection values ​​are less than the second threshold as the column troughs, and taking the other areas except the column troughs as the column peaks.

[0065] In this embodiment, the method specifically includes: scanning the column projections column by column, and according to the magnitude of the column projection values, the continuous intervals in which the row projection values ​​are less than the second threshold FVTh are called column trough regions, referred to as column troughs, and recorded as Vtrough. The other regions except the column troughs are called column peak regions, referred to as column peaks, and recorded as Vpeak;

[0066] Vtrough={(Vp1 j , Vp2 j )|j=0, 1,..., (W-1)∩Vp1 j <Vp2 j <FVTh}

[0067] Vpeak=Vp-Vtrough.

[0068] In practical applications, step S5 and step S6 may be executed simultaneously or in any order.

[0069] Step S7: Calculate the table line position according to the row trough and column trough to obtain the table structure.

[0070] In this embodiment, the table structure obtained by calculating the position of the table lines according to the row trough and the column trough includes: drawing a horizontal line through the midpoint of the row trough, and drawing a vertical line through the midpoint of the column trough; the set of all horizontal lines forms a horizontal line set, and the set of all vertical lines forms a vertical line set; each horizontal line intersects with each vertical line to obtain an intersection point, and a point set is set to include all intersection points; the intersection point of the first line segment of the horizontal line set and the first line segment of the vertical line set is used as the upper left corner, the intersection point of the first line segment of the horizontal line set and the last line segment of the vertical line set is used as the upper right corner, the intersection point of the last line segment of the horizontal line set and the first line segment of the vertical line set is used as the lower left corner, and the intersection point of the last line segment of the horizontal line set and the last line segment of the vertical line set is used as the lower right corner to generate a table structure.

[0071] In this embodiment, the following steps are specifically included:

[0072] Step S71: taking the midpoints of the row trough and the column trough as the ordinate and abscissa of the horizontal and vertical line segments of the frameless table image respectively;

[0073] Step S72: Extend the horizontal line segment and the vertical line segment from the known ordinate and abscissa respectively. If there is text in the area where the line segment passes, divide the line segment into two segments to obtain a horizontal line set and a vertical line set.

[0074] Step S73: using the horizontal line set and the vertical line set to find the intersection of each horizontal line segment and each vertical line segment to obtain a point set;

[0075] Step S74: Take the intersection of the first line segment of the horizontal line set and the first line segment of the vertical line set as the upper left corner, take the intersection of the first line segment of the horizontal line set and the last line segment of the vertical line set as the upper right corner, take the intersection of the last line segment of the horizontal line set and the first line segment of the vertical line set as the lower left corner, take the intersection of the last line segment of the horizontal line set and the last line segment of the vertical line set as the lower right corner, and generate a table structure.

[0076] In this embodiment, the specific calculation process may be:

[0077] The midpoints of the row trough and column trough are the ordinate y and abscissa x of the horizontal and vertical line segments of the wireless frame table image respectively;

[0078] Extend the horizontal and vertical line segments from the known y-coordinates and x-coordinates respectively. If there is text in the area where the line segment passes, divide the line segment into two segments to obtain the horizontal line set Horizontal and the vertical line set Vertical;

[0079] Let n be the number of elements in Horizontal and m be the number of elements in Vertical. i is the i-th straight line in Horizontal, j is the jth straight line in Vertical;

[0080] Hx1 i Represents Horizontal i The horizontal coordinate of the starting point, Hy1 i Represents Horizontal i The vertical coordinate of the starting point, Hx2 i Represents Horizontal i The abscissa of the end point, Hy2 i Represents Horizontal i The ordinate of the end point;

[0081] Vx1 j Indicates Vertical j The horizontal coordinate of the starting point, Vy1 j Indicates Vertical j The vertical coordinate of the starting point, Vx2 j Indicates Vertical j The horizontal coordinate of the end point, Vx2 j Indicates Vertical j The ordinate of the end point;

[0082] Use Horizontal and Vertical to find the intersection of each horizontal line segment and each vertical line segment to get the point set P;

[0083] The intersection of Horizontal1 and Vertical1 is the upper left corner LT, Horizontal1 and Vertical m The intersection point is the upper right corner RT, Horizontal n The intersection point with Vertical1 is the lower left corner LL, Horizontal n and Vertical m The intersection point is the lower right corner RL, that is, the table structure of all table lines is obtained.

[0084] In practical applications, the method may further include: expanding the obtained table line outward by 10 pixels, that is:

[0085] Hy11=Hy11-10

[0086] Hy21=Hy21-10

[0087] Hy1n =Hy1 n +10

[0088] Hy2 n =Hy2 n +10

[0089] Vx11=Vx11-10

[0090] Vx21=Vx21-10

[0091] Vx1 m =Vx1 m +10

[0092] Vx2 m =Vx2 m +10

[0093] Get a complete table with margins next to the table.

[0094] The recognized wireless frame table image table structure is input into a document with the suffix json in a certain format.

[0095] Embodiment 2

[0096] like Figure 1 , Figure 3 , Figure 4 and Figure 5 As shown, a method for identifying a table structure of a wireless frame table image comprises the following steps:

[0097] Step S1: Obtain a wireless frame table image;

[0098] In this step, the wireless frame table image can be obtained by scanning or taking a photo, and the image can be in RGB or any image format. In this embodiment, the wireless frame table image Io is obtained, and the image height is H and the image width is W.

[0099] Step S2: Process the image and convert it into a grayscale image;

[0100] First, the wireless frame table image Io is binarized into a grayscale image Ig. Specifically, the image binarization algorithm is used to convert the obtained wireless frame table image Io from the original RGB image into a grayscale image Ig, and the grayscale value of the pixel point in the i-th row and j-th column in Ig is represented by Pix(i, j).

[0101] Step S3: Accumulate the pixel values ​​of the pixel points in a single row of the grayscale image to obtain a row projection of the grayscale image;

[0102] Step S31: Accumulate the pixel values ​​of the pixels in the i-th row of Ig to obtain the row projection of the row of Ig, denoted as Hp i, which can be expressed as the following formula:

[0103]

[0104] Pix(i, j) is the grayscale value of the pixel in the i-th row and j-th column in Ig.

[0105] Step S32: Repeat step S31 for each row of Ig to obtain the row projection of each row of Ig, which is called the row projection of the grayscale image, abbreviated as row projection, denoted as Hp. The set Hp can be expressed as:

[0106] Hp={Hp i |i=0,1,...(H-1)}

[0107] Step S4: Accumulate the pixel values ​​of the pixel points in a single column of the grayscale image to obtain the column projection of the grayscale image;

[0108] Step S41: Accumulate the pixel values ​​of the pixels in the jth column of Ig to obtain the column projection of the column of Ig, denoted as Vp i , which can be expressed as the following formula:

[0109]

[0110] Step S42: Repeat step S41 for each column of Ig to obtain the column projection of each column of Ig, which is called the column projection of the grayscale image, abbreviated as column projection, denoted as Vp, and the set Vp can be expressed as;

[0111] Vp={Vp j |j=0, 1, ...(W-1)}

[0112] In practical applications, step S3 and step S4 may be executed simultaneously or in any order.

[0113] Step S5: Scan the line projection to obtain the traveling wave peak and the traveling wave trough;

[0114] The step S5 of scanning the line projection to obtain the traveling wave peak and the traveling wave trough comprises: scanning the line projection line by line, and according to the size of the line projection value, taking the continuous interval of the line projection value less than the first threshold as the traveling wave trough, and taking the other areas except the traveling wave trough as the traveling wave peak.

[0115] In this embodiment, it specifically includes: scanning the line projection line by line, and according to the size of the line projection value, the continuous interval where the line projection value is less than the first threshold FHTh is called the line trough area, referred to as the line trough, recorded as Htrough. The other areas except the line trough are called the line peak area, referred to as the line peak, recorded as Hpeak. The set Htrough and the set Hpeak can be expressed as:

[0116] Htrough={(Hp1 i , Hp2 i )|i=0, 1,..., (H-1)∩Hp1 i <Hp2 i <FHTh}

[0117] Hpeak=Hp-Htrough.

[0118] Step S6: Scan the column projection to obtain column wave peaks and column wave troughs;

[0119] The step S6 of scanning the column projections to obtain column peaks and column troughs includes: scanning the column projections column by column, and according to the size of the column projection values, taking the continuous intervals in which the column projection values ​​are less than the second threshold as the column troughs, and taking the other areas except the column troughs as the column peaks.

[0120] In this embodiment, the method specifically includes: scanning the column projections column by column, and according to the magnitude of the column projection values, the continuous intervals in which the row projection values ​​are less than the second threshold FVTh are called column trough regions, referred to as column troughs, and recorded as Vtrough. The other regions except the column troughs are called column peak regions, referred to as column peaks, and recorded as Vpeak;

[0121] Vtrough={(Vp1 j , Vp2 j )|j=0, 1,..., (W-1)∩Vp1 j <Vp2 j <FVTh}

[0122] Vpeak=Vp-Vtrough.

[0123] In practical applications, step S5 and step S6 may be executed simultaneously or in any order.

[0124] Step S7: Calculate the table line position according to the row trough and column trough to obtain the table structure.

[0125] In this embodiment, the table structure obtained by calculating the position of the table lines according to the row trough and the column trough includes: drawing a horizontal line through the midpoint of the row trough, and drawing a vertical line through the midpoint of the column trough; the set of all horizontal lines forms a horizontal line set, and the set of all vertical lines forms a vertical line set; each horizontal line intersects with each vertical line to obtain an intersection point, and a point set is set to include all intersection points; the intersection point of the first line segment of the horizontal line set and the first line segment of the vertical line set is used as the upper left corner, the intersection point of the first line segment of the horizontal line set and the last line segment of the vertical line set is used as the upper right corner, the intersection point of the last line segment of the horizontal line set and the first line segment of the vertical line set is used as the lower left corner, and the intersection point of the last line segment of the horizontal line set and the last line segment of the vertical line set is used as the lower right corner to generate a table structure.

[0126] In this embodiment, the following steps are specifically included:

[0127] Step S71: taking the midpoints of the row trough and the column trough as the ordinate and abscissa of the horizontal and vertical line segments of the frameless table image respectively;

[0128] Step S72: Extend the horizontal line segment and the vertical line segment from the known ordinate and abscissa respectively. If there is text in the area where the line segment passes, divide the line segment into two segments to obtain a horizontal line set and a vertical line set.

[0129] Step S73: using the horizontal line set and the vertical line set to find the intersection of each horizontal line segment and each vertical line segment to obtain a point set;

[0130] Step S74: Take the intersection of the first line segment of the horizontal line set and the first line segment of the vertical line set as the upper left corner, take the intersection of the first line segment of the horizontal line set and the last line segment of the vertical line set as the upper right corner, take the intersection of the last line segment of the horizontal line set and the first line segment of the vertical line set as the lower left corner, take the intersection of the last line segment of the horizontal line set and the last line segment of the vertical line set as the lower right corner, and generate a table structure.

[0131] In this embodiment, the specific calculation process may be:

[0132] The midpoints of the row trough and column trough are the ordinate y and abscissa x of the horizontal and vertical line segments of the wireless frame table image respectively;

[0133] Extend the horizontal and vertical line segments from the known y-coordinates and x-coordinates respectively. If there is text in the area where the line segment passes, divide the line segment into two segments to obtain the horizontal line set Horizontal and the vertical line set Vertical;

[0134] Let n be the number of elements in Horizontal and m be the number of elements in Vertical. i is the i-th straight line in Horizontal, j is the jth straight line in Vertical;

[0135] Hx1 i Represents Horizontal i The horizontal coordinate of the starting point, Hy1 i Represents Horizontal i The vertical coordinate of the starting point, Hx2 i Represents Horizontal i The abscissa of the end point, Hy2 i Represents Horizontal iThe ordinate of the end point;

[0136] Vx1 j Indicates Vertical j The horizontal coordinate of the starting point, Vy1 j Indicates Vertical j The vertical coordinate of the starting point, Vx2 j Indicates Vertical j The horizontal coordinate of the end point, Vx2 j Indicates Vertical j The ordinate of the end point;

[0137] Use Horizontal and Vertical to find the intersection of each horizontal line segment and each vertical line segment to get the point set P;

[0138] The intersection of Horizontal1 and Vertical1 is the upper left corner LT, Horizontal1 and Vertical m The intersection point is the upper right corner RT, Horizontal n The intersection point with Vertical1 is the lower left corner LL, Horizontal n and Vertical m The intersection point is the lower right corner RL, that is, the table structure of all table lines is obtained.

[0139] In practical applications, the method may further include: expanding the obtained table line outward by 10 pixels, that is:

[0140] Hy11=Hy11-10

[0141] Hy21=Hy21-10

[0142] Hy1 n =Hy1 n +10

[0143] Hy2 n =Hy2 n +10

[0144] Vx11=Vx11-10

[0145] Vx21=Vx21-10

[0146] Vx1 m =Vx1 m +10

[0147] Vx2 m =Vx2 m +10

[0148] Get a complete table with margins next to the table.

[0149] Step S8: Perform straight line detection on the grayscale image and modify the table structure according to the detected straight lines.

[0150] The recognized wireless frame table image table structure is input into a document with the suffix json in a certain format.

[0151] Embodiment 3

[0152] like Figure 2 , Figure 3 , Figure 4 and Figure 5 As shown, a method for identifying a table structure of a wireless frame table image comprises the following steps:

[0153] Step S1: Obtain a wireless frame table image;

[0154] In this step, the wireless frame table image can be obtained by scanning or taking a photo, and the image can be in RGB or any image format. In this embodiment, the wireless frame table image Io is obtained, and the image height is H and the image width is W.

[0155] Step S2: Process the image and convert it into a grayscale image;

[0156] First, the wireless frame table image Io is binarized into a grayscale image Ig. Specifically, the image binarization algorithm is used to convert the obtained wireless frame table image Io from the original RGB image into a grayscale image Ig, and the grayscale value of the pixel point in the i-th row and j-th column in Ig is represented by Pix(i, j).

[0157] Step S3: Accumulate the pixel values ​​of the pixel points in a single row of the grayscale image to obtain a row projection of the grayscale image;

[0158] Step S31: Accumulate the pixel values ​​of the pixels in the i-th row of Ig to obtain the row projection of the row of Ig, denoted as Hp i , which can be expressed as the following formula:

[0159]

[0160] Pix(i, j) is the grayscale value of the pixel in the i-th row and j-th column in Ig.

[0161] Step S32: Repeat step S31 for each row of Ig to obtain the row projection of each row of Ig, which is called the row projection of the grayscale image, abbreviated as row projection, denoted as Hp. The set Hp can be expressed as:

[0162] Hp={Hp i |i=0,1,...(H-1)}

[0163] Step S4: Accumulate the pixel values ​​of the pixel points in a single column of the grayscale image to obtain the column projection of the grayscale image;

[0164] Step S41: Accumulate the pixel values ​​of the pixels in the jth column of Ig to obtain the column projection of the column of Ig, denoted as Vp i , which can be expressed as the following formula:

[0165]

[0166] Step S42: Repeat step S41 for each column of Ig to obtain the column projection of each column of Ig, which is called the column projection of the grayscale image, abbreviated as column projection, denoted as Vp, and the set Vp can be expressed as;

[0167] Vp={Vp j |j=0, 1, ...(W-1)}

[0168] In practical applications, step S3 and step S4 may be executed simultaneously or in any order.

[0169] Step S5: Scan the line projection to obtain the traveling wave peak and the traveling wave trough;

[0170] The step S5 of scanning the line projection to obtain the traveling wave peak and the traveling wave trough comprises: scanning the line projection line by line, and according to the size of the line projection value, taking the continuous interval of the line projection value less than the first threshold as the traveling wave trough, and taking the other areas except the traveling wave trough as the traveling wave peak.

[0171] In this embodiment, it specifically includes: scanning the line projection line by line, and according to the size of the line projection value, the continuous interval where the line projection value is less than the threshold FHTh is called the line trough area, referred to as the line trough, recorded as Htrough. The other areas except the line trough are called the line peak area, referred to as the line peak, recorded as Hpeak. The set Htrough and the set Hpeak can be expressed as:

[0172] Htrough={(Hp1 i , Hp2 i )|i=0, 1,..., (H-1)∩Hp1 i <Hp2 i <FHTh}

[0173] Hpeak=Hp-Htrough.

[0174] Step S6: Scan the column projection to obtain column wave peaks and column wave troughs;

[0175] The step S6 of scanning the column projections to obtain column peaks and column troughs includes: scanning the column projections column by column, and according to the size of the column projection values, taking the continuous intervals in which the column projection values ​​are less than the second threshold as the column troughs, and taking the other areas except the column troughs as the column peaks.

[0176] In this embodiment, the method specifically includes: scanning the column projections column by column, and according to the magnitude of the column projection values, the continuous intervals in which the row projection values ​​are less than the threshold value FVTh are called column trough regions, referred to as column troughs, and recorded as Vtrough. The other regions except the column troughs are called column peak regions, referred to as column peaks, and recorded as Vpeak;

[0177] Vtrough={(Vp1 j , Vp2 j )|j=0, 1,..., (W-1)∩Vp1 j <Vp2 j <FVTh}

[0178] Vpeak=Vp-Vtrough.

[0179] In practical applications, step S5 and step S6 may be executed simultaneously or in any order.

[0180] The row troughs and column troughs obtained in step S5 and step S6 are collectively referred to as troughs, denoted as troughs, and the row peaks and column peaks are collectively referred to as peaks, denoted as peaks:

[0181] trough=Htrough+Vtrough

[0182] peak=Hpeak+Vpeak

[0183] In order to distinguish them from the troughs and peaks obtained in the subsequent steps, the traveling troughs and traveling peaks obtained in step S5 and step S6 are called the first traveling troughs and the first traveling peaks, respectively recorded as First_Htrough and First_Hpeak; the column troughs and column peaks are called the first column troughs and the first column peaks, respectively recorded as First_Vtrough and First_Vpeak; the troughs and peaks are called the first troughs and the first peaks, respectively recorded as First_trough and First_peak, and the expression formulas and set elements are the same as those in the previous steps. The traveling troughs and traveling peaks obtained in the subsequent steps are called the second traveling troughs and the second traveling peaks, respectively recorded as Second_Htrough and Second_Hpeak; the column troughs and column peaks are called the second column troughs and the second column peaks, respectively recorded as Second_Vtrough and Second_Vpeak; the troughs and peaks are called the second troughs and the second peaks, respectively recorded as Second_trough and Second_peak.

[0184] Step S7: Scan the Hang wave peak and the traveling wave trough respectively to obtain the secondary Hang wave peak and the secondary traveling wave trough;

[0185] On the basis of the traveling wave peaks, the traveling wave peak with the smallest range is taken as the traveling wave peak unit;

[0186] Scan a traveling wave peak starting from the traveling wave peak unit with fixed pixel points above and below as scanning frames;

[0187] If the average gray value of the traveling wave peak in the scanning frame is less than the average gray value of this traveling wave peak, it is considered that the traveling wave peak in the scanning frame belongs to the secondary traveling wave valley part, and a secondary traveling wave peak is obtained;

[0188] Take the end point of the secondary traveling wave peak as the seven scanning points and use the scanning frame to scan row by row. When the average grayscale value of the traveling wave peak in the scanning frame after scanning the secondary traveling wave trough is greater than the average grayscale value of this traveling wave peak, it is considered that the scanning of a secondary traveling wave trough is completed and a secondary traveling wave trough is obtained.

[0189] In this embodiment, it can be specifically:

[0190] Step S71: Secondary peak scanning. PR is the peak range, First_Hpeak v is the vth first traveling wave peak in First_Hpeak, First_Hpeak1 v is the starting position of its peak, First_Hpeak2 v is the end position of its peak, and its peak range PR v It can be expressed as:

[0191] PRv =First_Hpeak2 v -First_Hpeak1 v

[0192] Step S72: Based on the first traveling wave peak First_Hpeak, the first traveling wave peak with the smallest range is called the traveling wave peak unit, denoted as PU, that is:

[0193] PU=min(PR)

[0194] Step S73: Using the roughly equidistant property of the row, a row peak is scanned starting from PU with 5 pixels above and below as the scanning frame. The scanning frame is recorded as: Scan_Box((x, y), h, w), (x, y) represents the coordinates of the upper left corner of the scanning frame, and h represents the rectangular frame. The scanning frame of the i-th row can be expressed as:

[0195] Scan_Box i ((i-5, 0), 10, W) (i=PU, (PU+1),...,)

[0196] Step S74: If the average grayscale value PartAve of the wave peak in the scan frame is less than the average grayscale value TotalAve of the primary wave peak, that is, PartAve<TotalAve, then it is considered that the wave peak in the scan frame belongs to the secondary wave valley. A secondary wave peak scan is completed, and a secondary wave peak Second_Hpeak1 (Hp1 i , Hp2 i ).

[0197] Step S75: Scan the secondary wave valley, the ith secondary wave peak Second_Hpeak i The end point Hp2 i As the starting point of scanning, use the scanning box Scan_Box to scan line by line. When PartAve>TotalAve appears after scanning the secondary traveling trough, it is considered that the scanning of a secondary traveling trough is completed, and the i-th secondary traveling trough Second_Htrough is obtained. i (Hp2 i , Hp3 i );

[0198] Step S76: Take the ith secondary trough Second_Htrough i The end point of Hp3 i is the i+1th secondary peak Second_Hpeak i+1 The starting point is increased and PU is continued to perform the secondary wave peak scan, that is:

[0199] Hpi+1 =Hp3 i +PU

[0200] Step S77: repeating step S75 and step S76, and stopping the repetition when all the primary peaks are scanned, or when the remaining part of the secondary wave valley is less than one peak unit after the secondary wave peak is scanned;

[0201] Step S78: Repeat steps S71 to S77 for each primary wave peak to obtain a secondary wave peak and a secondary wave trough, that is:

[0202] Second_Hpeak={Second_Hpeak i |0≤i<H}

[0203] Second_Htrough={Second_Htrough i |0≤i<H}+First_Htrough.

[0204] Step S8: Scan the wave crest and wave trough to obtain the secondary wave crest and wave trough respectively;

[0205] On the basis of obtaining the wave valleys and wave peaks, a wave peak is scanned from left to right to find the maximum value point set and the minimum value point set in the wave peak;

[0206] Calculate the slopes of the maximum point and its nearby points and the minimum point and its nearby points on both sides respectively, and get the left slope set and the right slope set;

[0207] The part with the largest slope on both sides obtained with a maximum point as the center is a secondary wave peak. Except for the detected secondary wave peak, all other parts of a wave peak are secondary wave troughs.

[0208] Step S81: Secondary column wave peak scanning: Based on the obtained primary column wave valley and primary column wave peak, a primary column wave peak is scanned from left to right to find the maximum value point set Maximum_Point and the minimum value point set Minimum_Point in the primary column wave peak.

[0209] Step S82: Calculate the slopes of the maximum point and its nearby points and the minimum point and its nearby points on both sides respectively, and obtain the left slope set LiftK and the right slope set RightK;

[0210] Step S83: The part with the largest slope on both sides, centered at a maximum point, is a secondary wave peak Second_Vpeak j , except for the detected secondary wave peak, all other primary wave peaks are secondary wave valleys, that is:

[0211] Second_Vtrough j =First_Vpeak j -Second_Vpeak j

[0212] Step S84: Repeat steps S81 to S83 for each primary wave peak to obtain secondary wave peaks and secondary wave troughs, namely:

[0213] Second_Vpeak={Second_Vpeak j |0≤j<W}

[0214] Second_Vtrough={Second_Vtrough j |0≤j<W}+First_Vtrough.

[0215] Step S9: Calculate the table line position according to the row trough and column trough to obtain the table structure.

[0216] In this embodiment, the table structure obtained by calculating the position of the table lines according to the row trough and the column trough includes: drawing a horizontal line through the midpoint of the secondary row trough, and drawing a vertical line through the midpoint of the secondary column trough; the set of all horizontal lines forms a horizontal line set, and the set of all vertical lines forms a vertical line set; each horizontal line intersects with each vertical line to obtain an intersection point, and a point set is set to include all intersection points; the intersection point of the first line segment of the horizontal line set and the first line segment of the vertical line set is used as the upper left corner, the intersection point of the first line segment of the horizontal line set and the last line segment of the vertical line set is used as the upper right corner, the intersection point of the last line segment of the horizontal line set and the first line segment of the vertical line set is used as the lower left corner, and the intersection point of the last line segment of the horizontal line set and the last line segment of the vertical line set is used as the lower right corner to generate a table structure.

[0217] In this embodiment, the following steps are specifically included:

[0218] Step S91: taking the midpoints of the secondary row trough and the secondary column trough as the ordinate and abscissa of the horizontal line segment and the vertical line segment of the wireless frame table image respectively;

[0219] Step S92: Extend the horizontal line segment and the vertical line segment from the known ordinate and abscissa respectively. If there is text in the area where the line segment passes, divide the line segment into two segments to obtain a horizontal line set and a vertical line set.

[0220] Step S93: using the horizontal line set and the vertical line set to find the intersection of each horizontal line segment and each vertical line segment to obtain a point set;

[0221] Step S94: Take the intersection of the first line segment of the horizontal line set and the first line segment of the vertical line set as the upper left corner, take the intersection of the first line segment of the horizontal line set and the last line segment of the vertical line set as the upper right corner, take the intersection of the last line segment of the horizontal line set and the first line segment of the vertical line set as the lower left corner, take the intersection of the last line segment of the horizontal line set and the last line segment of the vertical line set as the lower right corner, and generate a table structure.

[0222] In this embodiment, the specific calculation process may be:

[0223] The midpoints of the row trough and column trough are the ordinate y and abscissa x of the horizontal and vertical line segments of the wireless frame table image respectively;

[0224] Extend the horizontal and vertical line segments from the known y-coordinates and x-coordinates respectively. If there is text in the area where the line segment passes, divide the line segment into two segments to obtain the horizontal line set Horizontal and the vertical line set Vertical;

[0225] Let n be the number of elements in Horizontal and m be the number of elements in Vertical. i is the i-th straight line in Horizontal, i is the jth straight line in Vertical;

[0226] Hx1 i Represents Horizontal i The horizontal coordinate of the starting point, Hy1 i Represents Horizontal i The vertical coordinate of the starting point, Hx2 i Represents Horizontal i The abscissa of the end point, Hy2 i Represents Horizontal i The ordinate of the end point;

[0227] Vx1 j Indicates Vertical j The horizontal coordinate of the starting point, Vy1 j Indicates Vertical j The vertical coordinate of the starting point, Vx2 j Indicates Vertical j The horizontal coordinate of the end point, Vx2 j Indicates Vertical j The ordinate of the end point;

[0228] Use Horizontal and Vertical to find the intersection of each horizontal line segment and each vertical line segment to get the point set P;

[0229] The intersection of Horizontal1 and Vertical1 is the upper left corner LT, Horizontal1 and Vertical m The intersection point is the upper right corner RT, Horizontal n The intersection point with Vertical1 is the lower left corner LL, Horizontal n and Vertical m The intersection point is the lower right corner RL, that is, the table structure of all table lines is obtained.

[0230] In practical applications, the method may further include: expanding the obtained table line outward by 10 pixels, that is:

[0231] Hy11=Hy11-10

[0232] Hy21=Hy21-10

[0233] Hy1 n =Hy1 n +10

[0234] Hy2 n =Hy2 n +10

[0235] Vx11=Vx11-10

[0236] Vx21=Vx21-10

[0237] Vx1 m =Vx1 m +10

[0238] Vx2 m =Vx2 m +10

[0239] Get a complete table with margins next to the table.

[0240] The recognized wireless frame table image table structure is input into a document with the suffix json in a certain format.

[0241] The algorithm and display provided herein are not inherently related to any particular computer, virtual system or other device. Various general purpose systems can also be used together with the teachings based on this. According to the above description, it is obvious that the structure required for constructing such systems. In addition, the present invention is not directed to any specific programming language either. It should be understood that various programming languages ​​can be utilized to realize the content of the present invention described herein, and the description of the above specific languages ​​is for disclosing the best mode of the present invention.

[0242] In the description provided herein, a large number of specific details are described. However, it is understood that embodiments of the present invention can be practiced without these specific details. In some instances, well-known methods, structures and techniques are not shown in detail so as not to obscure the understanding of this description.

[0243] Similarly, it should be understood that in order to streamline the present disclosure and aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of the present invention, various features of the present invention are sometimes grouped together into a single embodiment, figure, or description thereof. However, this disclosed method should not be interpreted as reflecting the intention that the claimed invention requires more features than those explicitly recited in each claim. More specifically, as reflected in the claims below, inventive aspects lie in less than all the features of the individual embodiments disclosed above. Therefore, the claims that follow the specific embodiment are hereby expressly incorporated into the specific embodiment, with each claim itself serving as a separate embodiment of the present invention.

[0244] Those skilled in the art will appreciate that the modules in the devices in the embodiments may be adaptively changed and arranged in one or more devices different from the embodiments. The modules or units or components in the embodiments may be combined into one module or unit or component, and in addition they may be divided into a plurality of submodules or subunits or subcomponents. Except that at least some of such features and / or processes or units are mutually exclusive, all features disclosed in this specification (including the accompanying claims, abstracts and drawings) and all processes or units of any method or device disclosed in this manner may be combined in any combination. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstracts and drawings) may be replaced by an alternative feature providing the same, equivalent or similar purpose.

[0245] In addition, those skilled in the art will appreciate that, although some embodiments herein include certain features included in other embodiments but not other features, the combination of features of different embodiments is meant to be within the scope of the present invention and form different embodiments. For example, in the claims below, any one of the claimed embodiments may be used in any combination.

[0246] It should be noted that the above embodiments illustrate the present invention rather than limit it, and that those skilled in the art may devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference symbol between brackets shall not be construed as a limitation on the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "one" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present invention may be implemented by means of hardware comprising a number of different elements and by means of a suitably programmed computer. In a unit claim enumerating a number of devices, several of these devices may be embodied by the same hardware item. The use of the words first, second, and third, etc., does not indicate any order. These words may be interpreted as names.

[0247] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by any technician familiar with the art within the technical scope disclosed in the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A method for identifying a table structure of a wireless frame table image, the method comprising the steps of: acquiring an image; processing the image and converting it into a grayscale image; characterized in that: The method further comprises the following steps: Accumulate the pixel values ​​of the pixels in a single row of the grayscale image to obtain the row projection of the grayscale image; Accumulate the pixel values ​​of the pixels in a single column of the grayscale image to obtain the column projection of the grayscale image; Scan the line projection to obtain the traveling wave peak and traveling wave trough; Scan the column projection to obtain column wave peaks and column wave troughs; The table structure is obtained by calculating the table line position according to the row trough and column trough, including: Draw horizontal lines through the midpoints of the traveling troughs, and draw vertical lines through the midpoints of the column troughs; the collection of all horizontal lines forms a horizontal line set, and the collection of all vertical lines forms a vertical line set; Each horizontal line intersects with each vertical line to get an intersection point, and a point set is set to include all intersection points; The intersection of the first line segment of the horizontal line set and the first line segment of the vertical line set is taken as the upper left corner, the intersection of the first line segment of the horizontal line set and the last line segment of the vertical line set is taken as the upper right corner, the intersection of the last line segment of the horizontal line set and the first line segment of the vertical line set is taken as the lower left corner, and the intersection of the last line segment of the horizontal line set and the last line segment of the vertical line set is taken as the lower right corner to generate a table structure.

2. The method for identifying a table structure of a wireless frame table image according to claim 1, characterized in that: Scanning the line projection to obtain the traveling wave peak and the traveling wave trough includes: scanning the line projection line by line, and according to the size of the line projection value, taking the continuous interval of the line projection value less than the first threshold as the traveling wave trough, and taking the other area except the traveling wave trough as the traveling wave peak.

3. The method for identifying a table structure of a wireless frame table image according to claim 1, characterized in that: The scanning of the column projections to obtain column wave peaks and column wave troughs includes: scanning the column projections column by column, and according to the magnitude of the column projection values, taking the continuous intervals in which the column projection values ​​are less than the second threshold as the column wave troughs, and taking the other areas except the column wave troughs as the column wave peaks.

4. The method for identifying a table structure of a wireless frame table image according to claim 1, characterized in that: The method comprises: after the step of scanning the row projection to obtain the traveling wave peak and the traveling wave trough is executed, the traveling wave peak and the traveling wave trough are scanned again to obtain the secondary traveling wave peak and the secondary traveling wave trough respectively, and the secondary traveling wave trough is used as the basis for calculating the table line position to obtain the table structure.

5. The method for identifying the table structure of a wireless frame table image according to claim 4, characterized in that: The scanning of the traveling wave peak and the traveling wave trough to obtain the secondary traveling wave peak and the secondary traveling wave trough comprises the following steps: On the basis of the traveling wave peaks, the traveling wave peak with the smallest range is taken as the traveling wave peak unit; Scan a traveling wave peak starting from the traveling wave peak unit with fixed pixel points above and below as scanning frames; If the average gray value of the traveling wave peak in the scanning frame is less than the average gray value of this traveling wave peak, it is considered that the traveling wave peak in the scanning frame belongs to the secondary traveling wave valley part, and a secondary traveling wave peak is obtained; Take the end point of the secondary traveling wave peak as the seven scanning points and use the scanning frame to scan row by row. When the average grayscale value of the traveling wave peak in the scanning frame after scanning the secondary traveling wave trough is greater than the average grayscale value of this traveling wave peak, it is considered that the scanning of a secondary traveling wave trough is completed and a secondary traveling wave trough is obtained.

6. The method for identifying a table structure of a wireless frame table image according to claim 1, characterized in that: The method comprises: after the step of scanning the column projection to obtain the column wave peak and the column wave trough is executed, the column wave peak and the column wave trough are scanned again to obtain the secondary column wave peak and the secondary column wave trough respectively, and the secondary column wave trough is used as the basis for calculating the table line position to obtain the table structure.

7. The method for identifying the table structure of a wireless frame table image according to claim 6, characterized in that: The step of scanning the wave crests and wave troughs to obtain the secondary wave crests and wave troughs comprises the following steps: On the basis of obtaining the wave trough and wave peak, a wave peak is scanned from left to right to find the maximum value point set and the minimum value point set in the wave peak; Calculate the slopes of the maximum point and its nearby points and the minimum point and its nearby points on both sides respectively, and get the left slope set and the right slope set; The part with the largest slope on both sides obtained with a maximum point as the center is a secondary wave peak. Except for the detected secondary wave peak, all other parts of a wave peak are secondary wave troughs.

8. The method for identifying a table structure of a wireless frame table image according to any one of claims 1 to 7, characterized in that: The method also includes the steps of: performing straight line detection on the grayscale image, and correcting the table structure according to the detected straight lines.

9. The method for identifying a table structure of a wireless frame table image according to claim 8, characterized in that: The method comprises: inputting the recognized table structure into a document with a suffix of json in a certain format.

Images