A method for detecting merged cells in a table

Abstract

The present invention discloses a method for detecting merged cells in a table. The method comprises the following steps: acquiring an image containing a table; performing image binarization processing to obtain a binary image; identifying the binary image to obtain all frame lines and intersection sets of a table structure; obtaining a cell set through the intersection set; calculating the average value of the grayscale value of each cell in a first column in units of cells to obtain an average value set of the grayscale values of the cells in the first column; calculating the median of all element values in the average value set; comparing the value of each element in the average value set with a predetermined multiple of the median, and if the element value is less than the predetermined multiple of the median, determining that there is no text at the beginning of a row corresponding to the element value, and merging the row with the previous row; and outputting the merged table structure, thereby making table recognition more accurate.

Description

Technical Field

[0001] The present invention relates to the technical field of image table recognition, and in particular to a method for detecting merged cells in a table. Background Art

[0002] With the rapid development of computer technology, the use of electronic documents is becoming more and more popular. Due to the problems of paper documents being easily damaged and difficult to preserve, it is often necessary to convert paper documents into digital form and store them in computers. For example, paper documents can be converted into digital images through technologies such as photography or scanning. Then, image processing and optical character recognition (OCR) can be used to recognize the text in the digital image for further editing or other applications. Among these paper documents, there is a special type of document - table documents, such as bank statements, value-added tax invoices, company reports, etc., which are widely used in our daily lives and work. Document table lines divide the entire document into different areas and are an important basis for document layout analysis and information structured processing. Therefore, research on the detection and removal of table lines in document images is of great significance.

[0003] In some tables, a column is too narrow, and the content of a cell in that column is much longer than the other cells in the column, causing the cell to wrap and require two rows. During the table detection and recognition process, these two rows should be merged, but there is currently no suitable solution. Summary of the Invention

[0004] In view of the problem in the current field of image table recognition technology that two or more rows cannot be merged to generate a table during detection, the present invention provides a table merge cell detection method, which can determine that there is no text in a cell based on the projection of the first column of a detected row being much lower than the first column of other rows, and merge the cell with the previous row to output the merged table structure.

[0005] To achieve the above objectives, the embodiments of the present invention adopt the following technical solutions:

[0006] A method for detecting merged cells in a table, the method comprising the following steps:

[0007] Get an image containing a table;

[0008] The image is binarized to obtain a binary image;

[0009] Recognize the binary image to obtain all the frame lines and intersection points of the table structure;

[0010] Get the cell set through the intersection set;

[0011] In the first column, the average value of the grayscale value of each cell is calculated in units of cells to obtain the average value set of the grayscale values of the cells in the first column;

[0012] Calculate the median of all element values in the mean set;

[0013] Compare the value of each element in the average value set with a predetermined multiple of the median. If the element value is less than the predetermined multiple of the median, determine that there is no text at the beginning of the row corresponding to the element value, and merge the row with the previous row.

[0014] Output the merged table structure.

[0015] According to one aspect of the present invention, the step of identifying the binary image to obtain all frame lines and intersection points of the table structure includes: identifying the binary image using a table recognition algorithm to obtain the table cell structure, and obtaining the horizontal line set and the vertical line set.

[0016] According to one aspect of the present invention, the step of identifying the binary image to obtain all frame lines and intersection points of the table structure includes: using the horizontal line set and the vertical line set to calculate the intersection point of each horizontal line segment and each vertical line segment to obtain the intersection point set.

[0017] According to one aspect of the present invention, the method of obtaining a cell set through an intersection set includes: expressing the cell set as Cell, where Cell(i,j) represents the cell in the i-th row and j-th column, and there are a total of n rows and m columns of cells; HCell i Represents all cells in row i, VCell j Represents all cells in column j, H i Indicates the row height of the cell in row i, W j Indicates the width of the cell in column j.

[0018] According to one aspect of the present invention, in the first column, the average grayscale value of each cell is calculated in units of cells to obtain the average value set of grayscale values of cells in the first column, including: representing the grayscale value of the pixel point in the i-th row and j-th column in VCell1 with Pix(i,j).

[0019] According to one aspect of the present invention, in the first column, the average value of the grayscale value of each cell is calculated in units of cells to obtain the average value set of the grayscale values of the cells in the first column, including: in VCell1, the average value of the grayscale value of each cell is calculated starting from the first cell to obtain the average value set PA, PA of the grayscale values of the cells in the first column. i Represents the average gray value of the i-th cell, that is:

[0020]

[0021] PA={PA i |1≤i≤n}.

[0022] According to one aspect of the present invention, the value of each element in the average value set is compared with a predetermined multiple of the median. If the element value is less than the predetermined multiple of the median, then determining whether the beginning of the row corresponding to the element value does not have text includes: searching all elements in the set PA one by one, and if the element PA exists i Less than PAM multiplied by the threshold (predetermined multiple) Threshold, that is, if:

[0023] PA i <PAM*Threshold

[0024] It is determined that there is no text at the beginning of the line.

[0025] According to one aspect of the present invention, obtaining all frame lines of the table structure includes obtaining a horizontal line set Horizontal and a vertical line set Vertical, Horizontal i is the i-th straight line in Horizontal, Vertical j is the jth line in Vertical; Hx1 i Represents Horizontal i The horizontal coordinate of the starting point, Hy1 i Represents Horizontal i The vertical coordinate of the starting point, Hx2 i Represents Horizontal i The horizontal coordinate of the end point, Hy2 i Represents Horizontal i The vertical coordinate of the end point; Vx1 j Indicates Vertical j The horizontal coordinate of the starting point, Vy1 j Indicates Vertical j The vertical coordinate of the starting point, Vx2 j Indicates Vertical j The horizontal coordinate of the end point, Vx2 j Indicates Vertical j The vertical coordinate of the end point.

[0026] According to one aspect of the present invention, the specific formula for merging the row with the previous row is:

[0027] Hx1 i =Hx1 i-1

[0028] Hx2 i =Hx2i-1

[0029] Horizontal=Horizontal-Horizontal i

[0030] According to one aspect of the present invention, outputting the merged table structure includes: inputting the merged table structure into a document with a suffix of json in a certain format.

[0031] Advantages of the present invention:

[0032] Based on the first column of a table, if a cell in that column doesn't have any content, it means that other columns in the row where that cell resides have "cell content that's significantly longer than the rest of the cells in that column, requiring two rows." If the projection of the first column of a detected row is significantly lower than the first column of another row, the cell is assumed to contain no text and is merged with the previous row. This avoids splitting cells in other columns that occupy multiple rows, preventing problems in later table processing and making table recognition more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0034] Figure 1 This is a flowchart of the table merge cell detection method described in the present invention. DETAILED DESCRIPTION

[0035] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0036] like Figure 1 As shown, a method for detecting merged cells in a table includes the following steps:

[0037] Step S1: obtaining an image containing a table;

[0038] In this step, a table image containing a table can be obtained by scanning or taking a photo, and the image can be in RGB or any other image format. In this embodiment, the obtained table image Io has an image height of H and an image width of W.

[0039] Step S2: binarization of the image to obtain a binary image;

[0040] First, the wireless frame table image Io is binarized into a binary image Ig. Specifically, the image binarization algorithm is used to convert the obtained wireless frame table image Io from the original RGB image into a binary grayscale image Ig.

[0041] Step S3: Recognize the binary image to obtain all frame lines and intersection points of the table structure;

[0042] In step S3, the binary image is identified to obtain all the frame lines and intersection sets of the table structure. The identification process can be performed by a variety of recognition algorithms. For example, the table line recognition method described in Chinese Patent ZL201910161725.1 can be used to identify the binary image to obtain all the frame lines of the table structure in this embodiment. Alternatively, a table recognition method such as that described in Chinese Patent ZL 201911375389.7 can be used to identify the binary image to obtain all the frame lines of the table structure in this embodiment. Other table recognition algorithms disclosed in the art can be used to identify and obtain the table frame lines in this embodiment. The frame lines include a horizontal line set and a vertical line set.

[0043] In this embodiment, steps S1 to S3 use the following technical solutions to realize the recognition of the table structure:

[0044] A method for identifying a table structure of a frameless table image, the method comprising the following steps:

[0045] Step S100: Acquire a frameless table image;

[0046] In this step, the frameless table image can be obtained by scanning or taking a photo, and the image can be in RGB or any other image format. In this embodiment, the frameless table image Io is obtained, and its image height is H and its image width is W.

[0047] Step S200: Processing the image and converting it into a binary image;

[0048] First, the wireless frame table image Io is binarized into a binary image Ig. Specifically, the image binarization algorithm is used to convert the obtained wireless frame table image Io from the original RGB image into a binary image Ig, and the grayscale value of the pixel point in the i-th row and j-th column in Ig is represented by Pix(i,j).

[0049] Step S300: Accumulate the pixel values of the pixels in a single row of the binary image to obtain a row projection of the binary image;

[0050] Step S301: Accumulate the pixel values of the pixels in the i-th row of Ig to obtain the row projection of Ig, which is recorded as Hp i , which can be expressed as the following formula:

[0051]

[0052] Pix(i,j) is the grayscale value of the pixel in the i-th row and j-th column in Ig.

[0053] Step S302: Repeat step S301 for each row of Ig to obtain the row projection of each row of Ig, which is called the row projection of the binary image, abbreviated as row projection, denoted as Hp. The set Hp can be expressed as:

[0054] Hp={Hp i |i=0,1,...(H-1)}

[0055] Step S400: Accumulate the pixel values of the pixels in a single column of the binary image to obtain a column projection of the binary image;

[0056] Step S401: Accumulate the pixel values of the pixels in the jth column of Ig to obtain the column projection of Ig, which is recorded as Vp i , which can be expressed as the following formula:

[0057]

[0058] Step S402: Repeat step S401 for each column of Ig to obtain the column projection of each column of Ig, which is called the column projection of the binary image, abbreviated as column projection, denoted as Vp. The set Vp can be expressed as;

[0059] Vp={Vp j |j=0,1,...(W-1)}

[0060] In practical applications, step S300 and step S400 may be executed simultaneously or in any order.

[0061] Step S500: Scan the line projection to obtain the traveling wave peaks and traveling wave troughs;

[0062] The step S500 of scanning the row projection to obtain the row peaks and the row troughs includes: scanning the row projections row by row, and according to the size of the row projection values, taking the continuous intervals in which the row projection values are less than the first threshold as the row troughs, and taking the other areas except the row troughs as the row peaks.

[0063] In this embodiment, the process specifically includes scanning the row projections row by row. Based on the row projection values, the continuous intervals where the row projection values are less than the threshold FHTh are referred to as row trough regions, referred to as row troughs, and denoted as Htrough. Regions other than the row troughs are referred to as row peak regions, referred to as row peaks, and denoted as Hpeak. The sets Htrough and Hpeak can be expressed as follows:

[0064] Htrough={(Hp1 i ,Hp2 i )|i=0,1,...,(H-1)∩Hp1 i <Hp2 i <FHTh}

[0065] Hpeak=Hp-Htrough.

[0066] Step S600: Scan the column projection to obtain column peaks and column troughs;

[0067] The step S600 of scanning the column projections to obtain column peaks and column troughs includes: scanning the column projections column by column, and according to the magnitude of the column projection values, taking the continuous intervals where the column projection values are less than the second threshold as the column troughs, and taking the other areas except the column troughs as the column peaks.

[0068] In this embodiment, the process specifically includes scanning the column projections column by column, and defining the continuous intervals where the row projection values are less than the threshold value FVTh as column trough regions, referred to as column troughs and denoted as Vtrough. The regions other than the column troughs are referred to as column peak regions, referred to as column peaks and denoted as Vpeak.

[0069] Vtrough={(Vp1 j ,Vp2 j )|j=0,1,...,(W-1)∩Vp1 j <Vp2 j <FVTh}

[0070] Vpeak=Vp-Vtrough.

[0071] In practical applications, step S500 and step S600 may be executed simultaneously or in any order.

[0072] Step S700: Calculate the table line positions according to the row troughs and column troughs to obtain a table structure.

[0073] In this embodiment, the step of calculating the table line positions according to the row troughs and column troughs to obtain the table structure includes the following steps:

[0074] Step S701: taking the midpoints of the row trough and the column trough as the ordinate and abscissa of the horizontal and vertical line segments of the frameless table image respectively;

[0075] Step S702: Extend the horizontal line segment and the vertical line segment from the known ordinate and abscissa respectively. If the line segment passes through an area where text exists, split the line segment into two segments to obtain a horizontal line set and a vertical line set.

[0076] Step S703: using the horizontal line set and the vertical line set, finding the intersection of each horizontal line segment and each vertical line segment to obtain an intersection point set;

[0077] In this embodiment, the specific calculation process may be:

[0078] The midpoints of the row trough and column trough are the ordinate y and abscissa x of the horizontal and vertical line segments of the frameless table image respectively;

[0079] Extend the horizontal and vertical line segments from the known y-coordinates and x-coordinates respectively. If there is text in the area where the line segment passes, split the line segment into two segments to obtain the horizontal line set Horizontal and the vertical line set Vertical;

[0080] Let n be the number of elements in Horizontal and m be the number of elements in Vertical. i is the i-th straight line in Horizontal, Vertical j is the jth straight line in Vertical;

[0081] Hx1 i Represents Horizontal i The horizontal coordinate of the starting point, Hy1 i Represents Horizontal i The vertical coordinate of the starting point, Hx2 i Represents Horizontal i The horizontal coordinate of the end point, Hy2 i Represents Horizontal i The vertical coordinate of the end point;

[0082] Vx1 j Indicates Vertical j The horizontal coordinate of the starting point, Vy1 j Indicates Vertical j The vertical coordinate of the starting point, Vx2 j Indicates Vertical j The horizontal coordinate of the end point, Vx2 j Indicates Vertical j The vertical coordinate of the end point;

[0083] Use Horizontal and Vertical to find the intersection of each horizontal line segment and each vertical line segment to obtain the intersection point set P;

[0084] Step S4: obtaining a cell set through the intersection set;

[0085] According to the fact that a cell is composed of 4 vertices, the cell set is obtained through the intersection set, including: expressing the cell set as Cell, Cell(i,j) represents the cell in the i-th row and j-th column, and there are n rows and m columns of cells in total; HCell i Represents all cells in row i, VCell j Represents all cells in column j, H i Indicates the row height of the cell in row i, W j Indicates the width of the cell in column j.

[0086] Step S5: in the first column, calculating the average value of the grayscale value of each cell in units of cells, and obtaining the average value set of the grayscale values of the cells in the first column;

[0087] The grayscale value of the pixel at the i-th row and j-th column in VCell1 is represented by Pix(i,j).

[0088] In VCell1, starting from the first cell, the average value of the grayscale value of each cell is calculated to obtain the average value set PA of the grayscale value of the first column of cells, PA i Represents the average gray value of the i-th cell, that is:

[0089]

[0090] PA={PA i |1≤i≤n}.

[0091] Step S6: Calculate the median of all element values in the average value set;

[0092] Calculate the median PAM of all element values in the set PA.

[0093] Step S7: Compare the value of each element in the average value set with a predetermined multiple of the median. If the element value is less than the predetermined multiple of the median, determine that there is no text at the beginning of the row corresponding to the element value, and merge the row with the previous row.

[0094] Specifically: search all elements in the average value set one by one, if there is an element value less than the predetermined multiple of the median, then determine that there is no text at the beginning of the line; including: search all elements in the set PA one by one, if there is an element PA iLess than PAM multiplied by the threshold (predetermined multiple) Threshold, where the threshold is the predetermined multiple, that is:

[0095] If PA i <PAM*Threshold

[0096] It is judged that there is no text at the beginning of the line;

[0097] The specific formula for merging the row with the previous row includes: removing the horizontal line between the row and the previous row from the Horizontal set; the specific expression for removing the horizontal line between the row and the previous row from the Horizontal set is:

[0098] Hx1 i =Hx1 i-1

[0099] Hx2 i =Hx2 i-1

[0100] Horizontal=Horizontal-Horizontal i .

[0101] Step S8: Output the merged table structure.

[0102] Outputting the merged table structure includes: inputting the merged table structure into a document with a suffix of json in a certain format.

[0103] The algorithm and display provided herein are not inherently related to any particular computer, virtual system or other device. Various general-purpose systems can also be used together with the teachings based on this. According to the above description, it is obvious that the structure required for constructing this type of system. In addition, the present invention is not directed to any specific programming language. It should be understood that various programming languages can be utilized to realize the content of the present invention described herein, and the above description of specific languages is for the purpose of disclosing the best mode of the present invention.

[0104] In the description provided herein, numerous specific details are described. However, it is understood that embodiments of the present invention may be practiced without these specific details. In some instances, well-known methods, structures, and techniques are not shown in detail so as not to obscure the understanding of this description.

[0105] Similarly, it should be understood that in order to streamline the present disclosure and aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together into a single embodiment, figure, or description thereof. However, this disclosed method should not be interpreted as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim. Rather, as reflected in the claims below, inventive aspects lie in less than all the features of the individual embodiments disclosed above. Accordingly, the claims following the detailed description are hereby expressly incorporated into this detailed description, with each claim standing on its own as a separate embodiment of the invention.

[0106] Those skilled in the art will appreciate that the modules in the devices in the embodiments may be adaptively changed and arranged in one or more devices different from the embodiments. The modules or units or components in the embodiments may be combined into one module or unit or component, and in addition may be divided into multiple submodules or subunits or subcomponents. All features disclosed in this specification (including the accompanying claims, abstracts and drawings) and all processes or units of any method or device disclosed herein may be combined in any combination, except that at least some of such features and / or processes or units are mutually exclusive. Unless expressly stated otherwise, each feature disclosed in this specification (including the accompanying claims, abstracts and drawings) may be replaced by an alternative feature providing the same, equivalent or similar purpose.

[0107] Furthermore, those skilled in the art will appreciate that although some embodiments herein include certain features included in other embodiments but not other features, combinations of features from different embodiments are intended to be within the scope of the present invention and to form different embodiments. For example, in the claims below, any of the claimed embodiments may be used in any combination.

[0108] It should be noted that the above embodiments illustrate rather than limit the invention, and that those skilled in the art may devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between brackets should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present invention may be implemented by means of hardware comprising several different elements and by means of appropriately programmed computers. In a unit claim enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third etc. does not indicate any order. These words may be interpreted as names.

[0109] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A method for detecting merged cells in a table, comprising the steps of: acquiring an image containing a table; performing image binarization processing to obtain a binarized image; characterized in that: The method further comprises the following steps: Recognize the binary image to obtain all the frame lines and intersection points of the table structure; The cell set is obtained through the intersection set, where the cell set is expressed as Cell, Cell(i,j) represents the cell in the i-th row and j-th column, and all cells in the i-th row are HCell i , all cells in column j are VCell j , the row height of the cell in row i is H i , the width of the cell in column j is W j ; In the first column of cells VCell1, starting from the first cell, the average value of the grayscale value of each cell is calculated in units of cells to obtain the average value set PA of the grayscale value of the first column of cells, where PA = {PA i |1≤i≤n}, PA i represents the average grayscale value of the i-th cell, Pix(x,y) represents the grayscale value of the pixel at (x,y) in VCell1; Calculate the median PAM of all element values in the average value set PA; Compare the value of each element in the average value set PA with a predetermined multiple of the median. If the element value is less than the predetermined multiple Threshold of the median, that is, if PA i < PAM * Threshold, it is determined that there is no text at the beginning of the row corresponding to the element value, and this row is merged with the previous row; Output the merged table structure.

2. The table merge cell detection method according to claim 1, characterized in that: The method of identifying the binary image to obtain all frame lines and intersection points of the table structure includes: identifying the binary image by a table recognition algorithm to obtain the table cell structure, and obtaining the horizontal line set and the vertical line set.

3. The table merge cell detection method according to claim 2, characterized in that: The identifying of the binary image to obtain all frame lines and intersection points of the table structure includes: using the horizontal line set and the vertical line set to calculate the intersection point of each horizontal line segment and each vertical line segment to obtain the intersection point set.

4. The method for detecting merged cells in a table according to any one of claims 1 to 3, wherein: The obtaining of all frame lines of the table structure includes obtaining a horizontal line set Horizontal and a vertical line set Vertical.

5. The table merge cell detection method according to claim 4, characterized in that: Merging the row with the previous row includes removing the horizontal line between the row and the previous row from the Horizontal set. The specific expression for removing the horizontal line between the row and the previous row from the Horizontal set is: Hx1 i =Hx1 i-1 Hx2 i =Hx2 i-1 Horizontal=Horizontal-Horizontal i Among them, Hx1 i Represents Horizontal i The horizontal coordinate of the starting point, Hy1 i Represents Horizontal i The vertical coordinate of the starting point, Hx2 i Represents Horizontal i The horizontal coordinate of the end point, Hy2 i Represents Horizontal i The vertical coordinate of the end point.

6. The table merge cell detection method according to claim 1, characterized in that: Outputting the merged table structure includes: inputting the merged table structure into a document with a suffix of json in a certain format.

Images