Table information extraction method and electronic device

Abstract

The present application provides a method, apparatus, device, and storage medium for extracting table information. The method includes: obtaining image information to be processed; extracting visible line information from the image information to obtain a visible line set; performing grouping processing on the visible line set, dividing the intersecting visible line segments in the visible line set into the same group to obtain a plurality of visible line groups; respectively performing contour analysis on each visible line group, and obtaining the table information in the image information based on the contour analysis result. The present application realizes the automatic extraction and analysis of lines in image information. By grouping visible lines and performing contour analysis, the corresponding table information is obtained. Without manual participation, the table recognition result is more accurate and can be applied to various table formats.

Description

Technical Field

[0001] The present application relates to the technical field of image processing. Specifically, it relates to a method, apparatus, device, and storage medium for extracting table information. Background Art

[0002] The graphical interface is a popular human-computer interaction method. Through the graphical interface, people can interact with machines more effectively. After information is sorted and calculated, in addition to using direct text expressions, tables are often used for display. Compared with text, tables can intuitively display the logical structure of data. Compared with images, the data in tables is more accurate and rich.

[0003] There are two common ways to identify table information in a graphical interface as follows:

[0004] Method 1: Staff observe the graphical interface with the naked eye. If a table is found, the position and structure of the table are recorded and stored in the system.

[0005] Method 2: The development of the graphical interface itself contains all underlying data. Software developers can output this table data to a user-specified location in a certain organizational manner.

[0006] Both of the above two methods have relatively large defects. The defects of Method 1 are: (1) Huge human consumption. Each table in the graphical interface needs to be manually inspected and identified. For tables with rich content, the analysis time will be very long. (2) The work intensity of workers is high. Workers need to face the computer screen for a long time, which places too much burden on the eyes and causes great damage to the eyes. The defects of Method 2 are: (1) It depends on the table data export method provided by the application developer of each graphical interface. For most graphical interface applications, no export interface is provided. (2) Even if the graphical interface provides a table interface, it still needs to be sorted into the same format for use, and the table formats provided by different applications are likely to be different. Therefore, it is still necessary to unify each interface manually. Summary of the Invention

[0007] The purpose of the embodiments of the present application is to provide a method, apparatus, device, and storage medium for extracting table information, which realizes the automatic extraction and analysis of lines in image information. Through grouping visible lines and contour analysis, the corresponding table information is obtained. No manual participation is required, and it can be applied to multiple table formats.

[0008] In the first aspect of the embodiments of the present application, a method for extracting table information is provided, including: obtaining image information to be processed; extracting visible line information in the image information to obtain a visible line set; performing grouping processing on the visible line set, dividing the intersecting visible line segments in the visible line set into the same group to obtain a plurality of visible line groups; respectively performing contour analysis on each visible line group, and obtaining table information in the image information based on the contour analysis result.

[0009] In one embodiment, the extracting visible line information in the image information to obtain a visible line set includes: performing binarization processing on the image information to obtain a binarized image, where the visible lines in the binarized image are the foreground, and the remaining information in the image information excluding the visible lines is the background; extracting visible line information in the image information according to the binarized image to obtain the visible line set.

[0010] In one embodiment, the performing binarization processing on the image information to obtain a binarized image includes: when there are both a first line with a pixel value larger than the local environment pixel value and a second line with a pixel value smaller than the local environment pixel value in the image information, performing binarization processing on the image information in a local binarization manner to obtain the binarized image.

[0011] In one embodiment, the performing binarization processing on the image information to obtain a binarized image includes: when there is a third line with the same pixel value as the local environment pixel value in the image information, performing binarization processing on the image information in a gradient threshold binarization manner to obtain the binarized image.

[0012] In one embodiment, the performing binarization processing on the image information to obtain a binarized image includes: using a fixed threshold binarization method to compare each pixel value in the image information with a predetermined binarization threshold, taking the pixels greater than the binarization threshold as the foreground, and taking the pixels less than or equal to the binarization threshold as the background.

[0013] In one embodiment, the extracting visible line information in the image information according to the binarized image to obtain the visible line set includes: performing an opening operation on the binary image using a line kernel, retaining the foreground information greater than the line kernel size in the line kernel direction to obtain the visible line set.

[0014] In one embodiment, the visible line information in the image information is extracted based on the binary image to obtain the visible line set, including: when there are broken visible line segments in the binary image, the number of pixels of the broken visible line segments at the broken points is calculated; the broken visible line segments whose number of pixels is less than or equal to a preset number are connected to form the same visible line segment, and the broken visible line segments whose number of pixels is greater than the preset number are treated as two different visible line segments, to obtain the visible line set corresponding to the image information.

[0015] In one embodiment, the contour analysis is performed on each of the visible line groups respectively, and the table information in the image information is obtained based on the contour analysis result, including: for each of the visible line groups, the inner contour enclosed by the visible line group is used as a pre-selected cell corresponding to the visible line group; for each of the visible line groups, the border line of the pre-selected cell is used as a candidate edge line, the edge line distance between two adjacent candidate edge lines is calculated in turn, and two adjacent candidate edge lines whose edge line distance is less than a preset threshold are merged into one edge line to obtain a final cell corresponding to each of the visible line groups; and the table information in the image information is determined based on multiple final cells corresponding to the multiple visible line groups.

[0016] In one embodiment, after performing contour analysis on each of the visible line groups respectively and obtaining the table information in the image information based on the contour analysis results, it also includes: eliminating cells in isolated rows or columns in the table information to obtain optimized table information in the image information.

[0017] A second aspect of an embodiment of the present application provides a table information extraction device, including: an acquisition module, used to acquire image information to be processed; an extraction module, used to extract visible line information in the image information to obtain a visible line set; a grouping module, used to group the visible line set, divide the intersecting visible line segments in the visible line set into the same group, and obtain multiple visible line groups; an analysis module, used to perform contour analysis on each of the visible line groups respectively, and obtain the table information in the image information based on the contour analysis results.

[0018] In one embodiment, the extraction module is used to: binarize the image information to obtain a binary image, wherein the visible lines in the binary image are the foreground, and the remaining information in the image information excluding the visible lines is the background; based on the binary image, extract the visible line information in the image information to obtain the visible line set.

[0019] In one embodiment, the image information is binarized to obtain a binary image, including: when the image information simultaneously contains a first line whose pixel value is larger than the local environment pixel value and a second line whose pixel value is smaller than the local environment pixel value, the image information is binarized using a local binarization method to obtain the binary image.

[0020] In one embodiment, the binarization processing of the image information to obtain the binarized image includes: when there is a third line with the same pixel value as the local environment in the image information, the image information is binarized using a gradient threshold binarization method to obtain the binarized image.

[0021] In one embodiment, the image information is binarized to obtain a binary image, including: using a fixed threshold binarization method, comparing the value of each pixel in the image information with a predetermined binarization threshold, and using pixels greater than the binarization threshold as foreground, and pixels less than or equal to the binarization threshold as background.

[0022] In one embodiment, extracting visible line information from the image information based on the binary image to obtain the visible line set includes: using a straight line kernel to perform an opening operation on the binary image, retaining foreground information that is larger than the size of the straight line kernel in the direction of the straight line kernel, and obtaining the visible line set.

[0023] In one embodiment, the visible line information in the image information is extracted based on the binary image to obtain the visible line set, including: when there are broken visible line segments in the binary image, the number of pixels of the broken visible line segments at the broken points is calculated; the broken visible line segments whose number of pixels is less than or equal to a preset number are connected to form the same visible line segment, and the broken visible line segments whose number of pixels is greater than the preset number are treated as two different visible line segments, to obtain the visible line set corresponding to the image information.

[0024] In one embodiment, the analysis module is used to: for each of the visible line groups, use the inner contour enclosed by the visible line group as a pre-selected cell corresponding to the visible line group; for each of the visible line groups, use the border line of the pre-selected cell as a candidate edge line, calculate the edge line distance between two adjacent candidate edge lines in turn, merge two adjacent candidate edge lines whose edge line distance is less than a preset threshold into one edge line, and obtain a final cell corresponding to each of the visible line groups; determine the table information in the image information based on multiple final cells corresponding to the multiple visible line groups.

[0025] In one embodiment, it further includes: a removing module, configured to, after respectively performing contour analysis on each of the visible line groups and obtaining table information in the image information based on the contour analysis results, remove the cells that are isolated in rows or columns in the table information, so as to obtain optimized table information in the image information.

[0026] A third aspect of the embodiments of the present application provides an electronic device, including: a memory for storing a computer program; a processor for executing the method according to the first aspect and any one of its embodiments of the present application.

[0027] A fourth aspect of the embodiments of the present application provides a non-transitory computer-readable storage medium for an electronic device, including: a program, which, when run by the electronic device, causes the electronic device to execute the method according to the first aspect and any one of its embodiments of the present application.

[0028] The table information extraction method, device, equipment and storage medium provided by the present application automatically analyze and extract visible lines in image information, then divide the intersecting visible line segments in the visible line set into the same group, and respectively perform contour analysis on each visible line group, so as to obtain the table information included in the image information. The entire recognition process does not require manual participation and can be applied to a variety of table formats with visible lines, improving the efficiency of table recognition. Description of the Drawings

[0029] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required to be used in the embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present application, and thus should not be regarded as limiting the scope. For those of ordinary skill in the art, other related drawings can be obtained based on these drawings without creative efforts.

[0030] Figure 1 It is a schematic structural diagram of an electronic device according to an embodiment of the present application;

[0031] Figure 2A It is a schematic flowchart of a table information extraction method according to an embodiment of the present application;

[0032] Figure 2B It is a schematic diagram of a visible line set obtained after visible line extraction is performed on a screenshot of a real graphical interface according to an embodiment of the present application;

[0033] Figure 3 It is a schematic flowchart of a table information extraction method according to an embodiment of the present application;

[0034] Figure 4 It is a schematic structural diagram of a table information extraction device according to an embodiment of the present application. Detailed Embodiments

[0035] The technical solutions in the embodiments of the present application will be described below with reference to the accompanying drawings in the embodiments of the present application. In the description of the present application, terms such as "first" and "second" are only used for descriptive distinction and cannot be understood as indicating or implying relative importance.

[0036] As Figure 1 shown, this embodiment provides an electronic device 1, including: at least one processor 11 and a memory 12. Figure 1 Taking one processor as an example. The processor 11 and the memory 12 are connected through a bus 10. The memory 12 stores instructions executable by the processor 11. When the instructions are executed by the processor 11, the electronic device 1 can execute all or part of the processes of the methods in the following embodiments to automatically extract and analyze visible lines in the image information to obtain corresponding table information, thereby improving the table recognition efficiency.

[0037] In one embodiment, the electronic device 1 can be a device such as a mobile phone, a tablet computer, a laptop computer, or a desktop computer.

[0038] Please refer to Figure 2A , which is a method for extracting table information according to an embodiment of the present application. This method can be executed by the Figure 1 electronic device 1 shown to automatically extract and analyze visible lines in the image information to obtain corresponding table information, thereby improving the table recognition efficiency. The method includes the following steps:

[0039] Step 201: Obtain the image information to be processed.

[0040] In this step, the image information to be processed can be a screenshot of a graphical interface. For example, a complete screenshot of the image interface is intercepted through the user's mobile phone as the image information to be processed, and the operation method is simple, improving the convenience of user use.

[0041] In one embodiment, the image to be processed can be a grayscale image or a color image. Grayscale images are common processing sources. If the original image is a color image, the color image can be converted into a grayscale image, which can reduce the calculation amount by about 2 / 3 and generally does not lose image information.

[0042] In the case where the table lines are similar in color to the background, the table lines may not be distinguishable from the background in the grayscale space. Therefore, this embodiment can also directly support color images for various operations. Subsequent operations such as comparison and summation between pixels involved in table line extraction and analysis are commonly used for single-channel grayscale images. For multi-channel color images, the embodiments of the present application can use the pre-weighting and post-weighting methods for processing. For example, the pre-weighting method directly weights the values of each channel of the color image to obtain a single-channel image, and then performs subsequent table extraction operations on the single-channel image.

[0043] The post-weighting method performs table extraction operations on the single-channel images of each channel of the color image separately, and then obtains the final single-channel image by weighting the extraction results of each channel. The weighting methods can include parameter weights, maximum values, and minimum values to suit different application scenarios.

[0044] Step 202: Extract the visible line information from the image information to obtain a visible line set.

[0045] In this step, the basic structure of a wired table is a visible line, so it is necessary to extract the visible lines from the image information. In the actual scenario, a visible line may have various changes in the graphical interface, such as color, width, line type, etc. During computer rendering, visible lines may also be disconnected or have color distortion. Visible lines are manifested by contrast with the background, and the color and texture of the background may also vary. Therefore, for different actual scenarios, different visible line extraction methods can be adopted to expand the applicable range of table recognition. For example, a combination of multiple continuity extraction methods and difference extraction methods can be used to extract the visible line information from the image information.

[0046] In one embodiment, the Hough transform method can also be used to map all the points in the image information to a space for visible line extraction.

[0047] In one embodiment, a convolutional neural network can also be used. By means of data training, it learns whether each pixel in the image is a line, and then performs line extraction through post-processing. The convolutional neural network has many parameters and powerful expressive ability, and depends on a large amount of data and computing power. After training, it can handle a variety of complex situations.

[0048] Step 203: Group the visible line set, and divide the intersecting visible line segments in the visible line set into the same group to obtain multiple visible line groups.

[0049] In this step, taking the screenshot of the graphical interface as the image to be processed as an example, the real graphical interface has very rich information, and the line information includes the position distribution of the lines on the image, such as Figure 2B As shown, it is a schematic diagram of the visible line set obtained after extracting the visible lines from the screenshot of the real graphical interface. The visible line segments included are very diverse, with different lengths, some scattered, and some continuously intersecting. In the actual scenario, the visible lines of the same table are always continuous. Therefore, the connected component extraction method can be used to Figure 2B analyze it, and Figure 2BWhich visible line segments in it can be connected? If two different visible line segments are connected, it often means that these two visible line segments intersect. And two intersecting visible line segments are very likely to belong to the same table or the same cell. Therefore, the intersecting and connected visible line segments can be divided into the same group, and in this way, multiple visible line groups can be obtained, where each visible line group may correspond to a table or a cell.

[0050] Step 204: Perform contour analysis on each visible line group respectively, and obtain the table information in the image information based on the contour analysis results.

[0051] In this step, a contour refers to a curve with the same color or density that connects all continuous points. Contour analysis can be performed on each group of visible lines respectively to obtain all the contours enclosed by each group of visible lines. Among these contours, there is the contour of the corresponding table. Therefore, the table information in the image information can be obtained based on the contour analysis results.

[0052] The above table information extraction method automatically analyzes and extracts the visible lines in the image information, then divides the intersecting visible line segments in the visible line set into the same group, and performs contour analysis on each visible line group respectively, so as to obtain the table information contained in the image information. The entire recognition process does not require manual participation and can be applied to various table formats with visible lines, improving the efficiency of table recognition.

[0053] Please refer to Figure 3 , which is the table information extraction method of an embodiment of this application. This method can be executed by the Figure 1 shown electronic device 1 to automatically extract and analyze the visible lines in the image information, obtain the corresponding table information, and improve the table recognition efficiency. The method includes the following steps:

[0054] Step 301: Obtain the image information to be processed. For details, refer to the description of step 201 in the above embodiment.

[0055] Step 302: Perform binarization processing on the image information to obtain a binarized image, where the visible lines in the binarized image are the foreground, and the remaining information in the image information excluding the visible lines is the background.

[0056] In this step, the image information to be processed is generally multi-valued image information. For example, it may be image information containing various pixel values (0 - 255). The binarization operation converts the input multi-valued image information into a binary image (also known as a binary image), that is, foreground / background. In this task, visible lines can be classified as the foreground, and the rest of the information as the background. The fixed-threshold binarization method can be used to compare each pixel value in the image information with a predetermined binarization threshold. Pixels greater than the binarization threshold are regarded as the foreground, and pixels less than or equal to the binarization threshold are regarded as the background.

[0057] In the actual scenario, the color relativity between the table lines and the environmental background in the image is often complex and changeable. For example, in the same image, sometimes the table line color is darker than the local environmental color in the image, sometimes it is lighter than the local environmental color in the background image, and even sometimes the table line color is the same as the environmental color in the image. In view of the above situation, a single binarization method cannot accurately process images in different situations. In the embodiments of the present application, multiple binarization methods can be pre-configured, and different binarization methods can be freely selected according to different situations, and multiple binarization methods can be freely combined. For example, some images have a single table line color, which is either darker or lighter than the image background. In this case, only one method is needed. For some images with complex and changeable table line colors, multiple binarization methods can be selected. Before binarizing the image, the image information to be processed can be classified, and different binarization methods can be selected based on different categories. For example, an image containing a single-color table line can be classified into one category, and the fixed-threshold binarization method can be selected. An image containing both table lines darker than the local environmental color and table lines lighter than the local environmental color can be classified into one category, and the local adaptive binarization method can be selected. In this way, it can be applied to various complex scenarios.

[0058] In one embodiment, step 302 may specifically further include: when there are both a first line with a pixel value greater than the local environmental pixel value and a second line with a pixel value less than the local environmental pixel value in the image information, the local binarization method is used to binarize the image information to obtain a binary image.

[0059] In this step, the fixed threshold binarization method cannot handle tables where the color relativity between the table lines and the environmental background in the image changes. For example, in the same table, sometimes the table line color is darker than the local environmental color in the image, and sometimes it is lighter than the local environmental color in the image. In such a scenario, the table lines are often distinct locally. Therefore, in this embodiment, a local adaptive threshold binarization method can be used to binarize the image in this scenario, that is, the threshold for determining whether the current pixel is a foreground is calculated based on the pixel information of the local environment in the image. The size of the local environment and the threshold calculation method can be adaptively adjusted. In this way, multiple different binarization thresholds can be obtained based on different situations in the same image, thereby achieving better binarization performance.

[0060] In one embodiment, step 302 may specifically further include: when there is a third line in the image information that has the same pixel value as the local environment pixels, the gradient threshold binarization method is used to binarize the image information to obtain a binarized image.

[0061] In some cases, the table lines may have exactly the same color as the local environment around the table lines. At this time, the conventional binarization method can no longer handle it. Therefore, this embodiment can also use the gradient threshold binarization method to handle this situation. Gradient threshold binarization can be divided into line binarization and difference binarization. First, the difference between each pixel and its adjacent pixels in each direction around it is calculated, which is the gradient. In the horizontal or vertical direction, in line binarization, the pixels with a gradient less than a certain threshold are regarded as the foreground, and these pixels are used for line extraction in the same direction in the next step. In contrast, in difference binarization, the pixels with a gradient greater than a certain threshold are regarded as the foreground, and these pixels are used for line extraction in the opposite direction in the next step. Line binarization can retain continuous lines regardless of the background. Difference binarization can retain all lines that have any difference from the background and can be used as a supplement to other binarization methods.

[0062] Step 303: According to the binarized image, extract the visible line information in the image information to obtain a visible line set.

[0063] In this step, after obtaining the binary image, visible line extraction is performed. In the binarized image, the distinction between the foreground and the background is more obvious, and the visible lines, as the foreground, are more conducive to the accuracy of line extraction. Therefore, a more accurate visible line set can be obtained based on the binarized image.

[0064] In one embodiment, step 303 may specifically include: using a line kernel to perform an opening operation on the binary image, and retaining the foreground information that is greater than the line kernel size in the direction of the line kernel to obtain a visible line set.

[0065] In this step, the set A is opened using the structural element B, that is, A is eroded using B, and then the eroded result is expanded using B. The opening operation can make the contour of the object smooth, disconnect narrow discontinuities and eliminate thin protrusions. Here, the foreground information is the visible line segment. The binary image can be used as the set A, and the straight line kernel can be used as the structural element B. The size of the straight line kernel can be set based on actual needs. For example, a rectangular structural element with a large aspect ratio can be used as the straight line kernel. For continuous visible line segments (foreground information), the set straight line kernel can be used as the structural element B to open the binary image (set A), so that only visible line segments larger than the size of the straight line kernel in the direction of the straight line kernel are retained in the binary image, and the visible line set contained in the binary image can be obtained.

[0066] In one embodiment, step 303 may further specifically include: when there are disconnected visible line segments in the binary image, the number of pixels of the disconnected visible line segments at the disconnection position is calculated. The disconnected visible line segments whose number of pixels is less than or equal to the preset number are connected to form the same visible line segment, and the disconnected visible line segments whose number of pixels is greater than the preset number are treated as two different visible line segments, to obtain a visible line set corresponding to the image information.

[0067] In actual scenes, due to binarization operations or computer screen rendering problems, table lines may be disconnected, so this embodiment can use the sum convolution method to calculate the sum of the number of foreground pixels of the disconnected visible line segment along the direction of the visible straight line at the disconnection point, and then compare the sum of the number of pixels with the preset number threshold to determine whether the disconnected visible line segment is a straight line. The preset number can be set based on actual needs. For example, the preset number is 2 pixels. Then, if the sum of the number of foreground pixels of the disconnected visible line segment along the direction of the visible straight line at the disconnection point is greater than 2, it means that the disconnected visible line segment belongs to two different visible straight line segments. If the sum of the number of foreground pixels at the disconnection point is less than or equal to 2, it means that although the disconnected visible line segment has a breakpoint, it actually belongs to the same visible straight line segment and may be disconnected in the humble operation step. Therefore, the disconnection point can be connected to repair it into a straight line segment.

[0068] Step 304: grouping the visible line set, and grouping the intersecting visible line segments in the visible line set into the same group to obtain multiple visible line groups. For details, refer to the description of step 203 in the above embodiment.

[0069] Step 305: For each visible line group, the inner contour enclosed by the visible line group is used as the pre-selected cell corresponding to the visible line group.

[0070] In this step, after obtaining the visible line group, cell parsing is performed. For each group of visible lines, contour analysis can be performed to obtain all contours (including inner contours and outer contours) surrounded by the visible line group, and the innermost inner contour is taken as the pre-selected cell.

[0071] In one embodiment, in an actual scenario, the text in the table may be connected to the table line, so in order to obtain more accurate cell information, abnormal cells can be removed from the pre-selected cells. Specifically, the length, width or area can be compared with a set threshold to remove cells that are too small.

[0072] In one embodiment, normal cells are often complete rectangles. In order to obtain more accurate pre-selected cells, cells with irregular shapes can be eliminated. For example, the minimum circumscribed rectangle of each pre-selected cell can be calculated, and the ratio of the contour area of ​​each pre-selected cell to its corresponding minimum circumscribed rectangle area can be calculated, and then the area ratio can be compared with the set ratio threshold to eliminate cells with abnormal shapes. If the pre-selected cells are relatively regular, the area ratio will be close to 1. Therefore, a ratio threshold can be selected based on actual needs to apply to different scenarios. For example, the value range of the ratio threshold can be 0.9 to 1. In this embodiment, the circumscribed rectangle refers to the rectangle with the smallest area that can completely contain the contour of the pre-selected cell.

[0073] Step 306: For each visible line group, the border line of the pre-selected cell is used as a candidate edge line, the edge line distance between two adjacent candidate edge lines is calculated in sequence, and the two adjacent candidate edge lines whose edge line distance is less than a preset threshold are merged into one edge line to obtain the final cell corresponding to each visible line group.

[0074] In this step, for each group of pre-selected cells, the four border line positions of each pre-selected cell in the group are used as candidate border lines. Since the visible line has width, and the accurate cell position does not include the line width, the two border line boundaries of two adjacent pre-selected cells are not consistent. In this embodiment, the line width threshold is set to merge similar candidate border lines. Specifically, for each group of pre-selected cells, the corresponding candidate border lines are sorted according to their actual positions on the image information, and the border line distance between the candidate border lines of two adjacent pre-selected cells is calculated in turn. The two adjacent candidate border lines whose border line distance is less than the preset threshold are merged into one border line to obtain the final cell corresponding to each visible line group. In this process, if the border line distance between the candidate border line located at the back and the current candidate border line is less than the preset threshold, the candidate border line located at the back can be discarded to achieve merging into one border line. If the border line distance is greater than or equal to the preset threshold, the candidate border line located at the back is used as an independent border line, and then each pre-selected cell is matched with the final border line to obtain the starting border line and the ending border line of the pre-selected cell. In this way, redundant border lines are removed, and a more accurate final cell can be obtained.

[0075] Step 307: Determine table information in the image information based on a plurality of final cells corresponding to a plurality of visible line groups.

[0076] In this step, the multiple final cells corresponding to the multiple visible line groups are arranged according to the corresponding relationship of their positions on the image information, thus obtaining the complete information of the wired table.

[0077] Step 308: Eliminate cells that are isolated in rows or columns in the table information to obtain optimized table information in the image information.

[0078] In this step, there are many visible lines in the graphical interface, which may be connected accidentally and form a table-like structure. The wired table may also be connected to other elements, thereby introducing a structure that does not belong to the table itself. In order to eliminate these isolated cells, the present embodiment can perform regularization and clipping on the table information obtained in step 307. Regularization means that for any cell of the table, there are always cells belonging to the same row and column as it, and such a table is a rectangle in appearance. The present embodiment can establish a list for each row and column of the table obtained in step 307, and for each cell in the table, it is placed in its corresponding row list. Finally, all row lists are traversed. If there is only one cell in a row list, it can be considered that the row and column are lonely, so it is deleted. If there are lonely cells in the traversal process, another round of regularization and clipping can be performed after the traversal is completed until there are no isolated elements, so that the optimized table information in the image information obtained is more accurate.

[0079] In one embodiment, the wired table in the original image information can also be directly detected by using object detection. For example, the commonly used FCOS algorithm is an excellent deep learning model for object detection. By using a deep learning model, powerful expressive ability can be obtained to handle various complex table situations.

[0080] The above table information extraction method depends on the developer of the application itself to provide an open interface for underlying data, and uses intelligent image recognition technology to actively recognize the wired table in the image interface. Compared with the existing full human eye recognition method, this method greatly reduces the manpower and saves the workload. Compared with the table extraction method that depends on the recognition ability of the application itself in the prior art, the method of this embodiment improves the accuracy of table recognition while greatly reducing the manpower, and can be applied to various image interface applications.

[0081] Please refer to Figure 4 , which is the table information extraction device 400 according to an embodiment of the present application. This device can be applied to Figure 1 the electronic device 1 shown in the figure to automatically extract and analyze the visible lines in the image information, obtain the corresponding table information, and improve the table recognition efficiency. The device includes: an acquisition module 401, an extraction module 402, a grouping module 403, and an analysis module 404. The principle relationships of each module are as follows:

[0082] The acquisition module 401 is used to acquire the image information to be processed.

[0083] The extraction module 402 is used to extract the visible line information in the image information to obtain a visible line set.

[0084] The grouping module 403 is used to perform grouping processing on the visible line set, divide the intersecting visible line segments in the visible line set into the same group, and obtain multiple visible line groups.

[0085] The analysis module 404 is used to perform contour analysis on each visible line group respectively, and obtain the table information in the image information based on the contour analysis result.

[0086] In one embodiment, the extraction module 402 is used to: perform binarization processing on the image information to obtain a binarized image, where the visible lines in the binarized image are the foreground, and the remaining information in the image information except for the visible lines is the background. According to the binarized image, extract the visible line information in the image information to obtain a visible line set.

[0087] In one embodiment, performing binarization processing on the image information to obtain a binarized image includes: when there are both a first line with a pixel value larger than the local environment pixel value and a second line with a pixel value smaller than the local environment pixel value in the image information, performing binarization processing on the image information by using a local binarization method to obtain a binarized image.

[0088] In one embodiment, the image information is binarized to obtain a binary image, including: when there is a third line with the same pixel value as the local environment in the image information, the image information is binarized using a gradient threshold binarization method to obtain a binary image.

[0089] In one embodiment, the image information is binarized to obtain a binary image, including: using a fixed threshold binarization method, comparing the value of each pixel in the image information with a predetermined binarization threshold, and using pixels greater than the binarization threshold as foreground, and pixels less than or equal to the binarization threshold as background.

[0090] In one embodiment, based on the binary image, visible line information in the image information is extracted to obtain a visible line set, including: using a straight line kernel to perform an opening operation on the binary image, retaining foreground information larger than the straight line kernel size in the straight line kernel direction, and obtaining a visible line set.

[0091] In one embodiment, according to the binary image, the visible line information in the image information is extracted to obtain the visible line set, including: when there are disconnected visible line segments in the binary image, the number of pixels of the disconnected visible line segments at the disconnection position is calculated. The disconnected visible line segments whose number of pixels is less than or equal to the preset number are connected to form the same visible line segment, and the disconnected visible line segments whose number of pixels is greater than the preset number are treated as two different visible line segments, to obtain the visible line set corresponding to the image information.

[0092] In one embodiment, the analysis module 404 is used to: for each visible line group, use the inner contour surrounded by the visible line group as the pre-selected cell corresponding to the visible line group. For each visible line group, use the border line of the pre-selected cell as a candidate edge line, calculate the edge line distance between two adjacent candidate edge lines in sequence, merge two adjacent candidate edge lines whose edge line distance is less than a preset threshold into one edge line, and obtain the final cell corresponding to each visible line group. Determine the table information in the image information based on the multiple final cells corresponding to the multiple visible line groups.

[0093] In one embodiment, it also includes: a removal module 405, which is used to remove cells in isolated rows or columns in the table information after performing contour analysis on each visible line group respectively and obtaining the table information in the image information based on the contour analysis results, so as to obtain optimized table information in the image information.

[0094] For a detailed description of the above table information extraction device 400, please refer to the description of the relevant method steps in the above embodiment.

[0095] An embodiment of the present invention further provides a non-transitory computer-readable storage medium for an electronic device, including: a program, when it runs on the electronic device, enabling the electronic device to execute all or part of the processes of the methods in the above embodiments. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), a flash memory, a hard disk drive (HDD), or a solid-state drive (SSD), etc. The storage medium can also include a combination of the above types of memories.

[0096] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations fall within the scope defined by the appended claims.

Claims

1. A method for extracting tabular information, characterized in that, Including: Obtain image information to be processed; Extract visible line information from the image information to obtain a set of visible lines; Perform grouping processing on the set of visible lines, and divide the intersecting visible line segments in the set of visible lines into the same group to obtain multiple groups of visible lines; Perform contour analysis on each group of visible lines respectively, and obtain table information in the image information based on the contour analysis results; Among them, the extracting visible line information from the image information to obtain a set of visible lines includes: Perform binarization processing on the image information to obtain a binarized image, where visible lines in the binarized image are foreground, and the remaining information in the image information excluding the visible lines is background; According to the binarized image, extract visible line information from the image information to obtain the set of visible lines; The performing binarization processing on the image information to obtain a binarized image includes: According to the category of the image, select a corresponding binarization processing method to perform binarization processing on the image to obtain a binarized image; where the binarization processing method includes a combination of multiple of fixed threshold binarization, local adaptive binarization, and gradient threshold binarization; Among them, the performing binarization processing on the image information to obtain a binarized image includes: Classify the first line with a pixel value larger than the local environment pixel value and the second line with a pixel value smaller than the local environment pixel value that exist simultaneously in the image information into one category, and perform binarization processing on the image information using the local binarization method to obtain the binarized image; Classify the third line with a pixel value the same as the local environment pixel value that exists in the image information into one category, and perform binarization processing on the image information using the gradient threshold binarization method to obtain the binarized image; Classify the image information that contains a single-color table line into one category, and perform binarization processing on the image information using the fixed threshold binarization method to obtain the binarized image; Among them, using the fixed threshold binarization method includes: Compare each pixel value in the image information with a predetermined binarization threshold, and pixels greater than the binarization threshold are used as foreground, and pixels less than or equal to the binarization threshold are used as background.

2. The method according to claim 1, wherein The extracting visible line information from the image information according to the binarized image to obtain the set of visible lines includes: Perform an opening operation on the binarized image using a line kernel, and retain the foreground information greater than the line kernel size in the direction of the line kernel to obtain the set of visible lines.

3. The method according to claim 1, wherein The extracting visible line information from the image information according to the binarized image to obtain the set of visible lines includes: When there are discontinuous visible line segments in the binarized image, calculate the number of pixels at the disconnection of the discontinuous visible line segments; Connect the discontinuous visible line segments with the number of pixels less than or equal to a preset number into the same visible line segment, and use the discontinuous visible line segments with the number of pixels greater than the preset number as two different visible line segments to obtain the set of visible lines corresponding to the image information.

4. The method according to claim 1, wherein The performing contour analysis on each of the visible line groups respectively and obtaining the table information in the image information based on the contour analysis result includes: For each of the visible line groups, an inner contour enclosed by the visible line group is used as a pre-selected cell corresponding to the visible line group; For each of the visible line groups, the border line of the pre-selected cell is used as a candidate edge line, the edge line distance between two adjacent candidate edge lines is calculated in sequence, and the two adjacent candidate edge lines whose edge line distance is less than a preset threshold are merged into one edge line to obtain a final cell corresponding to each of the visible line groups; Table information in the image information is determined based on a plurality of final cells corresponding to the plurality of visible line groups.

5. The method according to claim 1, wherein After performing contour analysis on each of the visible line groups and obtaining the table information in the image information based on the contour analysis result, the method further includes: Isolated cells in rows or columns in the table information are eliminated to obtain optimized table information in the image information.

6. An electronic device, characterized in that, include: Memory for storing computer programs; A processor, configured to execute the computer program to implement the method according to any one of claims 1 to 5.

Images