Table data extraction method, device, equipment and computer storage medium

Abstract

The embodiments of the present application provide a method, device, electronic device and computer storage medium for extracting table data; the method comprises: acquiring a table image; detecting table lines of the table image to obtain detection results; dividing a plurality of cells according to the detection results; dividing the plurality of cells into at least one title cell and at least one data cell according to a pre-acquired title image library; determining the title text in at least one title cell and the data text in at least one data cell; obtaining at least one group of table data according to the title text in at least one title cell and the data text in at least one data cell, wherein each group of table data in at least one group of table data comprises at least one title text and data text corresponding to the at least one title text.

Description

Technical Field

[0001] The present application relates to data collection technology for financial technology (Fintech), and is related to but not limited to a table data extraction method, device, electronic device and computer storage medium. Background Art

[0002] With the development of computer technology, more and more technologies are being applied in the financial field. Traditional finance is gradually transforming into financial technology. However, due to the security and real-time requirements of the financial industry, higher requirements are also placed on technology.

[0003] At present, image morphological transformation methods can be used to identify the table borders of table images, and then divide the cells to achieve data extraction in the table. However, in the related technology, after obtaining the cells, the contents of different types of cells are uniformly identified, resulting in the identified content being more messy, which is not conducive to determining the specific meaning of the values ​​in the table, and is not conducive to targeted processing of the table data. Summary of the invention

[0004] The embodiments of the present application provide a table data extraction method, device, electronic device and computer storage medium, which can solve the problem in the prior art that it is not conducive to targeted processing of table data.

[0005] The technical solution of the embodiment of the present application is implemented as follows:

[0006] The present application provides a method for extracting table data, the method comprising:

[0007] Get the table image;

[0008] Detecting the table lines of the table image to obtain a detection result; dividing a plurality of cells according to the detection result;

[0009] According to the pre-acquired title image library, the plurality of cells are divided into at least one title cell and at least one data cell; and the title text in the at least one title cell and the data text in the at least one data cell are determined;

[0010] At least one group of table data is obtained based on the title text in the at least one title cell and the data text in the at least one data cell, wherein each group of table data in the at least one group of table data includes at least one title text and data text corresponding to the at least one title text.

[0011] In some embodiments of the present application, dividing the plurality of cells into at least one title cell and at least one data cell according to the pre-acquired title image library includes:

[0012] Determine each non-blank cell among the plurality of cells; among the non-blank cells, use cells matching the title image in the title image library as title cells; and use cells among the non-blank cells excluding the title cells as data cells.

[0013] In some embodiments of the present application, the method further includes:

[0014] After determining the non-blank cells, determine the Hamming distance between the title image in the title image library and the hash value of any non-blank cell; determine the number of title images that meet a first set condition, wherein the first set condition is that the Hamming distance between the title image and the hash value of any non-blank cell is less than a distance threshold; and when the number of title images that meet the first set condition is greater than the number threshold, determine that the any non-blank cell matches the title image in the title image library.

[0015] In some embodiments of the present application, determining the title text in the at least one title cell includes:

[0016] Determine an image set, the image set comprising: a title image matching any one of the at least one title cell;

[0017] Determine, among the title images in the image set, a title image that satisfies a second set condition, wherein the second set condition is that a Hamming distance between the title image and a hash value of any one of the title cells is the smallest;

[0018] The title text of any one title cell is determined as: the title text corresponding to the title image that meets the second set condition.

[0019] In some embodiments of the present application, the method further includes:

[0020] Determine the positions of the plurality of cells according to the detection result;

[0021] The step of obtaining at least one set of table data according to the title text in the at least one title cell and the data text in the at least one data cell comprises:

[0022] Determine a first correspondence between the title cell and the data cell in the plurality of cells according to the positions of the plurality of cells; determine a second correspondence between the title text and the data text in the plurality of cells according to the first correspondence;

[0023] The at least one set of table data is obtained according to the second corresponding relationship.

[0024] In some embodiments of the present application, determining the first correspondence between the header cell and the data cell in the plurality of cells according to the positions of the plurality of cells includes:

[0025] According to the positions of the multiple cells, searching leftward in the same row of each data cell until a header cell is found; and / or, according to the positions of the multiple cells, searching upward in the same column of each data cell until a header cell is found;

[0026] According to the header cell found in the same row and / or the same column of each data cell, a first corresponding relationship between the found header cell and each data cell is determined.

[0027] In some embodiments of the present application, the detecting of the table lines of the table image to obtain the detection result includes:

[0028] Performing edge detection on the table image to obtain an image after edge detection;

[0029] Using a standard Hough transform method to detect the image after edge detection, to obtain a plurality of straight lines;

[0030] Determine the length of the shortest line segment in the image after edge detection and the maximum interval between adjacent parallel line segments in the image after edge detection according to the positional relationship of the plurality of straight lines;

[0031] According to the length of the shortest line segment in the image after edge detection and the maximum interval between adjacent parallel line segments in the image after edge detection, the image after edge detection is detected by using a probabilistic Hough transform method to obtain each line segment in the image after edge detection; and according to each line segment in the image after edge detection, the detection result is obtained.

[0032] In some embodiments of the present application, determining the length of the shortest line segment in the image after edge detection and the maximum interval between adjacent parallel line segments in the image after edge detection according to the positional relationship of the multiple straight lines includes:

[0033] Determine, according to the positional relationship of the plurality of straight lines, adjacent distances between respective horizontal lines of the plurality of straight lines and adjacent distances between respective vertical lines of the plurality of straight lines;

[0034] The shortest line segment length in the edge detected image and the maximum interval between adjacent parallel line segments in the edge detected image are determined according to adjacent distances between respective horizontal lines of the plurality of straight lines and adjacent distances between respective vertical lines of the plurality of straight lines.

[0035] In some embodiments of the present application, the detecting the image after edge detection using a probabilistic Hough transform method according to the shortest line segment length in the image after edge detection and the maximum interval between adjacent parallel line segments in the image after edge detection to obtain each line segment in the image after edge detection includes:

[0036] According to the length of the shortest line segment in the image after edge detection and the maximum interval between adjacent parallel line segments in the image after edge detection, the image after edge detection is detected by using a probabilistic Hough transform method to obtain the starting and ending points of each line segment in the image after edge detection;

[0037] Each line segment in the image after edge detection is determined according to the starting and ending points of each line segment in the image after edge detection and the plurality of straight lines.

[0038] The present application provides a table data extraction device, the device comprising:

[0039] An acquisition module, used for acquiring a table image;

[0040] A first processing module is used to detect the table lines of the table image to obtain a detection result; and divide a plurality of cells according to the detection result;

[0041] A second processing module is used to divide the plurality of cells into at least one title cell and at least one data cell according to a pre-acquired title image library; determine the title text in the at least one title cell and the data text in the at least one data cell;

[0042] The third processing module is used to obtain at least one group of table data based on the title text in the at least one title cell and the data text in the at least one data cell, wherein each group of table data in the at least one group of table data includes at least one title text and data text corresponding to the at least one title text.

[0043] In some embodiments of the present application, the second processing module is used to divide the multiple cells into at least one title cell and at least one data cell according to the pre-acquired title image library, including:

[0044] Determine each non-blank cell among the plurality of cells; among the non-blank cells, use cells matching the title image in the title image library as title cells; and use cells among the non-blank cells excluding the title cells as data cells.

[0045] In some embodiments of the present application, the second processing module is further used to:

[0046] After determining the non-blank cells, determine the Hamming distance between the title image in the title image library and the hash value of any non-blank cell; determine the number of title images that meet a first set condition, wherein the first set condition is that the Hamming distance between the title image and the hash value of any non-blank cell is less than a distance threshold; and when the number of title images that meet the first set condition is greater than the number threshold, determine that the any non-blank cell matches the title image in the title image library.

[0047] In some embodiments of the present application, the second processing module is used to determine the title text in the at least one title cell, including:

[0048] Determine an image set, the image set comprising: a title image matching any one of the at least one title cell;

[0049] Determine, among the title images in the image set, a title image that satisfies a second set condition, wherein the second set condition is that a Hamming distance between the title image and a hash value of any one of the title cells is the smallest;

[0050] The title text of any one title cell is determined as: the title text corresponding to the title image that meets the second set condition.

[0051] In some embodiments of the present application, the first processing module is further used to determine the positions of the plurality of cells according to the detection result;

[0052] The third processing module is used to obtain at least one set of table data according to the title text in the at least one title cell and the data text in the at least one data cell, including:

[0053] Determine a first correspondence between the title cell and the data cell in the plurality of cells according to the positions of the plurality of cells; determine a second correspondence between the title text and the data text in the plurality of cells according to the first correspondence;

[0054] The at least one set of table data is obtained according to the second corresponding relationship.

[0055] In some embodiments of the present application, the third processing module is used to determine the first corresponding relationship between the header cell and the data cell in the multiple cells according to the positions of the multiple cells, including:

[0056] According to the positions of the multiple cells, searching leftward in the same row of each data cell until a header cell is found; and / or, according to the positions of the multiple cells, searching upward in the same column of each data cell until a header cell is found;

[0057] According to the header cell found in the same row and / or the same column of each data cell, a first corresponding relationship between the found header cell and each data cell is determined.

[0058] In some embodiments of the present application, the first processing module is used to detect the table lines of the table image to obtain the detection result, including:

[0059] Performing edge detection on the table image to obtain an image after edge detection;

[0060] Using a standard Hough transform method to detect the edge-detected image, obtaining a plurality of straight lines;

[0061] Determine the length of the shortest line segment in the image after edge detection and the maximum interval between adjacent parallel line segments in the image after edge detection according to the positional relationship of the plurality of straight lines;

[0062] According to the length of the shortest line segment in the image after edge detection and the maximum interval between adjacent parallel line segments in the image after edge detection, the image after edge detection is detected by using a probabilistic Hough transform method to obtain each line segment in the image after edge detection; and according to each line segment in the image after edge detection, the detection result is obtained.

[0063] In some embodiments of the present application, the first processing module is used to determine the length of the shortest line segment in the image after edge detection and the maximum interval between adjacent parallel line segments in the image after edge detection according to the positional relationship of the plurality of straight lines, including:

[0064] Determine, according to the positional relationship of the plurality of straight lines, adjacent distances between respective horizontal lines of the plurality of straight lines and adjacent distances between respective vertical lines of the plurality of straight lines;

[0065] The shortest line segment length in the edge detected image and the maximum interval between adjacent parallel line segments in the edge detected image are determined according to adjacent distances between respective horizontal lines of the plurality of straight lines and adjacent distances between respective vertical lines of the plurality of straight lines.

[0066] In some embodiments of the present application, the first processing module is used to detect the image after edge detection using a probabilistic Hough transform method according to the shortest line segment length in the image after edge detection and the maximum interval between adjacent parallel line segments in the image after edge detection, so as to obtain each line segment in the image after edge detection, including:

[0067] According to the length of the shortest line segment in the image after edge detection and the maximum interval between adjacent parallel line segments in the image after edge detection, the image after edge detection is detected by using a probabilistic Hough transform method to obtain the starting and ending points of each line segment in the image after edge detection;

[0068] Each line segment in the image after edge detection is determined according to the starting and ending points of each line segment in the image after edge detection and the plurality of straight lines.

[0069] An embodiment of the present application provides an electronic device, the electronic device comprising:

[0070] A memory for storing executable instructions;

[0071] The processor is used to implement any one of the above table data extraction methods when executing the executable instructions stored in the memory.

[0072] An embodiment of the present application provides a computer-readable storage medium storing executable instructions for implementing any of the above-mentioned table data extraction methods when executed by a processor.

[0073] In an embodiment of the present application, a table image is acquired; table lines of the table image are detected to obtain detection results; a plurality of cells are divided according to the detection results; the plurality of cells are divided into at least one title cell and at least one data cell according to a pre-acquired title image library; a title text in the at least one title cell and a data text in the at least one data cell are determined; at least one group of table data is obtained according to the title text in the at least one title cell and the data text in the at least one data cell, wherein each group of table data in the at least one group of table data includes at least one title text and data text corresponding to the at least one title text.

[0074] It can be seen that in the embodiment of the present application, the title cells and data cells in the table image can be distinguished, and at least one group of table data can be generated based on the title text of the title cell and the data text of the data cell. Since each group of table data reflects the correspondence between the title text and the data text, the embodiment of the present application can determine the correspondence between the title text and the data text, so that each group of table data can represent the title and the data corresponding to the title, that is, the specific meaning of the numerical value of the table data can be determined. Compared with the solution of uniformly identifying the contents of different types of cells in the prior art, it is beneficial to carry out targeted processing for each group of table data on the basis of determining the specific meaning of the numerical value of the table data. BRIEF DESCRIPTION OF THE DRAWINGS

[0075] Figure 1 It is a schematic diagram of a process of extracting data of a table image in the related art;

[0076] Figure 2 is an optional flow chart of the table data extraction method provided in the embodiment of the present application;

[0077] Figure 3 It is a schematic diagram of the process of establishing a title image library corresponding to a financial statement in an embodiment of the present application;

[0078] Figure 4 It is the title image corresponding to the vocabulary net profit of the embodiment of the present application;

[0079] Figure 5 It is a schematic diagram of an optional composition structure of a table data extraction device according to an embodiment of the present application;

[0080] Figure 6 It is a schematic diagram of an optional composition structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0081] In the related art, the image morphological transformation method can be used to extract the data of the table image; Figure 1 This is a flow chart of extracting data from a table image in the related art, refer to Figure 1 , the process may include:

[0082] Step 101: Obtain a table image.

[0083] Exemplarily, the above table image may be an image of a financial statement, which is an accounting statement that reflects the capital and profit status of an enterprise or budget unit over a certain period of time.

[0084] Step 102: Perform table line detection on the table image to obtain the table lines of the table image.

[0085] Step 103: Divide cells based on table lines of the table image.

[0086] Step 104: Using an optical character recognition (OCR) method to identify the content in each cell.

[0087] Step 105: Output the content of each cell in text form.

[0088] In the related art, the scheme for extracting data from table images has the following disadvantages:

[0089] 1) After the table lines based on the table image divide the cells, the cell types are not distinguished, that is, the header cells and data cells are not distinguished, and the relationship between the header cells and the data cells is not maintained, resulting in a messy output of the recognition content, and the specific meaning of the numerical value in the cell cannot be obtained, which is not conducive to the storage of the data in the table and the targeted processing of the data in the table.

[0090] 2) The solutions for extracting data from table images in related technologies cannot identify the mathematical properties of table lines, for example, the properties of the starting point, end point, slope, etc. When the identified table lines are incomplete or interrupted, the intersection of the horizontal and vertical lines may be lost, resulting in inaccurate structure of the segmented cells, which in turn affects the data extraction from the table image.

[0091] In order to solve the above technical problems, a technical solution of the embodiment of the present application is proposed.

[0092] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of this application.

[0093] The embodiment of the present application provides a table data extraction method, device, electronic device and computer storage medium. The table data extraction method of the embodiment of the present application can be applied to a computer system composed of a terminal and / or a server, and can be operated with many other general or special computing system environments or configurations. Here, the terminal can be a thin client, a thick client, a handheld or laptop device, a microprocessor-based system, a set-top box, a programmable consumer electronic product, a network personal computer, a small computer system, etc., and the server can be a server computer system, a small computer system, a large computer system, and a distributed cloud computing technology environment including any of the above systems, etc.

[0094] Electronic devices such as terminals and servers can be described in the general context of computer system executable instructions (such as program modules) executed by a computer system. In general, program modules can include routines, programs, object programs, components, logic, data structures, etc., which perform specific tasks or implement specific abstract data types. Computer systems / servers can be implemented in a distributed cloud computing environment, where tasks are performed by remote processing devices linked through a communication network. In a distributed cloud computing environment, program modules can be located on local or remote computing system storage media including storage devices.

[0095] The following is an exemplary description of the table data extraction method in the embodiment of the present application.

[0096] Figure 2 An optional flow chart of the table data extraction method provided in the embodiment of the present application, such as Figure 2 As shown, the process may include:

[0097] Step 201: Acquire a table image.

[0098] In some embodiments, a table image of a specific type of table may be acquired. The specific type of table may be a financial statement or other types of tables. For example, the financial statement may be a balance sheet, an income statement, or a cash flow statement.

[0099] In some embodiments, the table image may be located in a local storage space or a network, which is not limited in this embodiment of the present application.

[0100] In some embodiments, the initial table image of a specific type of table may be captured by a screenshot operation or other methods, and then the initial table image of the specific type of table may be preprocessed to obtain a table image of the specific type of table.

[0101] In some embodiments, the initial table image can be denoted as I0. After image preprocessing of the initial table image I0, a table image I1 of a specific type of table can be obtained. The image preprocessing process for the initial table image I0 can include an image correction step and / or a grayscale conversion step.

[0102] It can be understood that the initial table image I0 may be skewed, and the skewness of the image will affect the accuracy of image recognition. For the skewed initial table image I0, the initial table image I0 can be corrected through an image correction step to obtain a corrected image, and the corrected image is an image without skewness; in some embodiments, the projection method can be used to correct the initial table image I0.

[0103] In some embodiments, the initial table image I0 is a color image including three channels of red, green and blue (RGB). The initial table image I0 contains a large amount of information and has a high data dimension. The initial table image I0 can be converted into a grayscale image by using an averaging method, that is, the RGB values ​​of the three channels of the same pixel in the initial table image I0 are averaged to obtain a new pixel value, which represents the brightness characteristics of the pixel.

[0104] Here, the process of converting the initial table image I0 into a grayscale image using the averaging method can be explained by formula (1).

[0105]

[0106] Where x and y represent the horizontal and vertical coordinates of the pixel point in the initial table image I0, respectively, I(x, y) represents the new pixel value of the pixel point (x, y), and I R(x,y) Represents the value of the R channel of the pixel point (x, y) in the initial table image I0, I G(x,y) Represents the value of the G channel of the pixel point (x, y) in the initial table image I0, I B(x,y) Represents the value of the B channel of the pixel (x, y) in the initial table image I0.

[0107] In some embodiments, after an initial table image of a specific type of table or a table image of a specific type of table is acquired, the initial table image or the table image may be stored in a local storage space.

[0108] Step 202: Detect table lines of the table image to obtain detection results; and divide multiple cells according to the detection results.

[0109] In the embodiment of the present application, the above detection result is the table lines of the table image; in some embodiments, the edge image detection method and the Hough transform method can be used to detect the table lines of the table image to obtain the detection result.

[0110] Here, the edge image detection method is used to detect points in digital images where the brightness changes significantly; the edge detection method can use the mutation property of the edge to detect points where the brightness changes significantly. Its basic idea is: by detecting the state of each pixel and its neighborhood in the image, it is determined whether the pixel is located on the boundary of an object. A pixel located on the boundary of an object has a relatively large change in grayscale value compared with the neighboring pixels. For example, the edges in the financial report image of an enterprise include text edges and table line edges.

[0111] In one implementation, the Canny edge detection operator may be used to perform edge detection on the table image I1 of the specific type of table to obtain a binary edge image I2.

[0112] The Hough transform method is a feature detection method that is widely used in image analysis, computer vision, and digital image processing. The Hough transform is often used to identify features in objects, such as lines.

[0113] In some embodiments, the Hough transform method can be used to detect straight lines. The principle of the Hough transform method for detecting straight lines is: mapping the pixel coordinates in the image to the Hough (parameter) space. Assuming that the coordinates of a pixel point in the image space are (x, y), the polar coordinates of any straight line in the image space are expressed as formula (2):

[0114] ρ=xcosθ+ysinθ (2)

[0115] Among them, ρ represents the polar radius of the pixel point (x, y), and θ represents the polar angle of the pixel point (x, y).

[0116] Any pixel point (x', y') in the image can be mapped to a curve represented by formula (3) in the Hough space.

[0117] ρ=x'cosθ+y'sinθ (3)

[0118] Based on the above records, p pixels in the image can be transformed to obtain p curves in the Hough space, where p is an integer greater than or equal to 2. When the p pixels are collinear, the p curves in the Hough space are compared to the same point. According to the coordinates of the intersection point and the above formula (2), the corresponding straight line can be restored, thus realizing straight line detection.

[0119] In an embodiment of the present application, after obtaining the detection results, the position of the lines of each cell can be determined according to the position of the table lines in the detection results, thereby dividing the cells; it can be understood that after dividing multiple cells, the starting point and end point of each line of each cell can be determined.

[0120] Step 203: according to the pre-acquired title image library, multiple cells are divided into at least one title cell and at least one data cell; and title text in the at least one title cell and data text in the at least one data cell are determined.

[0121] In some embodiments of the present application, the title image library may include at least one title image; exemplarily, if the acquired table image is a table image of a specific type of table, the title image library may include at least one title image in the specific type of table.

[0122] In the embodiment of the present application, for different tables of the same type, the title content in the table is highly similar. Based on this feature, the embodiment of the present application can pre-establish a title image library; illustratively, the cells of the company's financial statements can be divided into title cells and data cells, the title cell identifies an indicator, and the data cell is the value of the title cell indicator. There are certain specifications for the preparation of a company's financial statements. For the company's financial statements, the data domain of each company's financial statements is different (that is, the content of the data cell is different), but the title domain is very similar (that is, the content of the title cell is highly similar); for example, for the indicator in the balance sheet: accounts receivable, the balance sheets of different companies will use the title of accounts receivable.

[0123] In some embodiments of the present application, a title image library can be prepared in advance for the title text of a specific type of table; when the specific type of table is a financial report, refer to Figure 3 , the process of establishing the title image library corresponding to the financial statements may include:

[0124] Step A1: Collect standard vocabulary.

[0125] In some embodiments of the present application, standard vocabulary for the company's financial statements can be collected based on the data compilation specifications of the company's financial statements; for example, the standard vocabulary for the balance sheet includes: cash and cash equivalents, development expenses, accounts payable, prepaid expenses, etc.; the standard vocabulary for the income statement includes: administrative expenses, financial expenses, non-operating income, non-operating expenses, etc.; the standard vocabulary for the cash flow statement includes: net profit, depreciation of fixed assets, amortization of intangible assets, investment losses, etc.

[0126] Step A2: Standard vocabulary classification.

[0127] One implementation method of this step may be: classify the collected standard vocabulary according to the table type, and add tags to the standard vocabulary, the tags indicating the table type to which the standard vocabulary belongs; for example, if the table to which monetary funds belong is the balance sheet, then the tag of monetary funds is the balance sheet; if the table to which intangible asset amortization belongs is the cash flow statement, then the tag of intangible asset amortization is the cash flow statement.

[0128] Step A3: Title image creation.

[0129] In some embodiments of the present application, corresponding title images can be produced for the collected standard words, referring to Figure 4 , showing the title image corresponding to the word net profit.

[0130] It can be understood that when the above standard vocabulary is the title text in the title cell of the table, based on the above steps A1 to A3, a correspondence between the title text and the title image can be established.

[0131] In some embodiments of the present application, reference Figure 3 , after executing step A3, step A4 may be executed.

[0132] Step A4: Generate a hash value of the title image.

[0133] In some embodiments of the present application, a perceptual hashing algorithm may be used to generate a 64-bit hash value for each produced title image; the perceptual hashing algorithm may generate a hash value for each digital image to facilitate calculation of similarities between different images.

[0134] After executing step A4, each title image may include the following three dimensions of information: hash value, title text content, and table type to which the title text corresponding to the title image belongs; after determining information on any two dimensions of the above three dimensions, information on another dimension may be obtained.

[0135] In some embodiments of the present application, the similarity between different images can be determined based on the Hamming distance between the hash values ​​of different images; therefore, after the hash value of the title image is generated, it is convenient to subsequently determine the similarity between the title image and other images based on the hash value of the title image.

[0136] In the embodiment of the present application, the title image library can be composed of the produced title images, thereby realizing the production of the title image library.

[0137] In the embodiment of the present application, when the above-mentioned specific type of form is not a financial statement, the process of steps A1 to A4 can also be referred to to create a title image library, which will not be repeated here.

[0138] In some embodiments of the present application, the above-mentioned dividing the multiple cells into at least one title cell and at least one data cell according to the pre-acquired title image library may include:

[0139] Determine each non-blank cell among the plurality of cells; among the non-blank cells, use the cell that matches the title image in the title image library as the title cell; and use the cells excluding the title cell among the non-blank cells as the data cell.

[0140] In one implementation, after dividing multiple cells, the content area of ​​each cell can be extracted using a projection method; horizontal projection and vertical projection can be used to extract the area with peak mutation in the projection image. If there is no peak mutation area in the projection image corresponding to the cell, the cell is determined to be a blank cell; conversely, if there is a peak mutation area in the projection image corresponding to the cell, the cell is determined to be a non-blank cell. In the embodiment of the present application, blank cells can be marked, and no processing is required for the blank cells.

[0141] After determining the non-blank cells among the multiple cells, it can be determined whether each non-blank cell matches the title image in the title image library. If the non-blank cell matches the title image in the title image library, the non-blank cell is a title cell; if the non-blank cell does not match the title image in the title image library, the non-blank cell is a data cell.

[0142] It can be seen that the embodiment of the present application can accurately identify the types of divided cells and distinguish between header cells and data cells, which is conducive to determining the exact meaning of the contents in header cells and data cells, and further facilitates targeted processing of data of different types of cells.

[0143] Step 204: Obtain at least one group of table data based on the title text in at least one title cell and the data text in at least one data cell, wherein each group of table data in the at least one group of table data includes at least one title text and data text corresponding to the at least one title text.

[0144] In the embodiment of the present application, the positions of multiple cells can also be determined based on the detection results.

[0145] Accordingly, one implementation method of this step may be: determining a first correspondence between title cells and data cells in a plurality of cells based on the positions of a plurality of cells; determining a second correspondence between title texts and data texts in a plurality of cells based on the first correspondence; and obtaining the at least one set of table data based on the second correspondence.

[0146] In some embodiments of the present application, each set of table data may be stored in the form of a key-value pair, in which the key represents a title text in a set of table data, and the value represents a data text in a set of table data.

[0147] In some embodiments of the present application, each set of table data may also be stored in the form of a relational database table.

[0148] In practical applications, steps 201 to 204 can be implemented based on a processor of an electronic device, and the processor can be at least one of an Application Specific Integrated Circuit (ASIC), a Digital Signal Processor (DSP), a Digital Signal Processing Device (DSPD), a Programmable Logic Device (PLD), a Field Programmable Gate Array (FPGA), a CPU, a controller, a microcontroller, and a microprocessor. It can be understood that the electronic device that implements the above-mentioned processor function can also be other, and the embodiments of the present application are not limited thereto.

[0149] It can be seen that in the embodiment of the present application, the title cells and data cells in the table image can be distinguished, and at least one group of table data can be generated based on the title text of the title cell and the data text of the data cell. Since each group of table data reflects the correspondence between the title text and the data text, the embodiment of the present application can determine the correspondence between the title text and the data text, so that each group of table data can represent the title and the data corresponding to the title, that is, the specific meaning of the numerical value of the table data can be determined. Compared with the solution of uniformly identifying the contents of different types of cells in the prior art, it is beneficial to carry out targeted processing for each group of table data on the basis of determining the specific meaning of the numerical value of the table data.

[0150] In some embodiments of the present application, the above-mentioned detection of the table lines of the table image to obtain the detection result may be implemented in the following manner:

[0151] Perform edge detection on the table image to obtain an image after edge detection; use a standard Hough transform method to detect the image after edge detection to obtain multiple straight lines; determine the length of the shortest line segment in the image after edge detection and the maximum interval between adjacent parallel line segments in the image after edge detection according to the positional relationship of the multiple straight lines; based on the length of the shortest line segment in the image after edge detection and the maximum interval between adjacent parallel line segments in the image after edge detection, use a probabilistic Hough transform method to detect the image after edge detection to obtain each line segment in the image after edge detection; obtain the detection result according to each line segment in the image after edge detection.

[0152] It can be understood that the standard Hough transform method can be used to detect straight lines in an image. The lines detected by the standard Hough transform method will be automatically extended, resulting in a more obvious difference between the constructed table lines and the real table lines in the image, that is, the constructed table lines do not conform to the actual overdue. For example, it may cause non-intersecting table lines in the real table to intersect after extension.

[0153] In order to solve the problem that the constructed table lines do not conform to the actual overdue period, the embodiment of the present application proposes a solution of using the standard Hough transform method and the probabilistic Hough transform method to detect the table lines in sequence.

[0154] The probabilistic Hough transform method is an improved method of the standard Hough transform method. The probabilistic Hough transform method can extract line segments in the image and determine the starting and ending points of the line segments. However, in order to use the probabilistic Hough transform method, the following two additional parameters need to be obtained: the shortest line segment length (minLineLength) and the maximum allowed interval between line segments (maxLineGap). Here, in the table image, the maximum allowed interval between line segments is the maximum interval between the adjacent parallel line segments mentioned above.

[0155] It can be seen that compared with the prior art solution of only performing one standard Hough transform, the embodiment of the present application can perform one standard Hough transform, and based on the length of the shortest line segment in the image and the maximum interval between adjacent parallel line segments in the image after edge detection, perform image detection through probabilistic Hough transform, thereby determining each line segment in the image after edge detection and the starting and ending points of each line segment in the image after edge detection, that is, it can accurately and completely identify the table lines in the image, which is conducive to accurate cell division according to the table lines.

[0156] In some embodiments of the present application, the method for determining the length of the shortest line segment in the image after edge detection and the maximum interval between adjacent parallel line segments in the image after edge detection according to the positional relationship of multiple straight lines may be:

[0157] According to the positional relationship of the plurality of straight lines, the adjacent distances between the horizontal lines of the plurality of straight lines and the adjacent distances between the vertical lines of the plurality of straight lines are determined; according to the adjacent distances between the horizontal lines of the plurality of straight lines and the adjacent distances between the vertical lines of the plurality of straight lines, the length of the shortest line segment in the image after edge detection and the maximum interval between adjacent parallel line segments in the image after edge detection are determined.

[0158] It can be seen that the embodiment of the present application can use the detection results of the standard Hough transform to determine the length of the shortest line segment in the image after edge detection and the maximum interval between adjacent parallel line segments, which is conducive to using the probability transformation method to detect each line segment in the image and the starting and ending points of each line segment in the image after edge detection.

[0159] In some embodiments, the image after edge detection can be detected based on the length of the shortest line segment in the image after edge detection and the maximum interval between adjacent parallel line segments in the image after edge detection, and the probabilistic Hough transform method can be used to obtain the starting and ending points of each line segment in the image after edge detection; and each line segment in the image after edge detection is determined based on the starting and ending points of each line segment in the image after edge detection and the above-mentioned multiple straight lines.

[0160] It can be understood that according to the above records, in the related art, if only the standard Hough transform method is used to detect straight lines in an image, the detected lines will automatically extend, which may cause non-intersecting table lines in a real table to intersect after extension; while in the embodiment of the present application, the starting and ending points of each line segment in the image after edge detection can be determined based on the length of the shortest line segment in the image after edge detection and the maximum interval between adjacent parallel line segments, and then the line segments in the image after edge detection can be determined. By determining each line segment in the image after edge detection, the problem that the detected lines in the related art will automatically extend can be solved.

[0161] In some embodiments of the present application, in the above-mentioned binary edge image I2, the upper left corner of the image is defined as the origin of the coordinate axis, the downward direction is the positive direction of the y-axis, and the rightward direction is the positive direction of the x-axis. The binary edge image I2 is detected using the standard Hough transform method to obtain multiple straight lines; the set of horizontal lines in the multiple straight lines is (V1,…,V n ), where V1 to V n They represent the 1st to nth horizontal lines in the multiple straight lines arranged in the vertical order, and n is an integer greater than 1. The set of vertical lines in the multiple straight lines is (H1,…,H m ), where H1 to H m They respectively represent the 1st vertical line to the mth vertical line arranged in left-right order among the multiple straight lines, where m is an integer greater than 1.

[0162] After determining the set of horizontal lines in multiple straight lines, the vertical distance between any two adjacent horizontal lines can be calculated to obtain the set of vertical distances between adjacent horizontal lines h, h = (h1, .., h n-1 ), where a is 1 to n-1, h a Represents the vertical distance from the a-th horizontal line to the a+1-th horizontal line.

[0163] After determining the set of vertical lines among multiple straight lines, the horizontal distance between any two adjacent vertical lines can be calculated to obtain the set of horizontal distances between adjacent vertical lines, w = (w1, .., w m-1 ), where b is 1 to m-1, w b Represents the horizontal distance from the b-th vertical line to the b+1-th vertical line.

[0164] Determine the maximum value h of each element in the set h max and the minimum value h min , determine the maximum value w of each element in the set w max and the minimum value w min ; According to the structure of the table, we can determine: the minimum value h of each element in the set h min is the minimum height of the cell, the minimum value w of each element in the set w min is the minimum length of the cell, h min and w min The smaller value in is the above-mentioned shortest line segment length. Image detection based on the shortest line segment length and the probabilistic Hough transform method is beneficial to detecting the lines of all cells in the image.

[0165] According to the structure of the table, we can determine: the maximum value h of each element in the set h max is the maximum height of the cell, the maximum value w of each element in the set w max is the maximum length of the cell, h min and w min The larger value in is the maximum interval between the adjacent parallel line segments mentioned above; image detection based on the maximum interval between adjacent parallel line segments and the probabilistic Hough transform method is conducive to keeping the lines of different cells in the image independent and preventing the lines from extending and intersecting.

[0166] After obtaining the shortest line segment length and the maximum interval between adjacent parallel line segments in the binary edge image I2, the probabilistic Hough transform method can be used to detect the binary edge image I2 to obtain all line segments and the starting and ending points of all line segments in the table image, that is, to obtain the detection result; for example, in the above detection result, the i-th horizontal line segment can be recorded as (h i ,s i ,e i ), where h i represents the intercept of the i-th horizontal line segment on the y-axis, s i represents the coordinate of the starting point of the i-th horizontal line segment on the x-axis, e i represents the coordinate of the end point of the i-th horizontal line segment on the x-axis; in the above detection results, the j-th vertical line segment can be recorded as (w j ,t j ,d j ), where w j represents the intercept of the jth vertical line segment on the x-axis, t j represents the coordinate of the starting point of the j-th vertical line segment on the y-axis, d j Represents the y-coordinate of the end point of the j-th vertical line segment.

[0167] In some embodiments of the present application, after obtaining the detection results, each table line in the detection results may be marked on the initial table image I0 or the table image I1, thereby obtaining an image I3.

[0168] In some embodiments of the present application, after obtaining the image I3, a copy image of the image I3 may be generated, and then the copy image may be segmented according to the table lines to obtain an image of each cell in the table; when segmenting the copy image, it may first be segmented according to the rows to obtain a row set (L1, ..., L n ), where L1 to L n Respectively represent the images from the 1st row to the nth row. Then, cells can be segmented for each row of images, and the cell set (C k1 ,…,C km ). For cell C kj The coordinates of the four vertices of the upper left corner, upper right corner, lower right corner, and lower left corner are (w j ,h k )、(w j+1 ,h k )、(w j+1 ,h k+1 ) and (w j ,h k+1 ). Based on the above recorded content, the horizontal coordinate and vertical coordinate of the vertex of each cell can be determined.

[0169] It can be seen that the embodiment of the present application can use edge detection combined with two Hough transform detection methods to accurately identify the table lines in the table image, and can determine the mathematical properties of the table lines, such as the starting point, end point and slope of the line segments in the table lines. That is, compared with the detection scheme of performing a standard Hough transform in the related technology, the embodiment of the present application can better restore the table structure and obtain the specific position of each cell in the table image, that is, the exact position of each cell can be located.

[0170] In some embodiments of the present application, the implementation method of determining whether a non-blank cell matches a title image in a title image library may be:

[0171] After determining each non-blank cell, determine the Hamming distance between the title image in the title image library and the hash value of any non-blank cell; determine the number of title images that meet the first set condition, and the first set condition is: the Hamming distance between the title image and the hash value of any of the non-blank cells is less than a distance threshold; when the number of title images that meet the first set condition is greater than the number threshold, determine that any of the non-blank cells matches the title image in the title image library; when the number of title images that meet the first set condition is greater than the number threshold, determine that any of the non-blank cells does not match the title image in the title image library.

[0172] In an embodiment of the present application, the distance threshold and the quantity threshold can be pre-set according to actual conditions. For example, the distance threshold can be empirically set to 8, 9, 10, 11 or 12; the quantity threshold can be the product of the number of title images in the title image library and a set ratio. For example, the set ratio can be 80%, 85%, 90%, etc.

[0173] In actual implementation, the hash value of the image of the non-blank cell can be determined first. As an implementation method, the above-mentioned perceptual hash algorithm can be used to calculate the hash value of the image of the non-blank cell. After the hash value of the image of the non-blank cell is obtained, the Hamming distance between the hash value of the image of the non-blank cell and the hash value of each title image in the title image library is calculated to determine the number of title images that meet the first set condition.

[0174] It can be seen that the embodiment of the present application can easily determine whether a non-blank cell matches a title image in a title image library based on the Hamming distance between the hash values ​​of the non-blank cell and the title image.

[0175] In some embodiments of the present application, the implementation method of determining the title text in the at least one title cell may be:

[0176] Determine an image set, the image set comprising: a title image matching any one of the at least one title cell;

[0177] The title text of any one of the above title cells is determined in the title text corresponding to the title image of the image set; the title text corresponding to the title image of the image set is predetermined.

[0178] It can be seen that the embodiment of the present application can determine the title text of any title cell based on the title image matching any title cell and the title text corresponding to the title image, which is relatively easy to implement.

[0179] In some embodiments of the present application, in the title text corresponding to the title image of the image set, the implementation method of determining the title text of any one of the above title cells may be:

[0180] Among the title images in the image set, a title image that satisfies a second set condition is determined. The second set condition is that the Hamming distance between the title image and a hash value of any title cell is the smallest.

[0181] The title text of any title cell is determined to be: the title text corresponding to the title image that meets the second set condition.

[0182] In actual implementation, the title image with the smallest Hamming distance to the hash value of any of the above title cells can be selected from the title images matching the above title cells, and the title text of any of the above title cells can be determined based on the title text corresponding to the selected title image.

[0183] It can be seen that the embodiment of the present application can select the title image most similar to the title cell according to the Hamming distance between the hash value of the title image and the title cell, and thus can more accurately obtain the title text of the title cell.

[0184] In some other embodiments, an OCR method may also be used to recognize the image in the title cell to obtain the title text of the title cell.

[0185] In some embodiments of the present application, an OCR method may be used to recognize images in data cells to obtain data text in the data cells.

[0186] In some embodiments of the present application, the implementation method of determining the above-mentioned first corresponding relationship may be:

[0187] According to the positions of the above-mentioned multiple cells, searching leftward in the same row of each data cell until a header cell is found; and / or, according to the positions of the above-mentioned multiple cells, searching upward in the same column of each data cell until a header cell is found;

[0188] According to the header cell found in the same row and / or the same column of each data cell, a first corresponding relationship between the found header cell and each data cell is determined.

[0189] The following is an illustrative description of how to determine the first corresponding relationship with reference to the table structure diagrams shown in Table 1 and Table 2.

[0190] Table 1

[0191] title data title data title data title data title data title data title data title data title data title data

[0192] Table 1 shows a table structure with only a left header. For the table structure shown in Table 1, each data cell is taken as a base cell, and an adjacent header cell is found in the same row to the left. The adjacent header cell found corresponds to the base cell.

[0193] Table 2

[0194] title title title title title data data title data data title data data title data data title data data title data data title data data title data data title data data title data data

[0195] Table 2 shows a table structure with both left and top headers. For the table structure shown in Table 2, first, each data cell is used as a reference cell, and the adjacent header cell is found in the same row to the left. The adjacent header cell found corresponds to the reference cell. Then, each data cell is used as a reference cell, and the adjacent header cell is found in the same column as above. The adjacent header cell found corresponds to the reference cell. The adjacent header cells found in the row direction and the column direction can jointly determine the meaning of the reference cell.

[0196] It should be noted that the table structures shown in Table 1 and Table 2 are merely two exemplary table structures of the embodiments of the present application, and the embodiments of the present application are not limited thereto.

[0197] In summary, the embodiments of the present application can distinguish the divided cells according to the title image of the title image library, and can easily determine the correspondence between the title cells and the data cells according to the positional relationship between the title cells and the data cells, which is beneficial for each set of output table data to have a clear meaning and facilitates targeted processing of each set of table data.

[0198] Based on the table data extraction method proposed in the above embodiment, the embodiment of the present application further proposes a table data extraction device; Figure 5 FIG. 1 is a schematic diagram of an optional structure of a table data extraction device according to an embodiment of the present application. Figure 5 As shown, the table data extraction device 500 may include:

[0199] An acquisition module 501 is used to acquire a table image;

[0200] The first processing module 502 is used to detect the table lines of the table image to obtain a detection result; and divide a plurality of cells according to the detection result;

[0201] The second processing module 503 is used to divide the multiple cells into at least one title cell and at least one data cell according to the pre-acquired title image library; determine the title text in the at least one title cell and the data text in the at least one data cell;

[0202] The third processing module 504 is used to obtain at least one group of table data based on the title text in the at least one title cell and the data text in the at least one data cell, wherein each group of table data in the at least one group of table data includes at least one title text and data text corresponding to the at least one title text.

[0203] In some embodiments of the present application, the second processing module 503 is used to divide the multiple cells into at least one title cell and at least one data cell according to the pre-acquired title image library, including:

[0204] Determine each non-blank cell among the plurality of cells; among the non-blank cells, use cells matching the title image in the title image library as title cells; and use cells among the non-blank cells excluding the title cells as data cells.

[0205] In some embodiments of the present application, the second processing module 503 is further used to:

[0206] After determining the non-blank cells, determine the Hamming distance between the title image in the title image library and the hash value of any non-blank cell; determine the number of title images that meet a first set condition, wherein the first set condition is that the Hamming distance between the title image and the hash value of any non-blank cell is less than a distance threshold; and when the number of title images that meet the first set condition is greater than the number threshold, determine that the any non-blank cell matches the title image in the title image library.

[0207] In some embodiments of the present application, the second processing module 503 is used to determine the title text in the at least one title cell, including:

[0208] Determine an image set, the image set comprising: a title image matching any one of the at least one title cell;

[0209] Determine, among the title images in the image set, a title image that satisfies a second set condition, wherein the second set condition is that a Hamming distance between the title image and a hash value of any one of the title cells is the smallest;

[0210] The title text of any one title cell is determined as: the title text corresponding to the title image that meets the second set condition.

[0211] In some embodiments of the present application, the first processing module 502 is further configured to determine the positions of the plurality of cells according to the detection result;

[0212] The third processing module is used to obtain at least one set of table data according to the title text in the at least one title cell and the data text in the at least one data cell, including:

[0213] Determine a first correspondence between the title cell and the data cell in the plurality of cells according to the positions of the plurality of cells; determine a second correspondence between the title text and the data text in the plurality of cells according to the first correspondence;

[0214] The at least one set of table data is obtained according to the second corresponding relationship.

[0215] In some embodiments of the present application, the third processing module 504 is used to determine the first corresponding relationship between the header cell and the data cell in the multiple cells according to the positions of the multiple cells, including:

[0216] According to the positions of the multiple cells, searching leftward in the same row of each data cell until a header cell is found; and / or, according to the positions of the multiple cells, searching upward in the same column of each data cell until a header cell is found;

[0217] According to the header cell found in the same row and / or the same column of each data cell, a first corresponding relationship between the found header cell and each data cell is determined.

[0218] In some embodiments of the present application, the first processing module 502 is used to detect the table lines of the table image to obtain the detection result, including:

[0219] Performing edge detection on the table image to obtain an image after edge detection;

[0220] Using a standard Hough transform method to detect the edge-detected image, obtaining a plurality of straight lines;

[0221] Determine the length of the shortest line segment in the image after edge detection and the maximum interval between adjacent parallel line segments in the image after edge detection according to the positional relationship of the plurality of straight lines;

[0222] According to the length of the shortest line segment in the image after edge detection and the maximum interval between adjacent parallel line segments in the image after edge detection, the image after edge detection is detected by using a probabilistic Hough transform method to obtain each line segment in the image after edge detection; and according to each line segment in the image after edge detection, the detection result is obtained.

[0223] In some embodiments of the present application, the first processing module 502 is used to determine the length of the shortest line segment in the image after edge detection and the maximum interval between adjacent parallel line segments in the image after edge detection according to the positional relationship of the plurality of straight lines, including:

[0224] Determine, according to the positional relationship of the plurality of straight lines, adjacent distances between respective horizontal lines of the plurality of straight lines and adjacent distances between respective vertical lines of the plurality of straight lines;

[0225] The shortest line segment length in the edge detected image and the maximum interval between adjacent parallel line segments in the edge detected image are determined according to adjacent distances between respective horizontal lines of the plurality of straight lines and adjacent distances between respective vertical lines of the plurality of straight lines.

[0226] In some embodiments of the present application, the first processing module 502 is used to detect the image after edge detection using a probabilistic Hough transform method according to the length of the shortest line segment in the image after edge detection and the maximum interval between adjacent parallel line segments in the image after edge detection, to obtain each line segment in the image after edge detection, including:

[0227] According to the length of the shortest line segment in the image after edge detection and the maximum interval between adjacent parallel line segments in the image after edge detection, the image after edge detection is detected by using a probabilistic Hough transform method to obtain the starting and ending points of each line segment in the image after edge detection;

[0228] Each line segment in the image after edge detection is determined according to the starting and ending points of each line segment in the image after edge detection and the plurality of straight lines.

[0229] In practical applications, the acquisition module 501, the first processing module 502, the second processing module 503 and the third processing module 504 can all be implemented using a processor of an electronic device, and the processor can be at least one of an ASIC, a DSP, a DSPD, a PLD, an FPGA, a CPU, a controller, a microcontroller, and a microprocessor. It can be understood that the electronic device that implements the functions of the processor can also be other, and the embodiments of the present application are not limited thereto.

[0230] It should be noted that the description of the above device embodiment is similar to the description of the above method embodiment, and has similar beneficial effects as the method embodiment. For technical details not disclosed in the device embodiment of the present application, please refer to the description of the method embodiment of the present application for understanding.

[0231] It should be noted that in the embodiment of the present application, if the above-mentioned table data extraction method is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium, including a number of instructions to enable a computer device (which can be a terminal, a server, etc.) to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a U disk, a mobile hard disk, a read-only memory (ROM), a disk or an optical disk. In this way, the embodiment of the present application is not limited to any specific combination of hardware and software.

[0232] Correspondingly, an embodiment of the present application further provides a computer program product, which includes computer executable instructions, and the computer executable instructions are used to implement any one of the table data extraction methods provided in the embodiments of the present application.

[0233] Correspondingly, an embodiment of the present application further provides a computer storage medium, on which computer executable instructions are stored, and the computer executable instructions are used to implement any one of the table data extraction methods provided in the above embodiments.

[0234] The present application also provides an electronic device, Figure 6 A schematic diagram of an optional structure of an electronic device provided in an embodiment of the present application, such as Figure 6 As shown, the electronic device 600 includes:

[0235] Memory 601, used to store executable instructions;

[0236] The processor 602 is configured to implement any one of the above table data extraction methods when executing the executable instructions stored in the memory 601.

[0237] The processor 602 may be at least one of an ASIC, a DSP, a DSPD, a PLD, a FPGA, a CPU, a controller, a microcontroller, and a microprocessor.

[0238] The above-mentioned computer-readable storage medium / memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory (Flash Memory), a magnetic surface memory, an optical disk, or a compact disc read-only memory (CD-ROM) and the like; it can also be various terminals including one or any combination of the above-mentioned memories, such as mobile phones, computers, tablet devices, personal digital assistants, etc.

[0239] It should be noted here that the description of the above storage medium and device embodiments is similar to the description of the above method embodiments, and has similar beneficial effects as the method embodiments. For technical details not disclosed in the storage medium and device embodiments of this application, please refer to the description of the method embodiments of this application for understanding.

[0240] It should be understood that the "some embodiments" mentioned throughout the specification means that specific features, structures or characteristics related to the embodiments are included in at least one embodiment of the present application. Therefore, the "in some embodiments" appearing throughout the specification may not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It should be understood that in various embodiments of the present application, the size of the sequence number of the above-mentioned processes does not mean the order of execution, and the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application. The above-mentioned sequence numbers of the embodiments of the present application are only for description and do not represent the advantages and disadvantages of the embodiments.

[0241] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0242] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0243] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units; they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the embodiment of the present application.

[0244] In addition, all functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may be a separate unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0245] Alternatively, if the above-mentioned integrated unit of the present application is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application can essentially or in other words, the part that contributes to the relevant technology can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for the automatic test line of the device to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROMs, magnetic disks or optical disks.

[0246] The methods disclosed in several method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.

[0247] The features disclosed in several method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments or device embodiments.

[0248] The above is only an implementation method of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for extracting table data, characterized in that: The method comprises: Get the table image; Detecting the table lines of the table image to obtain a detection result; dividing a plurality of cells according to the detection result; According to the pre-acquired title image library, the plurality of cells are divided into at least one title cell and at least one data cell; and the title text in the at least one title cell and the data text in the at least one data cell are determined; According to the title text in the at least one title cell and the data text in the at least one data cell, at least one set of table data is obtained, wherein each set of table data in the at least one set of table data includes at least one title text and data text corresponding to the at least one title text; The detecting of the table lines of the table image to obtain the detection result includes: Perform edge detection on the table image to obtain an edge detected image; use a standard Hough transform method to detect the edge detected image to obtain multiple straight lines; determine the shortest line segment length in the edge detected image and the maximum interval between adjacent parallel line segments in the edge detected image based on the positional relationship of the multiple straight lines; based on the shortest line segment length in the edge detected image and the maximum interval between adjacent parallel line segments in the edge detected image, use a probabilistic Hough transform method to detect the edge detected image to obtain each line segment in the edge detected image; obtain the detection result based on each line segment in the edge detected image.

2. The table data extraction method according to claim 1, characterized in that: The step of dividing the plurality of cells into at least one title cell and at least one data cell according to the pre-acquired title image library comprises: Determine each non-blank cell among the plurality of cells; among the non-blank cells, use cells that match the title image in the title image library as title cells; and use cells among the non-blank cells excluding the title cells as data cells.

3. The table data extraction method according to claim 2, characterized in that: The method further comprises: After determining the non-blank cells, determine the Hamming distance between the title image in the title image library and the hash value of any non-blank cell; determine the number of title images that meet a first set condition, wherein the first set condition is that the Hamming distance between the title image and the hash value of any non-blank cell is less than a distance threshold; and when the number of title images that meet the first set condition is greater than the number threshold, determine that the any non-blank cell matches the title image in the title image library.

4. The table data extraction method according to claim 3, characterized in that: The determining the title text in the at least one title cell comprises: Determine an image set, the image set comprising: a title image matching any one of the at least one title cell; Determine, among the title images in the image set, a title image that satisfies a second set condition, wherein the second set condition is that a Hamming distance between the title image and a hash value of any one of the title cells is the smallest; The title text of any one title cell is determined as: the title text corresponding to the title image that meets the second set condition.

5. The table data extraction method according to any one of claims 1 to 4, characterized in that: The method further comprises: Determine the positions of the plurality of cells according to the detection result; The step of obtaining at least one set of table data according to the title text in the at least one title cell and the data text in the at least one data cell comprises: Determine a first correspondence between the title cell and the data cell in the plurality of cells according to the positions of the plurality of cells; determine a second correspondence between the title text and the data text in the plurality of cells according to the first correspondence; The at least one set of table data is obtained according to the second corresponding relationship.

6. The table data extraction method according to claim 5, characterized in that: The determining, according to the positions of the plurality of cells, a first corresponding relationship between the header cell and the data cell in the plurality of cells comprises: According to the positions of the multiple cells, searching leftward in the same row of each data cell until a header cell is found; and / or, according to the positions of the multiple cells, searching upward in the same column of each data cell until a header cell is found; According to the header cell found in the same row and / or the same column of each data cell, a first corresponding relationship between the found header cell and each data cell is determined.

7. The table data extraction method according to claim 1, characterized in that: Determining the shortest line segment length in the image after edge detection and the maximum interval between adjacent parallel line segments in the image after edge detection according to the positional relationship of the plurality of straight lines comprises: Determine, according to the positional relationship of the plurality of straight lines, adjacent distances between respective horizontal lines of the plurality of straight lines and adjacent distances between respective vertical lines of the plurality of straight lines; The shortest line segment length in the edge detected image and the maximum interval between adjacent parallel line segments in the edge detected image are determined according to adjacent distances between respective horizontal lines of the plurality of straight lines and adjacent distances between respective vertical lines of the plurality of straight lines.

8. The table data extraction method according to claim 1 or 7, characterized in that: The method of detecting the image after edge detection by using a probabilistic Hough transform method according to the length of the shortest line segment in the image after edge detection and the maximum interval between adjacent parallel line segments in the image after edge detection to obtain each line segment in the image after edge detection includes: According to the length of the shortest line segment in the image after edge detection and the maximum interval between adjacent parallel line segments in the image after edge detection, the image after edge detection is detected by using a probabilistic Hough transform method to obtain the starting and ending points of each line segment in the image after edge detection; Each line segment in the image after edge detection is determined according to the starting and ending points of each line segment in the image after edge detection and the plurality of straight lines.

9. A table data extraction device, characterized in that: The device comprises: An acquisition module, used for acquiring a table image; A first processing module is used to detect the table lines of the table image to obtain a detection result; and divide a plurality of cells according to the detection result; A second processing module is used to divide the plurality of cells into at least one title cell and at least one data cell according to a pre-acquired title image library; determine the title text in the at least one title cell and the data text in the at least one data cell; A third processing module, configured to obtain at least one set of table data according to the title text in the at least one title cell and the data text in the at least one data cell, wherein each set of table data in the at least one set of table data includes at least one title text and data text corresponding to the at least one title text; The first processing module is specifically used to perform edge detection on the table image to obtain an edge detected image; use a standard Hough transform method to detect the edge detected image to obtain multiple straight lines; determine the shortest line segment length in the edge detected image and the maximum interval between adjacent parallel line segments in the edge detected image based on the positional relationship of the multiple straight lines; based on the shortest line segment length in the edge detected image and the maximum interval between adjacent parallel line segments in the edge detected image, use a probabilistic Hough transform method to detect the edge detected image to obtain each line segment in the edge detected image; and obtain the detection result based on each line segment in the edge detected image.

10. An electronic device, characterized in that: The electronic device comprises: A memory for storing executable instructions; A processor is used to implement the table data extraction method described in any one of claims 1 to 8 when executing the executable instructions stored in the memory.

11. A computer-readable storage medium, characterized in that: Executable instructions are stored, and when executed by a processor, the table data extraction method according to any one of claims 1 to 8 is implemented.

Images