Table information extraction method, device and storage medium

Abstract

The embodiment of the present application relates to the field of computer technology, and discloses a table information extraction method, device, and storage medium. The method includes: obtaining an initial table of a target image, and a correspondence between each target text area of ​​the target image and each cell in the initial table; then determining a target cell group in the initial table; if the sum of the first number of target text areas corresponding to each cell in the m2th row meets a first preset condition, then merging the m2th row and the m1th row of the initial table to obtain a target table; finally, based on the correspondence between each target text area and each cell in the initial table, filling the target table with text data in the target text area corresponding to each cell in the target table to obtain table data. By using the embodiment of the present application, it is possible to implement table structuring of images containing non-full-line tables, thereby extracting table data of images containing non-full-line tables.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a table information extraction method, device, and storage medium. Background Art

[0002] Detecting and structuring tables in text data images—generating tables based on the text data in the image—can be applied to a variety of business scenarios, such as expense lists or insurance policies. Currently, there are many technologies for structuring tables with complete lines, but for images containing incomplete lines, how to structure these images is a pressing technical challenge. Summary of the Invention

[0003] The embodiments of the present application provide a table information extraction method, device and storage medium, which can realize table structuring of an image containing a non-full line table, thereby extracting the table data of the image containing the non-full line table.

[0004] In one aspect, an embodiment of the present application provides a method for extracting table information, comprising:

[0005] Acquire an initial table of a target image, and a correspondence between each target text area of ​​the target image and each cell in the initial table, wherein each target text area is obtained by performing text detection processing on the target image;

[0006] Determining a target cell group in the initial table, the target cell group includes a first cell and a second cell, the first cell and the second cell are located in the same column, the first cell is located in the m1th row of the initial table, the second cell is located in the m2th row of the initial table, m1 and m2 are both positive integers, m1 is less than m2, and the first cell and the second cell both have corresponding target text areas;

[0007] If the sum of a first number of target text areas corresponding to the cells in the m2-th row meets a first preset condition, merging the m2-th row and the m1-th row of the initial table to obtain a target table, wherein the target text areas corresponding to the cells in the m1-th row of the target table include the target text areas corresponding to the corresponding cells in the m1-th row of the initial table and the target text areas corresponding to the corresponding cells in the m2-th row;

[0008] Based on the correspondence between each target text area and each cell in the initial table, the text data in the target text area corresponding to each cell in the target table is filled into the target table to obtain table data.

[0009] In one aspect, an embodiment of the present application provides a table information extraction device, the table information extraction device comprising an acquisition unit, a determination unit, a merging unit, and a filling unit, wherein:

[0010] The acquisition unit is configured to acquire an initial table of a target image, and a correspondence between each target text area of ​​the target image and each cell in the initial table, wherein each target text area is obtained by performing text detection processing on the target image;

[0011] The determining unit is configured to determine a target cell group in the initial table, the target cell group including a first cell and a second cell, the first cell and the second cell being located in the same column, the first cell being located in the m1th row of the initial table, the second cell being located in the m2th row of the initial table, m1 and m2 being positive integers, m1 being less than m2, and the first cell and the second cell both having corresponding target text areas;

[0012] The merging unit is configured to merge the m2th row and the m1th row of the initial table to obtain a target table if a first quantity sum of target text areas corresponding to the individual cells in the m2th row satisfies a first preset condition, wherein the target text areas corresponding to the individual cells in the m1th row of the target table include the target text areas corresponding to the corresponding cells in the m1th row of the initial table and the target text areas corresponding to the corresponding cells in the m2th row;

[0013] The filling unit is configured to fill the target table with text data in the target text area corresponding to each cell in the target table based on the correspondence between each target text area and each cell in the initial table, so as to obtain table data.

[0014] In one aspect, an embodiment of the present application provides an electronic device, the electronic device including an input interface and an output interface, and further including:

[0015] a processor adapted to implement one or more instructions; and

[0016] A computer storage medium stores one or more instructions, wherein the one or more instructions are suitable for being loaded by the processor and executing the above table information extraction method.

[0017] On the one hand, an embodiment of the present application provides a computer storage medium, in which computer program instructions are stored. When the computer program instructions are executed by a processor, they are used to execute the above-mentioned table information extraction method.

[0018] On the one hand, an embodiment of the present application provides a computer program product or a computer program, wherein the computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium; a processor of an electronic device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions. When the computer instructions are executed by the processor, they are used to execute the above-mentioned table information extraction method.

[0019] In an embodiment of the present application, an initial table of the target image and the correspondence between each target text area of ​​the target image and each cell in the initial table are first obtained, and then a target cell group is determined in the initial table, wherein the target cell group includes a first cell and a second cell, the first cell and the second cell are located in the same column, the first cell is located in the m1th row in the initial table, and the second cell is located in the m2th row in the initial table; then, it is determined whether the sum of the first number of target text areas corresponding to each cell in the m2th row meets a first preset condition, and if so, the m2th row and the m1th row of the initial table are merged to obtain a target table; finally, based on the correspondence between each target text area and each cell in the initial table, the text data in the target text area corresponding to each cell in the target table is filled into the target table to obtain table data. In an embodiment of the present application, the initial position of each target text area in the initial table can be determined by obtaining the initial table and the correspondence between each target text area of ​​the target image and each cell in the initial table; then, by determining the target cell group in the initial table and judging whether the sum of the first quantity meets the first preset condition, the cells that should be merged in the initial table are determined, thereby obtaining a target table, wherein the target text areas in two cells in the target table that should originally belong to the same cell but are corresponding to different rows are merged. Therefore, this method avoids the situation where the text data of the line breaks is filled into different cells due to the lack of clear table lines, resulting in confusion of text data in the extracted table, thereby achieving the goal of not relying on tables with clear table lines such as full-line tables to determine which cells in the table certain target text areas are located, thereby achieving table structuring of images containing non-full-line tables and extracting table data of images containing non-full-line tables. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0021] Figure 1 This is a structural diagram of a table information extraction system provided in an embodiment of the present application;

[0022] Figure 2 This is a flow chart of a table information extraction method provided in an embodiment of the present application;

[0023] Figure 3 This is a schematic diagram of a merged cell provided in an embodiment of the present application;

[0024] Figure 4 is a schematic diagram of another merged cell provided in an embodiment of the present application;

[0025] Figure 5 This is a schematic diagram of another merged cell provided in an embodiment of the present application;

[0026] Figure 6 This is a schematic diagram of another merged cell provided in an embodiment of the present application;

[0027] Figure 7 This is a flow chart of another table information extraction method provided in an embodiment of the present application;

[0028] Figure 8a is a schematic diagram of an initial arrangement provided in an embodiment of the present application;

[0029] Figure 8b This is a schematic diagram of a rearrangement method provided in an embodiment of the present application;

[0030] Figure 8c is a schematic diagram of another rearrangement method provided in an embodiment of the present application;

[0031] Figure 9 This is a schematic diagram of another merged cell provided in an embodiment of the present application;

[0032] Figure 10 This is a flow chart of another method for extracting table information provided in an embodiment of the present application;

[0033] Figure 11 is a pixel distribution histogram provided in an embodiment of the present application;

[0034] Figure 12 This is a schematic diagram of text segmentation provided by an embodiment of the present application;

[0035] Figure 13 This is a flow chart of another method for extracting table information provided in an embodiment of the present application;

[0036] Figure 14 This is a schematic diagram of another merged cell provided in an embodiment of the present application;

[0037] Figure 15 This is a flow chart of another method for extracting table information provided in an embodiment of the present application;

[0038] Figure 16 This is a schematic diagram of the structure of a table information extraction device provided in an embodiment of the present application;

[0039] Figure 17 This is a structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0040] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0041] In order to perform table structuring on an image containing a non-full-line table and then extract the table data of the image containing the non-full-line table, an embodiment of the present application provides a table information extraction scheme, first obtaining the initial table of the target image, and the correspondence between each target text area of ​​the target image and each cell in the initial table; then determining the target cell group in the initial table, wherein the target cell group includes a first cell and a second cell, the first cell and the second cell are in the same column, the first cell is in the m1th row in the initial table, the second cell is in the m2th row in the initial table, m1 and m2 are both positive integers, m1 is less than m2, and the first cell and the second cell are in the m2th row in the initial table. There are corresponding target text areas for each cell; then, if the sum of the first number of target text areas corresponding to each cell in the m2-th row meets the first preset condition, the m2-th row and the m1-th row of the initial table are merged to obtain a target table, wherein the target text areas corresponding to each cell in the m1-th row of the target table include the target text areas corresponding to the corresponding cells in the m1-th row of the initial table and the target text areas corresponding to the corresponding cells in the m2-th row; finally, based on the correspondence between each target text area and each cell in the initial table, the text data in the target text areas corresponding to each cell in the target table are filled into the target table to obtain table data.

[0042] In addition, an image containing a non-full-line table may be an image containing a half-line table, an image containing a wireless table, or an image containing a full-line table, a half-line table, and a wireless table at the same time, which is not limited here. A full-line table refers to a table in which each cell has table lines on the top, bottom, left, and right four borders, a half-line table refers to a table in which only at most three of the top, bottom, left, and right four borders of each cell have table lines, and a wireless table refers to a table in which each cell has no table lines on the top, bottom, left, and right four borders. Optionally, this solution is a general table information extraction method, which is applicable to table extraction from images containing non-full-line tables as well as table extraction from images containing full-line tables, which is not limited here.

[0043] In one embodiment, the above-mentioned table information extraction scheme can be executed by a terminal device, wherein the terminal device can include any one or more of a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart car-mounted device, and a smart wearable device; the terminal device can also be an independent server, a cloud server, a server cluster, or a distributed system, etc., which is not limited here. The terminal device can use an image captured by a user as a target image, or select an image from an image database in the terminal device as the target image, and then obtain an initial table of the target image and the correspondence between each target text area of ​​the target image and each cell in the initial table; then the terminal device determines a target cell group in the initial table; then, the terminal device determines whether the sum of the first number of target text areas corresponding to each cell in the m2th row meets a first preset condition, and if so, the terminal device merges the m2th row of the initial table with the m1th row to obtain a target table; finally, based on the correspondence between each target text area and each cell in the initial table, the terminal device fills the target table with the text data in the target text area corresponding to each cell in the target table to obtain table data; in addition, the terminal device can also output the table data to facilitate the user to view or confirm the extracted table data.

[0044] Based on the above table information extraction solution, the embodiment of the present application provides a table information extraction system. Figure 1 , is a structural diagram of a table information extraction system provided in an embodiment of the present application. Figure 1The table information extraction system shown may include a terminal device 101 and a server 102. The terminal device 101 may include any one or more of a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart car-mounted device, and a smart wearable device. The server 102 may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. The terminal device 101 and the server 102 may be directly or indirectly connected in communication via wired or wireless communication, which is not limited in this application.

[0045] In one embodiment, a user of terminal device 101 can select a captured image as a target image, or select an image from an image database in terminal device 101 as a target image. Terminal device 101 then uploads the target image to server 102. Server 102 obtains an initial table of target images and the correspondence between each target text area of ​​the target image and each cell in the initial table. Server 102 then determines a target cell group in the initial table. Server 102 then determines whether the sum of a first number of target text areas corresponding to each cell in row m2 satisfies a first preset condition. If so, server 102 merges row m2 with row m1 of the initial table to obtain a target table. Finally, based on the correspondence between each target text area and each cell in the initial table, server 102 populates the target table with text data in the target text area corresponding to each cell in the target table to obtain table data. Finally, server 102 sends the table data to terminal device 101, which then outputs the table data to facilitate the user to view and confirm the extracted table data.

[0046] Based on the above table information extraction solution and table information extraction system, the present application embodiment provides a table information extraction method. Figure 2 , which is a flow chart of a table information extraction method provided in an embodiment of the present application. Figure 2 The table information extraction method shown can be used by Figure 1 The server or terminal device shown is executed. Figure 2 The table information extraction method shown may include the following steps:

[0047] S201 , obtaining an initial table of a target image, and a correspondence between each target text area of ​​the target image and each cell in the initial table.

[0048] In the embodiments of the present application, the target image may be a captured image, an image selected from multiple images in an image database, or an image received by a server or terminal device, without limitation. The terminal device or server may select an image by a user selecting one or more images from multiple images as the target image, or by the terminal device or server randomly selecting one or more images from multiple images as the target image, without limitation.

[0049] In an embodiment of the present application, the initial table may include m×n cells, where m is determined based on the position information of the target text area with the minimum vertical coordinate and the target text area with the maximum vertical coordinate in the target image, and n is determined based on the position information of the target text area with the minimum horizontal coordinate and the target text area with the maximum horizontal coordinate in the target image. Both m and n are positive integers, m refers to the number of rows of the initial table, and n refers to the number of columns of the initial table.

[0050] Specifically, the maximum number of target text areas between the target text area with the smallest horizontal coordinate and the target text area with the largest horizontal coordinate in the same vertical coordinate range in the target image can be detected, and then 2 can be added to the maximum number to determine n in the initial table, that is, the number of columns.

[0051] For example, there are four target text areas between the target text area A1 with the smallest horizontal coordinate and the target text area A2 with the largest horizontal coordinate, whose vertical coordinate is between 200 and 300 pixels in the target image; there are three target text areas between the target text area B1 with the smallest horizontal coordinate and the target text area B2 with the largest horizontal coordinate, whose vertical coordinate is between 500 and 600 pixels in the target image. Therefore, it can be determined that the maximum number of target text areas between the target text area with the smallest horizontal coordinate and the target text area with the largest horizontal coordinate within the same vertical coordinate range is 4, thereby determining n in the initial table to be 6. Optionally, the method for determining m in the initial table is similar to the method for determining n, which will not be repeated here.

[0052] Optionally, the horizontal coordinates of all target text areas in the target image can be determined first, and the target text areas whose vertical coordinate differences are within a preset difference value can be determined as the same text area group. Then, the number of text area groups can be counted to determine the text area group as m in the initial table.

[0053] For example, the preset difference is set to 20 pixels, the vertical coordinate of the target text area 1 in the target image is 100-200 pixels, the vertical coordinate of the target text area 2 in the target image is 110-210 pixels, the vertical coordinate of the target text area 3 in the target image is 300-400 pixels, the vertical coordinate of the target text area 4 in the target image is 298-390 pixels, the vertical coordinate of the target text area 5 in the target image is 396-550 pixels, and the vertical coordinate of the target text area 6 in the target image is 700-800 pixels. The vertical coordinate of target text area 7 in the target image is 712 to 790 pixels; therefore, it can be determined that target text area 1 and target text area 2 are a text area group, target text area 3 and target text area 4 are a text area group, and target text area 6 and target text area 7 are a text area group; and although the vertical coordinate of target text area 5 partially overlaps with the horizontal coordinate of target text area 3, the difference between the vertical coordinates is greater than 20 pixels, so target text area 5 is a separate text area group; finally, it can be determined that m in the initial table is 4.

[0054] Optionally, the method for determining n in the initial table is similar to the method for determining m, which is not described in detail here.

[0055] In the embodiment of the present application, the method for obtaining the correspondence between each target text region of the target image and each cell in the initial table means that each target text region should belong to each cell in the initial table. In other words, if the target text region P is located at the 2nd row and 1st column in the target image, then the target text region P will also correspond to the cell at the 2nd row and 1st column in the initial table.

[0056] Optionally, each target text area is obtained by performing text detection processing on the target image. Specifically, the target image can be detected by optical character recognition (OCR) technology to obtain each target text area; or the target image can be subjected to text detection processing by a trained text detection model to obtain each target text area. Exemplarily, the text detection model can be a deep learning model such as DBNet (a differentiable binary segmentation network for text detection), CTPN (a network for text detection including a convolutional neural network and a recurrent neural network), SegLink (a convolutional neural network that can detect text with a rotation angle), etc. Different models can be flexibly selected based on different needs, which is not limited here. Among them, the process of training the text detection model is a technical means commonly used by those skilled in the art and will not be elaborated here.

[0057] Optionally, after performing text detection processing on the target image to obtain all text areas in the target image, table detection processing can also be performed on the target image to determine the table area of ​​the target image, and then the text area belonging to the table area in all text areas of the target image is determined as the target text area. Specifically, the method of performing table detection processing on the target image can be to train a machine learning model such as Yolov5 (a real-time object detection model), and then use the trained model to detect tables such as line tables, half-line tables, and full-line tables in the target image to obtain the coordinates of each table, thereby determining the table area, and then it can be determined to which table each text area obtained by the text detection processing should belong, so as to distinguish the text content in different tables, and facilitate subsequent table structuring processing.

[0058] S202: Determine a target cell group in the initial table, wherein the target cell group includes a first cell and a second cell, wherein the first cell is located in row m1 of the initial table, and the second cell is located in row m2 of the initial table, wherein m1 and m2 are both positive integers, and m1 is smaller than m2.

[0059] In an embodiment of the present application, the first and second cells of the target cell group are located in the same column, and both the first and second cells have corresponding target text areas. Specifically, the first and second cells of the target cell group can be two cells located in the same column in two adjacent rows in the initial table; the first and second cells of the target cell group can also be two non-adjacent rows in the initial table, but a third cell located between the first and second cells does not have a corresponding target text area; wherein the third cell refers to a cell located in the same column as the first cell and in a row between rows m1 and m2 in the initial table.

[0060] For example, in the initial table, cell W1 is in row 5, column 4, cell W2 is in row 3, column 3, cell W3 is in row 4, column 4, cell W4 is in row 2, column 3, cell W5 is in row 4, column 3, and cell W6 is in row 5, column 3. All cells except W2 and W5 have corresponding target text regions. Since W1 and W3 are in adjacent rows and the same column, they can be determined to be a target cell group. Furthermore, since W4 and W6 are in the same column, and there are no corresponding target text regions for W2 and W5 between them, they can also be determined to be a target cell group.

[0061] S203 : If the sum of the first number of target text areas corresponding to the cells in the m2 th row meets a first preset condition, the m2 th row and the m1 th row of the initial table are merged to obtain a target table.

[0062] S204 , based on the correspondence between each target text area and each cell in the initial table, fill the target table with text data in the target text area corresponding to each cell in the target table to obtain table data.

[0063] In an embodiment of the present application, based on the correspondence between each target text area and each cell in the initial table, the text data in the target text area corresponding to each cell in the target table is filled into the target table to obtain the tabular data, which can be: writing the text data in each target text area into each cell corresponding to each target text area to obtain a table containing text data, thereby generating the tabular data. Optionally, the format of the tabular data can be a table such as Excel that is easy to view and use, or a table such as JSON (JavaScript Object Notation, JS Object Notation) that is convenient for subsequent business development through tabular data, which is not limited here.

[0064] In an embodiment of the present application, the first preset condition may be that the sum of the first quantity is less than the sum of the second quantity of the target text areas corresponding to the cells in the m1-th row. Then, a method for determining whether the sum of the first quantity satisfies the first preset condition may be to obtain the sum of the second quantity of the target text areas corresponding to the cells in the m1-th row; if the sum of the second quantity is greater than the sum of the first quantity, then determining that the sum of the first quantity satisfies the first preset condition.

[0065] For example, see the attached Figure 3 , Figure 3 The initial table of the target image 301 and the target text area corresponding to each cell in the initial table are shown in FIG. Figure 3 As shown in the initial table 302 in the target image 301, since "111111111111" and "xxxxxxxxxxxx" which should belong to the same cell in the target image 301 are both wrapped, "111111111111" will be recognized as "11111111" and "1111", and "xxxxxxxxxxxx" will be recognized as "xxxxxx" and "xxxxxx", so there are a total of 4 rows of cells in the obtained initial table 302. Through the positional relationship between the cells in the initial table 302 and the corresponding relationship between the cells and the target text area, it can be determined as follows: Figure 3The eight target cell groups shown are as follows. In each target cell group, the cell at the top is the first cell, and the cell at the bottom is the second cell. There are three cells in the third row of the initial table 302 where the cell 304 in the target cell group 4 is located. The target text areas corresponding to the first two cells in the third row are "1111" and "xxxxxx" respectively, while the last cell in the third row has no corresponding target text area. Therefore, it can be determined that the first quantity sum of the target text areas corresponding to the cells in the third row where the cell 304 in the target cell group 4 is located is 2. There are three cells in the second row of the initial table 302 where the cell 303 is located. The target text areas corresponding to the three cells are "1111", "xxxxxx" and "300.00" respectively. Therefore, it can be determined that the second quantity sum of the target text areas corresponding to the cells in the second row where the cell 303 in the target cell group 4 is located is 3. Therefore, it can be determined that the second quantity sum is greater than the first quantity sum, thereby determining to merge the third row of the initial table 302 into the second row, and finally obtaining the target table 307, wherein, as shown in FIG. Figure 3 The target table 307 shown indicates the target text area corresponding to each cell.

[0066] In one embodiment, the first preset condition may be: the sum of the first quantities is less than or equal to the first preset value. Optionally, the first preset value may be less than or equal to the sum of the second quantities of the target text areas corresponding to the cells in the m1-th row. The first preset value may be manually set or system set, which is not limited here. For example, the first preset value may be 1, 5, 8, etc. Optionally, the method for determining whether the sum of the first quantities meets the first preset condition may be: if the sum of the second quantities is less than or equal to the first preset value, then it is determined that the sum of the first quantities meets the first preset condition.

[0067] For example, see the attached Figure 4 , Figure 4 Another schematic diagram of merging cells is shown. Set the first preset value to 1, and obtain the initial table and the target text area corresponding to each cell in the initial table as shown in FIG. Figure 4 By comparing the first quantity sum of the target text areas corresponding to the cells in the row where the second cell of each target cell group in the initial table 401 is located with the first preset value, it can be determined that only the target cell group consisting of the cell in the 4th row and the 2nd column and the cell in the 5th row and the 2nd column meets the first preset condition. Therefore, it can be determined that the 5th row is merged into the 4th row, and finally the target table 402 is obtained, where, as shown in FIG. Figure 4 The target table 402 shown is labeled with the target text area corresponding to each cell.

[0068] In one embodiment, if the sum of the first quantities of target text areas corresponding to the cells in the m2th row meets the first preset condition, the m2th row of the initial table is merged with the m1th row to obtain the target table. The method can be: 1) if the sum of the first quantities of target text areas corresponding to the cells in the m2th row meets the first preset condition, the m2th row of the initial table is merged with the m1th row to obtain an updated table; 2) the updated table is used as the initial table, and the target cell group is triggered to be determined in the initial table until the sum of the first quantities of target text areas corresponding to the cells in the m2th row does not meet the first preset condition, then the initial table to which the cells corresponding to the target text areas that do not meet the first preset condition belong is determined as the target table, wherein the target table includes p×q cells, p is less than or equal to m, p and q are positive integers, p refers to the number of rows in the target table, q refers to the number of columns in the target table, and m refers to the number of rows in the initial table. Since rows may be merged in the process of updating the table, p is less than or equal to m.

[0069] Specifically, when the sum of the first number of target text areas corresponding to the cells in the m2th row where the second cell in the target cell group is located meets the first preset condition, the m1th row where the first cell in the target cell group is located is merged with the m2th row where the second cell in the target cell group is located; after completing the merge of the m1th row and the m2th row, an updated table is obtained. Then, a new target cell group is determined again in the updated table, and it is determined whether there is a target cell group that meets the first preset condition in the new target cell group. If so, the merge is continued and the table is updated; if not, the updated table is determined as the target table.

[0070] For example, see the attached Figure 5 , Figure 5A schematic diagram of another type of merged cells is shown. The first preset condition can be set as the sum of the first quantities of each target cell group is less than or equal to 1. The initial table 502 of the target image 501 is obtained, wherein the initial table 502 indicates the target text area corresponding to each cell. The initial table 502 is a table with 7 rows and 3 columns as shown in table 503. For the convenience of description, the cells of table 503 are encoded in sequence to obtain cells 1 to 21. Through the positional relationship between the cells in table 503 and the correspondence between the cells and the target text area, it can be determined that the target cell groups of table 503 include: {1,4}, {4,7}, {7,10}, {10,13}, {13,19}, {2,5}, {5,14}, {14,17}, {17,20}, {3,6}, {6,15}, {15,21}. We can select any target cell group from the multiple target cell groups in table 503 for evaluation. For example, after selecting target cell group {10, 13}, the sum of the first quantities in row 5, where cell 13 is located, is 3, which does not meet the first preset condition. We then select another target cell group from table 503, such as {14, 17}. Since the sum of the first quantities in row 6, where cell 17 is located, is 1, which meets the first preset condition, we can determine to merge row 6 of table 503 into row 5, thereby obtaining an updated table, namely, table 504.

[0071] After obtaining table 504, it can be determined that the target cell groups of table 504 include: {1,4}, {4,7}, {7,10}, {10,(13,16)}, {(13,16),19}, {2,5}, {5,(14,17)}, {(14,17),20}, {3,6}, {6,(15,18)}, {(15,18),21}. Any target cell group can be selected from the multiple target cell groups in table 504 for judgment. For example, after selecting target cell group {1,4}, the first quantity sum of the second row where cell 4 is located is 3, which does not meet the first preset condition. Another target cell group is retrieved from table 504. If the target cell group {4, 7} is retrieved, since the first quantity sum of the third row where cell 7 is located is 1, which satisfies the first preset condition, it can be determined that the third row in table 504 is merged with the second row, thereby obtaining another updated table, namely table 505. Similarly, after updating table 505 again, table 506 is obtained.

[0072] After obtaining table 506, it can be determined that the target cell groups of table 506 include: {1, (4, 7, 10)}, {(4, 7, 10), (13, 16)}, {(13, 16), 19}, {2, (5, 8, 11)}, {(5, 8, 11), (14, 17)}, {(14, 17), 20}, {3, (6, 9, 12)}, {(6, 9, 12), (15, 18)}, {(15, 18), 21}. Since no target cell group in table 506 satisfies the first preset condition, table 506 can be determined to be the target table. Finally, based on the correspondence between each target text area and each cell in the initial table, the text data in the target text area corresponding to each cell in the target table is filled into the target table, and table data 507 can be obtained. For example, the cells in the 2nd row and 1st column of table 506 are merged by the original cells 4, 7 and 10, then the text data in the target text area corresponding to the original cells 4, 7 and 10 will be filled into the cells in the 2nd row and 1st column. Therefore, it can be determined that the text data in the cells in the 2nd row and 1st column of table 506 is "1111111112222222223333".

[0073] In one embodiment, the method of determining the target cell group in the initial table can be: 1) determining at least one cell group in the initial table, each cell group including a first cell and a second cell; 2) obtaining the target text area corresponding to the first cell in each cell group and the distance between the target text area corresponding to the second cell in each cell group in the target image; 3) determining the cell group with the smallest distance as the target cell group.

[0074] Optionally, the distance between the target text area corresponding to the first cell and the target text area corresponding to the second cell in the target image can be: the distance between the area center point of the target text area corresponding to the first cell and the area center point of the target text area corresponding to the second cell; it can also be the distance between the area bottom border of the target text area corresponding to the first cell and the area top border of the target text area corresponding to the second cell; optionally, it can also be other distances between the target text area corresponding to the first cell and the target text area corresponding to the second cell in the target image, which is not limited here. Optionally, the distance can be pixels, or unit lengths such as millimeters and centimeters, which is not limited here. For example, the distance can be 100 pixels, 0.8 mm, etc.

[0075] Specifically, under normal circumstances, the smaller the distance between the target text areas corresponding to two cells that each occupy a row in the initial table, the more likely it is that the two target text areas should belong to the same cell. However, since the line break is recognized as belonging to two cells, the possibility is greater. Therefore, in order to further improve the merging efficiency, the cell group with the smallest distance can be determined as the target cell group.

[0076] Optionally, since during the design process of expense details, insurance reimbursement and other invoices, the text data in the first row or column is often text data such as table names or headers that will not have line breaks, the cells in the first row or column can be excluded in the process of determining the target cell group.

[0077] For example, see the attached Figure 6 , Figure 6 A schematic diagram of another type of merged cells is shown. The first preset condition is set as follows: the sum of the first number of target text areas corresponding to the cells in the m2th row of the initial table where the second cell in the target cell group is located is less than or equal to 1. The initial table 602 of the target image 601 is obtained, wherein the initial table 602 indicates the target text area corresponding to each cell. The initial table 602 is a table with 5 rows and 3 columns as shown in table 603. For the convenience of description, the cells of table 603 are encoded in sequence to obtain cells 1 to 15. Through the positional relationship between the cells in table 503 and the correspondence between the cells and the target text area, it can be determined that the cell group of table 603 includes: {1,4}, {4,7}, {7,13}, {2,5}, {5,8}, {8,11}, {11,14}, {3,6}, {6,9}, {9,15}. Then, the distance between the target text area corresponding to the first cell and the target text area corresponding to the second cell in each cell group in the target image 601 is obtained, and each cell group is sorted from small to large according to the distance. The sorted cell groups are: {8,11}, {11,14}, {1,4}, {2,5}, {3,6}, {4,7}, {5,8}, {6,9}, {7,13}, {9,15}. Therefore, it can be determined that the cell group {4,7} with the smallest distance in the cell group of table 603 is the target cell group. Since the first quantity sum of the target text areas corresponding to the third row where cell 7 is located meets the first preset condition, it can be determined that the third row in table 603 is merged into the second row to obtain the target table 604. Finally, the text data in the target text area corresponding to each cell in the target table 604 is filled into the cell to obtain table data 605.

[0078] Optionally, in the method of obtaining the target table by cyclically updating the initial table, each time the updated table is obtained, the target cell group in the updated table can be determined by comparing the distance between the target text areas corresponding to two cells.

[0079] In an embodiment of the present application, an initial table of the target image and the correspondence between each target text area of ​​the target image and each cell in the initial table are first obtained, and then a target cell group is determined in the initial table, wherein the target cell group includes a first cell and a second cell, the first cell and the second cell are located in the same column, the first cell is located in the m1th row in the initial table, and the second cell is located in the m2th row in the initial table; then, it is determined whether the sum of the first number of target text areas corresponding to each cell in the m2th row meets a first preset condition, and if so, the m2th row and the m1th row of the initial table are merged to obtain a target table; finally, based on the correspondence between each target text area and each cell in the initial table, the text data in the target text area corresponding to each cell in the target table is filled into the target table to obtain table data. In an embodiment of the present application, the initial position of each target text area in the initial table can be determined by obtaining the initial table and the correspondence between each target text area of ​​the target image and each cell in the initial table; then, by determining the target cell group in the initial table and judging whether the sum of the first quantity meets the first preset condition, the cells that should be merged in the initial table are determined, thereby obtaining a target table, wherein the target text areas in two cells in the target table that should originally belong to the same cell but are corresponding to different rows are merged. Therefore, this method avoids the situation where the text data of the line breaks is filled into different cells due to the lack of clear table lines, resulting in confusion of text data in the extracted table, thereby achieving the goal of not relying on tables with clear table lines such as full-line tables to determine which cells in the table certain target text areas are located, thereby achieving table structuring of images containing non-full-line tables and extracting table data of images containing non-full-line tables.

[0080] Based on the above table information extraction solution and table information extraction system, the present application embodiment provides another table information extraction method. Figure 7 , which is a flow chart of another table information extraction method provided in an embodiment of the present application. Figure 7 The table information extraction method shown can be used by Figure 1 The server or terminal device shown is executed. Figure 7 The table information extraction method shown may include the following steps:

[0081] S701: Acquire position information of each target text region in the target image, wherein the position information may include the horizontal coordinate and the vertical coordinate of each target text region in the target image.

[0082] In an embodiment of the present application, the target image can be subjected to rotation correction processing before obtaining the position information of each target text area in the target image, wherein the rotation correction processing can include large-angle (90°, 180°, 270°) and small-angle tilt correction (within 45°), so that the position information of the target text areas that should be located in the same row or column detected during subsequent OCR recognition will not have large errors. For example, if the vertical coordinates of the three target text areas in the same row differ greatly due to the tilt of the target image, then when the table is subsequently structured, these three target text areas can easily be considered to belong to three different rows of cells.

[0083] S702: Determine an initial table of the target image based on the position information of each target text area.

[0084] In an embodiment of the present application, the method of determining the initial table of the target image based on the position information of each target text area can be: 1) arranging each target text area based on the position information of each target text area to obtain the arranged target text area; 2) determining the initial table based on the arranged target text area, wherein the initial table may include m×n cells, m is determined based on the target text area with the smallest vertical coordinate and the target text area with the largest vertical coordinate in the arranged target text area, and n is determined based on the target text area with the smallest horizontal coordinate and the target text area with the largest horizontal coordinate in the arranged target text area.

[0085] In one embodiment, the target text regions are arranged based on their position information to obtain the arranged target text regions in the following manner:

[0086] 1) Obtaining the position information of the center point of each target text region in the target image. The position information of the center point of each target text region in the target image refers to the horizontal and vertical coordinates of the center point of each target text region in the target image.

[0087] 2) sorting the target text areas based on the position information of the area center points of the target text areas to obtain sorted target text areas, wherein the ordinate of the area center point of the xth target text area in the sorted target text areas is less than or equal to the ordinate of the area center point of the x+1th target text area, where x is a positive integer and is less than the number of target text areas in the sorted target text areas.

[0088] Specifically, after obtaining the position information of the area center point of each target text area, each target text area is first sorted in ascending order according to the vertical coordinates of the area center point of each target text area; if the vertical coordinates of the area center points of multiple target text areas are the same, or the vertical coordinates of the area center points of multiple target text areas are very small, then each target text area in the multiple target text areas is sorted in ascending order according to the horizontal coordinates of the area center point of each target text area, and finally the sorted target text areas are obtained.

[0089] For example, there are a total of 8 target text areas in the target image, among which the coordinates of the center point of target text area 1 are (100, 100), the coordinates of the center point of target text area 2 are (200, 105), the coordinates of the center point of target text area 3 are (400, 98), the coordinates of the center point of target text area 4 are (300, 100), the coordinates of the center point of target text area 5 are (200, 305), the coordinates of the center point of target text area 6 are (100, 302), the coordinates of the center point of target text area 7 are (300, 299), and the coordinates of the center point of target text area 8 are (400, 308).

[0090] Then, target text areas with center point ordinate differences of less than 10 pixels are defined as target text areas with identical ordinates. This means that the center points of target text areas 1-4 and 5-8 have identical ordinates. The horizontal coordinates of the center points of target text areas 1-4 and 5-8 are then compared. The resulting sorted target text areas are: target text area 1, target text area 2, target text area 4, target text area 3, target text area 6, target text area 5, target text area 7, and target text area 8.

[0091] 3) obtaining position information of each region border of the first target text region in the sorted target text regions in the target image;

[0092] 4) Arranging the sorted target text regions based on the ordinates of the first region border and the second region border of the first target text region and the ordinates of the region center points of each target text region subsequent to the first target text region to obtain initially arranged target text regions;

[0093] Specifically, the first region border may be the uppermost border of the target text region in the target image, and the second region border may be the lowermost border of the target text region in the target image.

[0094] In addition, the method for arranging the sorted target text areas based on the vertical coordinates of the first area border and the second area border of the first target text area, and the vertical coordinates of the area center points of each target text area after the first target text area can be: determining a first difference between the vertical coordinates of the area center points of each target text area after the first target text area and the first area border, and a second difference between the vertical coordinates of the area center points of each target text area after the first target text area and the second area border, first arranging the target text areas after the first target text area in ascending order of the first difference values; if the first difference values ​​of multiple target text areas are the same, or the first difference values ​​of multiple target text areas are very small, then arranging the target text areas in the multiple target text areas in ascending order of the second difference values ​​of each target text area, and finally obtaining the arranged target text areas.

[0095] In one embodiment, the method for arranging the sorted target text regions based on the vertical coordinates of the first region border and the second region border of the first target text region and the vertical coordinates of the region center points of each target text region subsequent to the first target text region to obtain the initially arranged target text regions may also be:

[0096] ① Traversing each target text area in the sorted target text area, determining a target text area whose center point's ordinate is greater than the ordinate of the first area border of the first target text area and less than the ordinate of the second area border of the first target text area, and the determined target text area and the first target text area are located in the same row in the target text area after the initial arrangement;

[0097] ② If the ordinate of the area center point of the next target text area determined in the sorted target text areas is greater than the ordinate of the first area border of the first target text area, and greater than the ordinate of the second area border of the first target text area, then the next target text area is determined to be located in the next row of the first target text area in the target text areas after the initial arrangement;

[0098] ③ Take the next target text area as the first target text area, and trigger the execution of the target text area whose vertical coordinate of the center point in the sorted target text area is greater than the vertical coordinate of the first area border of the first target text area, and smaller than the vertical coordinate of the second area border of the first target text area, so as to obtain the target text area after initial arrangement.

[0099] The order of the target text areas located in the same row in the target text areas after the initial arrangement is the same as the order of the target text areas located in the same row in the target text areas after the sorting.

[0100] For example, see Figure 8a , Figure 8a A schematic diagram of an initial arrangement is shown. After performing text detection on the target image 801, target text regions a to k are obtained; then, based on the position information of the center points of each target text region, each target text region is sorted, and the sorted target text regions are obtained as shown in FIG. Figure 8a Array G is shown. The first target text region a and the next target text region b in array G are taken out. Since the vertical coordinate of the region center point 802 of target text region b is greater than the vertical coordinate of the first region border 803 of target text region a and smaller than the vertical coordinate of the second region border 804 of target text region a, it can be determined that target text region b and target text region a are located in the same row.

[0101] Next, target text area b is taken as the first target text area. The next target text area is target text area c. Clearly, the ordinate of target text area c's area center point 805 is greater than the ordinate of target text area b's first area border 806, and greater than the ordinate of target text area b's second area border 807. Therefore, target text area c is determined to be located in the next row after target text area b. Therefore, the target text areas located in row 1 are determined to include target text area a and target text area b. Furthermore, since target text area a precedes target text area b in array G, target text area a also precedes target text area b in row 1.

[0102] Finally, if Figure 8a As shown, the first target text area and the next target text area are repeatedly determined and compared, and finally it can be determined that there are 5 rows of target text areas after the initial arrangement, among which the first row includes target text areas a~b, the second row includes target text areas c~d, the third row includes target text area f, the fourth row includes target text areas g~i, and the fifth row includes target text areas j~k.

[0103] 5) Based on the horizontal coordinates of the third area border and the fourth area border of the first target text area after the initial arrangement, and the horizontal coordinates of the area center points of other target text areas in the target text area after the initial arrangement, the target text areas after the initial arrangement are rearranged to obtain arranged target text areas.

[0104] Specifically, the third region border may be the rightmost border of the target text region in the target image, and the fourth region border may be the leftmost border of the target text region in the target image.

[0105] Furthermore, the target text areas after the initial arrangement are rearranged based on the abscissas of the third area border and the fourth area border of the first target text area after the initial arrangement, and the abscissas of the area center points of the other target text areas after the initial arrangement, to obtain the arranged target text areas in a manner that may be:

[0106] ① Determining, within the initially sorted target text areas, target text areas located in the same column after the arrangement, where the abscissa of the area center point of the target text areas in the same column is greater than the abscissa of the third area border of the target text area located in the first row of the target text areas in the same column, and is less than the abscissa of the fourth area border of the target text area located in the first row of the target text areas in the same column;

[0107] ② Based on the horizontal coordinate of the center point of each column of target text area, determine the nth position of each column of target text area in the arranged target text area. i Columns are formed to obtain the target text area after the arrangement. If each target text area is located in the mth target text area after the initial arrangement i row, then each target text area is located in the mth target text area after arrangement i OK, m i 、n i is a positive integer, m i Less than or equal to m, n i Less than or equal to n. The nth i Column refers to any column in the initial table including m×n cells, the mth i A row refers to any row in an initial table consisting of m×n cells.

[0108] Specifically, based on the horizontal coordinate of the center point of each column of target text area, it is determined that each column of target text area is located in the nth position in the arranged target text area. i The column method can be: determine the number of columns of target text areas in each column in descending order of the horizontal coordinate of the area center point of the first target text area in each column of target text areas, for example, the column of target text areas in which the horizontal coordinate of the area center point of the first target text area in each column of target text areas is the smallest is used as the first column of target text areas.

[0109] Optionally, based on the horizontal coordinate of the center point of each column of target text area, it is determined that each column of target text area is located in the nth position of the arranged target text area. i The column method can also be: calculate the average value of the horizontal coordinates of the center points of all target text areas in each column of target text areas, and determine the column number of each column of target text areas in descending order of the average values ​​of the horizontal coordinates, such as taking the column of target text areas with the smallest average value of the horizontal coordinates as the first column of target text areas.

[0110] For example, see Figure 8b , Figure 8b A schematic diagram of a rearrangement method is shown. After performing text detection processing on the target image 808, target text areas a to k can be obtained, wherein the area center point of each target text area is marked as a point at the area center of each target text area, such as the area center point of target text area a is point 817. It can be seen from the target image 808 that the horizontal coordinates of the area center points of the target text areas d, f, and h are greater than the horizontal coordinates of the third area border 812 of the target text area a, and are smaller than the horizontal coordinates of the fourth area border 811 of the target text area a. Therefore, it can be determined that the target text areas a, d, f, and h are in the same column; similarly, it can be determined that the target text areas b, e, i, and k are in the same column, and the target text areas c, g, and j are in the same column.

[0111] At the same time, since target text area c has the smallest horizontal coordinate among the target text areas in the first row of the three columns, followed by target text area a, and the largest horizontal coordinate of target text area b, it can be determined that target text areas c, g, and j are located in the first column, target text areas a, d, f, and h are located in the second column, and target text areas b, e, i, and k are located in the third column. Then, as shown in structure 815, after determining the row and column positions of each target text area, it can be determined that the maximum number of rows and columns of the target text areas in target image 808 is 5, and the maximum number of columns is 3; finally, it can be determined that initial table 816 includes 5×6 cells.

[0112] In one embodiment, the method of rearranging the initially arranged target text areas based on the abscissas of the third area border and the fourth area border of the first target text area after the initial arrangement and the abscissas of the area center points of the other target text areas after the initial arrangement to obtain the arranged target text areas may also be:

[0113] ① Traversing the target text areas in the first row of the target text areas after the initial arrangement, determining a target text area whose region center point has a horizontal coordinate greater than the horizontal coordinate of the third region border of the target text area in the first row and smaller than the horizontal coordinate of the fourth region border of the target text area in the first row, and the determined target text area and the target text area in the first row are located in the same column in the target text area after the arrangement;

[0114] ② If there are target text areas that are not in the same column as the target text areas in the first row, then determine the target text areas in the row closest to the first row among the target text areas that are not in the same column as the target text areas in the first row, and then use the target text areas that are not in the same column as the target text areas in the first row as the target text areas after initial sorting, and use the remaining target text areas in the closest row as the target text areas in the first row after initial arrangement, triggering the execution of traversal of the target text areas in the first row after initial arrangement, and determining the target text areas in the target text areas after initial sorting whose horizontal coordinates of the center points of the regions are greater than the horizontal coordinates of the third region border of the target text area in the first row and smaller than the horizontal coordinates of the fourth region border of the target text area in the first row, so as to obtain the arranged target text areas.

[0115] ③ Based on the horizontal coordinate of the center point of each column of target text area, determine the nth position of each column of target text area in the arranged target text area. i columns to obtain the arranged target text area.

[0116] For example, see the attached Figure 8c , Figure 8c A schematic diagram showing another rearrangement method is shown. Figure 8a As shown, the target text areas in the first row of the target text areas after the initial arrangement can be determined to be target text areas a and target text areas b. Then, the target text areas c, d, and e in the second row of the target text areas after the initial arrangement are traversed. Since the horizontal coordinate of the center point of target text area d in the second row is the only target text area that is greater than the horizontal coordinate of the third area border of the target text area in the first row and smaller than the horizontal coordinate of the fourth area border of the target text area in the first row, it can be determined that target text area d is in the same column as target text area a. Similarly, the target text areas in the third to fifth rows of the target text area after the initial arrangement are traversed in sequence. Finally, it can be determined that the target text areas in the same column as target text area a include target text areas d, f, and h, and the target text areas in the same column as target text area b include target text areas e, i, and k.

[0117] After traversing the target text areas located in the first row among the target text areas after the initial arrangement, it is found that there are still remaining target text areas c and target text areas j in the target text areas after the initial arrangement, that is, there are target text areas that are not in the same column as the target text areas located in the first row.

[0118] At this point, the target text areas after the initial arrangement are determined to be target text areas c, g, and j. Furthermore, since target text area c is located in the second row, which is closest to the first row, target text area c is determined to be the target text area in the first row. Then, by traversing rows 2 to 5, it is determined that target text area g and target text area j are located in the same column as target text area c.

[0119] In addition, since among the target text areas in the 3 columns, the target text area c has the smallest horizontal coordinate, followed by the target text area a, and the target text area b has the largest horizontal coordinate; therefore, structure 815 can be obtained, and finally, through structure 815, it can be determined that the initial table 816 includes 5×6 cells.

[0120] S703: Obtain the corresponding relationship between each target text area of ​​the target image and each cell in the initial table.

[0121] S704: Determine a target cell group in the initial table, where the target cell group includes a first cell and a second cell, the first cell being located in row m1 of the initial table, and the second cell being located in row m2 of the initial table. m1 and m2 are both positive integers, m1 is less than m2, the first cell and the second cell of the target cell group are located in the same column, and both the first cell and the second cell have corresponding target text areas.

[0122] S705 : If the sum of the first number of target text areas corresponding to the cells in the m2th row meets a first preset condition, the m2th row and the m1th row of the initial table are merged to obtain a target table.

[0123] S706 , based on the correspondence between each target text area and each cell in the initial table, fill the target table with the text data in the target text area corresponding to each cell in the target table to obtain table data.

[0124] In an embodiment of the present application, if the sum of the first number of target text areas corresponding to the individual cells in the m2th row meets the first preset condition, then the m2th row of the initial table is merged with the m1th row to obtain the target table in the following manner: 1) determining the first data length of the text data of the target text area corresponding to the first cell, and the second data length of the text data of the target text area corresponding to the second cell; 2) if the sum of the first number of target text areas corresponding to the individual cells in the m2th row meets the first preset condition, and the first data length is greater than the second data length, then the m2th row of the initial table is merged with the m1th row to obtain the target table. Alternatively, the first data length may be greater than or equal to the second data length, or the first data length may be equal to the second data length.

[0125] Specifically, for text data with line breaks, the text data that is moved to the next line is generally text data that cannot be written in the table of the previous line. Therefore, the data length of the text data that is moved to the next line is generally not longer than the data length of the text data in the previous line. Therefore, while the sum of the first quantities meets the first preset condition, it is possible to further determine whether the first data length is greater than or equal to the second data length, thereby improving the accuracy in determining cell merging.

[0126] Optionally, the first data length and the second data length may be the text data length of the target text area, such as 200 pixels, 20 mm, etc., or the number of characters of the text data in the target text area, such as 10 characters, 8 characters, etc.

[0127] In one embodiment, if the sum of the first number of target text areas corresponding to the cells in the m2th row meets the first preset condition, the m2th row of the initial table is merged with the m1th row to obtain the target table. The method can also be: 1) performing line detection processing on the target image to obtain at least one line; 2) if there is no line in the target image that is located between the target text area corresponding to the first cell and the target text area corresponding to the second cell, then merging the m2th row of the initial table with the m1th row to obtain the target table. The at least one line can be a horizontal line or a diagonal line, which is not limited here. Optionally, the at least one line in step 1) can also be screened by factors such as the position, length and angle of the line, removing lines that are too short or have a relatively biased inclination angle, and determining at least one target line; then judging whether there is any target line in the target image that is located between the target text area corresponding to the first cell and the target text area corresponding to the second cell, and if so, merging the m2th row of the initial table with the m1th row to obtain the target table.

[0128] Optionally, the method of performing line detection processing on the target image can be to first binarize the target image, and then detect the target image after binarization processing by using the horizontal line detection interface (such as Huffman line detection interface, etc.) of OpenCV (a cross-platform computer vision and machine learning software library released under the Apache2.0 license), thereby obtaining at least one line in the target image. Optionally, it is also possible to train a deep learning model such as a regional convolutional neural network (Regions with CNN features, R-CNN), and determine the lines in the target image through the trained model. Optionally, other methods can also be used to perform line detection processing on the target image, which are not limited here.

[0129] Specifically, in the target image, if there is a line between the target text area corresponding to the first cell of the target cell group and the target text area corresponding to the second cell, then the target text area corresponding to the first cell and the target text area corresponding to the second cell cannot be recognized as two lines of target text areas due to line breaks, so they should not be merged.

[0130] For example, see the attached Figure 9 , Figure 9 A schematic diagram of another type of merged cells is shown. The first preset condition is set as follows: the sum of the first number of target text areas corresponding to the cells in the m2-th row where the second cell in the target cell group is located in the initial table is less than or equal to 1. After text detection processing is performed on the target image 901, the target text areas A to I can be obtained, and the initial table 902 of the target image 901 can be obtained through steps 701 to 702, wherein the target text areas corresponding to the cells are indicated in the initial table 902. Among the 6 target cell groups in the initial table 902, the target cell groups that meet the first preset condition include a target cell group consisting of a target text area G and a target text area H, and a target cell group consisting of a target text area C and a target text area I.

[0131] Then, line detection processing is performed on target image 901 to obtain line 904. By comparing the positions of target text areas G and H with line 904, it can be determined that line 904 is located above target text areas G and H. At the same time, by comparing the positions of target text areas C and I with line 904, it can be determined that line 904 is located between target text areas C and I. Therefore, only the cells corresponding to target text area G and the cells corresponding to target text area H can be merged. Finally, target table 902 can be obtained, and the text data in the target text area is filled into the corresponding cells to obtain table data 903.

[0132] Optionally, if the sum of the first numbers of target text areas corresponding to the cells in the m2th row meets the first preset condition, the m2th row and the m1th row of the initial table are merged to obtain the target table. The method can also be: if the sum of the first numbers of target text areas corresponding to the cells in the m2th row meets the first preset condition, the first data length is greater than the second data length, and there is no line in the target image between the target text area corresponding to the first cell and the target text area corresponding to the second cell, then the m2th row and the m1th row of the initial table are merged to obtain the target table.

[0133] In an embodiment of the present application, based on the correspondence between each target text area and each cell in the initial table, the text data in the target text area corresponding to each cell in the target table is filled into the target table. The method of obtaining the table data can be referred to the specific implementation method in step S204 and will not be repeated here.

[0134] In an embodiment of the present application, the position information of each target text area in the target image is first obtained, and then the initial table of the target image is determined based on the position information of each target text area; then the correspondence between each target text area of ​​the target image and each cell in the initial table is obtained, and then a target cell group is determined in the initial table, wherein the target cell group includes a first cell and a second cell, the first cell and the second cell are located in the same column, the first cell is located in the m1th row in the initial table, and the second cell is located in the m2th row in the initial table; then it is determined whether the sum of the first number of target text areas corresponding to each cell in the m2th row meets the first preset condition, and if so, the m2th row and the m1th row of the initial table are merged to obtain the target table; finally, based on the correspondence between each target text area and each cell in the initial table, the text data in the target text area corresponding to each cell in the target table is filled into the target table to obtain table data. In the embodiment of the present application, by judging whether the horizontal and vertical coordinates of the center point of each target text area are within the horizontal and vertical coordinate range of the area border of the previous target text area, it is possible to effectively avoid the situation where the coordinate information of the target text box area located in the same column or row in the detected target image is erroneous due to the tilt of the target image, and is assigned to different rows and columns, thereby causing a large error in the number of rows and columns of the initial table finally constructed. Then, by judging whether the sum of the first quantities meets the first preset condition, the cells that should be merged in the initial table can be determined, thereby obtaining the target table; therefore, this method avoids the situation where the text data of the line breaks is filled into different cells due to the lack of clear table lines, resulting in confusion of text data in the extracted table, thereby achieving the goal of not relying on tables with clear table lines such as full-line tables to determine which cells in the table certain target text areas are located in, thereby achieving table structuring of images containing non-full-line tables, and extracting table data of images containing non-full-line tables.

[0135] Based on the above table information extraction solution and table information extraction system, the present application embodiment provides another table information extraction method. Figure 10 , which is a flow chart of another table information extraction method provided in an embodiment of the present application. Figure 10 The table information extraction method shown can be used by Figure 1 The server and terminal devices shown are executed. Figure 10 The table information extraction method shown may include the following steps:

[0136] S1001: The terminal device sends the target image to the server.

[0137] In the embodiment of the present application, the terminal device may send the target image to the server through wireless communication or wired communication, or after encryption, or through other means, which are not limited here.

[0138] S1002: The server performs text detection on the target image to obtain at least one initial text area.

[0139] The specific implementation of performing text detection processing on the target image to obtain at least one initial text area can refer to the specific implementation of performing text detection processing on the target image to obtain various target text areas in step S201, which will not be repeated here.

[0140] S1003: The server obtains the segmentation positions of each initial text area.

[0141] In an embodiment of the present application, the method of obtaining the segmentation position of each initial text area may be: first perform character detection processing on each initial text area, determine the distance between adjacent characters in each initial text area, and if the distance between two adjacent characters is greater than a preset distance, determine the position between the two adjacent characters as the segmentation position. Optionally, the preset distance may be pixel points, or may be a unit length such as millimeters or centimeters, which is not limited here. Optionally, the method of performing character detection processing on each initial text area may be to detect the target image through optical character recognition technology to obtain each character; or may be to detect the target image through deep learning models such as DBNet, CTPN, SegLink, etc., to obtain each character, which is not limited here. Among them, the process of training the text detection model is a technical means commonly used by those skilled in the art, and will not be repeated here.

[0142] In one embodiment, the segmentation positions of each initial text region may be obtained by:

[0143] 1) After detecting each initial text region, extracting the text region image corresponding to each initial text region;

[0144] 2) performing binarization processing on each text region image to obtain a processed text region image. For example, the pixels of the pixels that belong to the characters in each text region image can be set to 255, and the pixels that do not belong to the characters can be set to 0;

[0145] 3) Accumulating the pixel values ​​of the pixels in the same column in each processed text region image to obtain a pixel distribution histogram; wherein the abscissa may be the pixel index of the processed text region image, and the ordinate may be the pixel accumulated value of the pixel in a certain column;

[0146] 4) If the cumulative pixel values ​​of n consecutive columns in the pixel distribution histogram are less than a preset pixel value, the positions of the n consecutive columns in the text region image are determined as segmentation positions. Here, n is a positive integer greater than a second preset value. Both n and the second preset value can be manually set or system-set, such as 10, 20, 100, etc., and are not limited here.

[0147] For example, see the attached Figure 11 , Figure 11 A pixel distribution histogram is shown. The horizontal axis is the pixel index, which is used to indicate the pixels of the 1st to 20th columns of the processed text area image; the vertical axis can be the accumulated value of the pixels in each column. Set the second preset value to 20 and n to 3, then Figure 11 The positions of the pixels in the 7th to 11th columns in the processed text region image can be determined as the segmentation positions.

[0148] S1004: The server segments each initial text region based on the segmentation position to obtain a first initial text subregion and a second initial text subregion, wherein the horizontal coordinate of the first initial text subregion in the target image is smaller than the horizontal coordinate of the second initial text subregion in the target image.

[0149] S1005 : The server performs character recognition processing on the first initial text sub-region to determine a first character in the first initial text sub-region, and performs character recognition processing on the second initial text sub-region to determine a second character in the second initial text sub-region.

[0150] In an embodiment of the present application, the horizontal coordinate of the first initial text sub-region in the target image is smaller than the horizontal coordinate of the second initial text sub-region in the target image, which means that: in the target image, the first initial text sub-region is located to the left of the second initial text sub-region. Specifically, because the common writing habit is to write from left to right, it is determined that the horizontal coordinate of the first initial text sub-region in the target image is smaller than the horizontal coordinate of the second initial text sub-region in the target image; if the writing habit changes to writing from right to left, it is determined that the horizontal coordinate of the first initial text sub-region in the target image is greater than the horizontal coordinate of the second initial text sub-region in the target image. Optionally, the positional relationship between the first initial text sub-region and the second initial text sub-region can change accordingly with the change of writing habits, which is not limited here.

[0151] In the embodiment of the present application, the first character is the last character of the text data of the first initial text sub-region, and the second character is the first character of the text data of the second initial text sub-region. The specific implementation of the character recognition process can be referred to the specific implementation of the character recognition process in step S1003, and will not be repeated here.

[0152] S1006. If the character type of the first character is not the preset type and the character type of the second character is not the preset type, the server determines that the first initial text sub-region and the second initial text sub-region are different target text regions.

[0153] In the embodiments of the present application, the preset type may be Chinese symbol characters and English symbol characters. Specifically, general insurance policies, consumption details and other bills are usually printed. During the font printing process, English numeral characters generally occupy one character width, Chinese characters and Chinese characters occupy two character widths. For symbols commonly used in bills such as colons, dashes, dots, etc., since the actual pixels occupied by the symbols are relatively few, Chinese symbol characters such as ":" and ";" and English symbol characters such as "." and "!" often have large blank areas before and after, which are likely to be determined as splitting positions in step S1003. Therefore, after obtaining the splitting positions, the first character in the first initial text sub-region and the second character in the second initial text sub-region can be recognized. Then, by determining whether the character types of the first character and the second character are not the preset type, it can be determined whether the initial text region needs to be split according to the splitting positions. After removing the incorrect splitting positions, the position information such as coordinates and text data of the target text region obtained by splitting using the correct splitting positions is beneficial for subsequent tabular extraction of the target image.

[0154] For example, please refer to the appendix Figure 12 , Figure 12 which shows a schematic diagram of text adhesion splitting. After performing image recognition on the target image 1201, 4 initial text regions can be obtained. Then, the splitting position of the initial text region 1202 can be obtained as position A, and the splitting position of the initial text region 1203 is position B. By splitting the initial text region 1202 at position A and performing character recognition processing, it can be determined that the first character of the first initial text sub-region of the initial text region 1202 is ":", and the second character of the second initial text sub-region is "male". Among them, the first character ":" is of the preset type. Therefore, the initial text region 1202 does not need to be split, and the initial text region 1202 is the target text region.

[0155] By splitting the initial text region 1203 at position B and performing character recognition processing, it can be determined that the first character of the first initial text sub-region of the initial text region 1203 is "yuan", and the second character of the second initial text sub-region is "6", both of which do not belong to the preset type; therefore, the initial text region 1203 needs to be split at position B, and the final target text region is as shown in the target image 1204.

[0156] S1007: The server obtains the initial table of the target image, and the correspondence between each target text area of ​​the target image and each cell in the initial table.

[0157] S1008: The server determines a target cell group in the initial table, where the target cell group includes a first cell and a second cell, where the first cell is located in row m1 of the initial table and the second cell is located in row m2 of the initial table. m1 and m2 are both positive integers, and m1 is smaller than m2.

[0158] S1009: If the sum of the first number of target text areas corresponding to the cells in the m2th row meets a first preset condition, the server merges the m2th row and the m1th row of the initial table to obtain a target table.

[0159] S1010 , based on the correspondence between each target text area and each cell in the initial table, the server fills the text data in the target text area corresponding to each cell in the target table into the target table to obtain table data.

[0160] The specific implementation of steps S1007 to S1010 may refer to the specific implementation of steps S201 to S204 and will not be described in detail here.

[0161] S1011, the server sends the form data to the terminal device.

[0162] In the embodiment of the present application, in addition to the table data, prompt information including successful table data extraction, failed table data extraction, etc. can also be sent to the terminal device, which is not limited here. Among them, the method of sending the table data to the terminal device in step S1010 is a technical means commonly used by those skilled in the art and is not repeated here.

[0163] In an embodiment of the present application, after obtaining the segmentation position, each initial text area will be segmented based on the segmentation position to obtain a first initial text sub-area and a second initial text sub-area, and then character recognition processing will be performed on the first initial text sub-area and the second initial text sub-area to determine the first character and the second character. Finally, by judging whether the character type of the first character or the second character is not a preset type, it is determined whether the initial text area needs to be segmented into the first initial text sub-area and the second initial text sub-area. Since characters that are close to each other in the target image are prone to different texts being recognized as the same text area during the text detection process, and the recognition error of the text area is likely to lead to subsequent initial table construction errors and cell merging errors, therefore, by correctly segmenting the initial text area, the accuracy of the table structuring can be effectively improved. In addition, by judging whether the sum of the first quantity meets the first preset condition, the cells that should be merged in the initial table are determined, thereby obtaining the target table. This can avoid the situation where the text data with line breaks is filled into different cells due to the lack of clear table lines, resulting in confusion in the text data in the extracted table. This can achieve the goal of not relying on tables with clear table lines such as full-line tables to determine which cells in the table certain target text areas are located in, thereby achieving table structuring of images containing non-full-line tables and extracting table data of images containing non-full-line tables.

[0164] Based on the above table information extraction solution and table information extraction system, the present application embodiment provides another table information extraction method. Figure 13 , which is a flow chart of another table information extraction method provided in an embodiment of the present application. Figure 13 The table information extraction method shown can be used by Figure 1 The server or terminal device shown is executed. Figure 13 The table information extraction method shown may include the following steps:

[0165] S1301: Obtain an initial table of a target image and a correspondence between each target text area of ​​the target image and each cell in the initial table, wherein each target text area is obtained by performing text detection processing on the target image.

[0166] In an embodiment of the present application, the method for obtaining the initial table of the target image can be: first obtain the position information of each target text area in the target image, where the position information may include the horizontal coordinate and vertical coordinate of each target text area in the target image; and then determine the initial table of the target image based on the position information of each target text area.

[0167] Specifically, the method of determining the initial table of the target image based on the position information of each target text area can be: 1) arranging each target text area based on the position information of each target text area to obtain the arranged target text area; 2) determining the initial table based on the arranged target text area.

[0168] Optionally, the target text regions are arranged based on their position information, and a method for obtaining the arranged target text regions may be:

[0169] 1) Obtaining the position information of the center point of each target text region in the target image. The position information of the center point of each target text region in the target image refers to the horizontal and vertical coordinates of the center point of each target text region in the target image.

[0170] 2) Based on the position information of the area center points of each target text area, the target text areas are sorted to obtain sorted target text areas, wherein the abscissa of the area center point of the xth target text area in the sorted target text areas is less than or equal to the abscissa of the area center point of the x+1th target text area, where x is a positive integer and is less than the number of the sorted target text areas.

[0171] Specifically, after obtaining the position information of the area center point of each target text area, each target text area is first sorted in ascending order according to the horizontal coordinates of the area center point of each target text area; if the horizontal coordinates of the area center points of multiple target text areas are the same, or the horizontal coordinates of the area center points of multiple target text areas are very small, then each target text area in the multiple target text areas is sorted in ascending order according to the vertical coordinates of the area center point of each target text area, and finally the sorted target text areas are obtained.

[0172] 3) obtaining position information of each region border of the first target text region in the sorted target text regions in the target image;

[0173] 4) Arranging the sorted target text regions based on the horizontal coordinates of the third region border and the fourth region border of the first target text region and the horizontal coordinates of the region center points of each target text region subsequent to the first target text region to obtain initially arranged target text regions;

[0174] In one embodiment, the method for arranging the sorted target text areas based on the abscissas of the third and fourth area borders of the first target text area and the abscissas of the area center points of each target text area subsequent to the first target text area to obtain the initially arranged target text areas may also be:

[0175] ① Traversing each target text area in the sorted target text area, determining a target text area whose region center point has a horizontal coordinate greater than the horizontal coordinate of the third region border of the first target text area and smaller than the horizontal coordinate of the second region border of the first target text area, and the determined target text area and the first target text area are located in the same column in the target text area after the initial arrangement;

[0176] ② If the abscissa of the area center point of the next target text area determined in the sorted target text areas is greater than the abscissa of the third area border of the first target text area, and greater than the abscissa of the fourth area border of the first target text area, then the next target text area is determined to be located in the next column of the first target text area in the target text areas after the initial arrangement;

[0177] ③ Take the next target text area as the first target text area, and trigger the execution of the target text area whose horizontal coordinate of the center point of the area after determination is greater than the horizontal coordinate of the third area border of the first target text area, and smaller than the horizontal coordinate of the second area border of the first target text area, so as to obtain the target text area after initial arrangement.

[0178] The order of the target text areas located in the same column in the target text areas after the initial arrangement is the same as the order of the target text areas located in the same column in the target text areas after the sorting.

[0179] 5) Based on the vertical coordinates of the first area border and the second area border of the first target text area after the initial arrangement, and the vertical coordinates of the area center points of other target text areas in the target text area after the initial arrangement, the target text areas after the initial arrangement are rearranged to obtain arranged target text areas.

[0180] In one embodiment, the target text areas after the initial arrangement are rearranged based on the vertical coordinates of the first area border and the second area border of the first target text area after the initial arrangement, and the vertical coordinates of the area center points of the other target text areas after the initial arrangement, to obtain the arranged target text areas in a manner that can be:

[0181] ① Determining, within the initially sorted target text areas, target text areas located in the same row after the arrangement, where the ordinate of the region center point of the target text area in the same row is greater than the ordinate of the first region border of the target text area located in the first column of the target text area in the same row, and is less than the ordinate of the second region border of the target text area located in the first column of the target text area in the same row;

[0182] ② Based on the vertical coordinate of the center point of each target text area, determine the mth position of each target text area in the arranged target text area. i Among them, if each target text area is located in the nth target text area after the initial arrangement i Column, then each target text area is located in the nth i List.

[0183] In one embodiment, the target text areas after the initial arrangement are rearranged based on the vertical coordinates of the first area border and the second area border of the first target text area after the initial arrangement, and the vertical coordinates of the area center points of other target text areas after the initial arrangement, to obtain the arranged target text areas. Alternatively, the method of rearranging the target text areas after the initial arrangement can be:

[0184] ① Traversing the target text areas in the first column of the target text areas after the initial arrangement, determining a target text area whose abscissa of the center point of the area in the target text areas after the initial arrangement is greater than the ordinate of the first area border of the target text areas in the first column and less than the ordinate of the second area border of the target text areas in the first column, and the determined target text area and the target text area in the first column are located in the same row in the target text areas after the arrangement;

[0185] ② If there are target text areas that are not in the same row as the target text areas in the first column, then determine the target text areas in the column closest to the first column among the target text areas that are not in the same row as the target text areas in the first column, and then use the target text areas that are not in the same row as the target text areas in the first column as the target text areas after initial arrangement, and use the remaining target text areas in the closest row as the target text areas in the first row after initial sorting, triggering the execution of traversal of the target text areas in the first column in the target text areas after initial arrangement, and determining the target text areas in the target text areas after initial arrangement whose horizontal coordinates of the center points of the areas are greater than the vertical coordinates of the first area borders of the target text areas in the first column and smaller than the vertical coordinates of the second area borders of the target text areas in the first column, and the determined target text areas are located in the same row as the target text areas in the first column in the arranged target text areas.

[0186] ③ Based on the vertical coordinate of the center point of each line of target text area, determine the mth position of each line of target text area in the arranged target text area. i rows to obtain the arranged target text area.

[0187] In one embodiment, the method for obtaining each target text area of ​​the target image can be: 1) performing text detection processing on the target image to obtain at least one initial text area; 2) obtaining the segmentation position of each initial text area; 3) performing segmentation processing on each initial text area based on the segmentation position to obtain a third initial text sub-area and a fourth initial text sub-area, and the vertical coordinate of the third initial text sub-area in the target image is smaller than the vertical coordinate of the fourth initial text sub-area in the target image; 4) performing character recognition processing on the third initial text sub-area to determine the third character in the third initial text sub-area, and performing character recognition processing on the fourth initial text sub-area to determine the fourth character in the fourth initial text sub-area, the third character is the last character of the text data of the third initial text sub-area, and the fourth character is the first character of the text data of the fourth initial text sub-area; 5) if the character type of the third character is not a preset type, and the character type of the third character is not a preset type, then determine that the third initial text sub-area and the fourth initial text sub-area are different target text areas.

[0188] The specific implementation of step S1301 is similar to the specific implementation of step S201, step 701 and steps S1001 to S1006. Please refer to the specific implementation of step S201 and step 701, which will not be repeated here.

[0189] S1302 : Determine a target cell group in the initial table, the target cell group including a first cell and a second cell, the first cell is located in the n1th column of the initial table, and the second cell is located in the n2th column of the initial table.

[0190] In an embodiment of the present application, the first cell and the second cell are located in the same row, and both the first cell and the second cell have corresponding target text areas. n1 and n2 are both positive integers, and n1 is less than n2. Specifically, the first cell and the second cell of the target cell group can be two cells located in the same row in two adjacent columns in the initial table; the first cell and the second cell of the target cell group can also be two non-adjacent columns in the initial table, but the third cell located between the first cell and the second cell does not have a corresponding target text area; wherein the third cell refers to a cell located in the same row as the first cell and in a column between columns n1 and n2 in the initial table.

[0191] S1303: If the third sum of the target text areas corresponding to the cells in the n2th column satisfies the second preset condition, the n2th column and the n1th column of the initial table are merged to obtain a target table. The target text areas corresponding to the cells in the n1th column of the target table include the target text areas corresponding to the corresponding cells in the n1th column of the initial table and the target text areas corresponding to the corresponding cells in the n2th column.

[0192] S1304 , based on the correspondence between each target text area and each cell in the initial table, fill the target table with the text data in the target text area corresponding to each cell in the target table to obtain table data.

[0193] In an embodiment of the present application, the second preset condition may be that the sum of the third quantities is less than the sum of the fourth quantities of the target text areas corresponding to the cells in the n1th column. Then, a method for determining whether the sum of the third quantities satisfies the second preset condition may be to obtain the sum of the fourth quantities of the target text areas corresponding to the cells in the n1th column; if the sum of the fourth quantities is greater than the sum of the third quantities, then determining that the sum of the third quantities satisfies the second preset condition.

[0194] For example, see the attached Figure 14 , Figure 14 The initial table of the target image 1401 and the target text area corresponding to each cell in the initial table are shown in FIG. Figure 14As shown in the initial table 1402 in the target image 1401, since "2246567" and "transfer student this semester" which should belong to the same cell in the target image 1401 are both wrapped, "2246567" will be recognized as "2246" and "567", and "transfer student this semester" will be recognized as "this semester" and "transfer student". Therefore, there are a total of 5 columns of cells in the obtained initial table 1402. Through the positional relationship between the cells in the initial table 1402 and the corresponding relationship between the cells and the target text area, it can be determined as follows: Figure 14 Among the 11 target cell groups shown, the one on the left in each target cell group is the first cell, and the one on the right is the second cell. Among them, there are a total of 3 cells in the 4th column of the initial table 1402 where the cell 1404 in the target cell group 3 is located, among which the target text area corresponding to the first cell in the 4th column is "567", the target text area corresponding to the third cell in the 4th column is "transfer student", and the second cell in the 4th column has no corresponding target text area. Therefore, it can be determined that the third quantity sum of the target text areas corresponding to each cell in the 4th column where the cell 1404 in the target cell group 3 is located is 2, and there are a total of 3 cells in the 3rd column of the initial table 1402 where the cell 1403 is located, among which the target text areas corresponding to the three cells are "2246", "Xiaobai" and "this semester" respectively. Therefore, it can be determined that the fourth quantity sum of the target text areas corresponding to each cell in the 3rd column where the cell 1403 in the target cell group 3 is located is 3. Therefore, it can be determined that the fourth quantity sum is greater than the third quantity sum, so it is determined to merge the 4th column of the initial table 1402 into the 3rd column, and finally obtain the target table 1407, among which, as shown in FIG. Figure 14 The target table 1407 shown indicates the target text area corresponding to each cell.

[0195] In one embodiment, the second preset condition may be: the sum of the first quantities is less than or equal to a third preset value. Optionally, the third preset value may be less than or equal to the sum of the fourth quantities of the target text areas corresponding to the cells in the n1th column. The third preset value may be manually set or system set, which is not limited here. For example, the third preset value may be 1, 5, 8, etc. Optionally, the method for determining whether the sum of the third quantities meets the second preset condition may be: if the sum of the second quantities is less than or equal to the third preset value, then determining that the sum of the third quantities meets the second preset condition.

[0196] In one embodiment, if the sum of the third quantities of the target text areas corresponding to the cells in the n2th column meets the second preset condition, the n2th column and the n1th column of the initial table are merged to obtain the target table. The method can be: determining the first data length of the text data of the target text area corresponding to the first cell, and the second data length of the text data of the target text area corresponding to the second cell; if the sum of the third quantities of the target text areas corresponding to the cells in the n2th column meets the second preset condition, and the first data length is greater than the second data length, then the n2th column and the n1th column of the initial table are merged to obtain the target table.

[0197] In one embodiment, if the third sum of the target text areas corresponding to the cells in the n2th column satisfies the second preset condition, then the n2th column of the initial table is merged with the n1th column to obtain the target table by performing line detection processing on the target image to obtain at least one line; if the third sum of the target text areas corresponding to the cells in the n2th column satisfies the second preset condition, and there is no line in the target image between the target text area corresponding to the first cell and the target text area corresponding to the second cell, then the n2th column of the initial table is merged with the n1th column to obtain the target table. The at least one line can be a vertical line or a diagonal line, which is not limited here.

[0198] In one embodiment, the method for determining the target cell group in the initial table can be: 1) determining at least one cell group in the initial table, each cell group including a first cell and a second cell; 2) obtaining the target text area corresponding to the first cell in each cell group and the distance between the target text area corresponding to the second cell in each cell group in the target image; 3) determining the cell group with the smallest distance as the target cell group.

[0199] In one embodiment, if the sum of the third quantities of target text areas corresponding to the cells in the n2th column meets the second preset condition, the n2th column and the n1th column of the initial table are merged to obtain the target table. The method can also be: if the sum of the third quantities of target text areas corresponding to the cells in the n2th column meets the second preset condition, the n2th column and the n1th column of the initial table are merged to obtain an updated table; the updated table is used as the initial table, and the determination of the target cell group in the initial table is triggered until the sum of the third quantities of target text areas corresponding to the cells in the n2th column does not meet the second preset condition, then the initial table to which the cells corresponding to the target text areas that do not meet the second preset condition belong is determined as the target table, and the target table includes p×q cells, p is less than or equal to m, and p and q are positive integers.

[0200] Among them, the specific implementation of steps S1302-step S1504 is the same as or similar to the specific implementation of steps S202-204, steps 702-706 and steps S1002-S1010. Please refer to the specific implementation of steps S202-204, steps 702-706 and steps S1002-S1010, which will not be repeated here.

[0201] In an embodiment of the present application, the position information of each target text area in the target image is first obtained, and then the initial table of the target image is determined based on the position information of each target text area; then the correspondence between each target text area of ​​the target image and each cell in the initial table is obtained, and then a target cell group is determined in the initial table, wherein the target cell group includes a first cell and a second cell, the first cell and the second cell are located in the same row, the first cell is located in the n1th row in the initial table, and the second cell is located in the n2th column in the initial table; then it is determined whether the sum of the third quantities of target text areas corresponding to each cell in the n2th column meets the second preset condition, and if so, the n2th row and the n1th column of the initial table are merged to obtain the target table; finally, based on the correspondence between each target text area and each cell in the initial table, the text data in the target text area corresponding to each cell in the target table is filled into the target table to obtain table data. In an embodiment of the present application, by judging whether the sum of the third quantity meets the second preset condition, the cells that should be merged in the initial table can be determined, thereby obtaining the target table; therefore, this method avoids the situation where the text data of the column change is filled into different cells due to the lack of clear table lines, resulting in confusion of text data in the extracted table, thereby achieving no reliance on tables with clear table lines such as full-line tables to determine in which cells of the table certain target text areas are located, thereby achieving table structuring of images containing non-full-line tables, and extracting table data of images containing non-full-line tables.

[0202] Based on the above table information extraction solution and table information extraction system, the present application embodiment provides another table information extraction method. Figure 15 , which is a flow chart of another table information extraction method provided in an embodiment of the present application. Figure 15 The table information extraction method shown can be used by Figure 1 The server or terminal device shown is executed. Figure 15 The table information extraction method shown may include the following steps:

[0203] S1501, obtaining an initial table of a target image, a correspondence between each target text area of ​​the target image and each cell in the initial table, and a display mode of text data in each target text area, wherein each target text area is obtained by performing text detection processing on the target image.

[0204] S1502, based on the display method of text data in each target text area, determine the target cell group in the initial table, the target cell group includes a first cell and a second cell, the position relationship and display method of the first cell and the second cell in the initial table match, wherein the second cell is located next to the first cell in the initial table.

[0205] In the embodiment of the present application, the position relationship and display mode matching of the first cell and the second cell in the initial table refers to:

[0206] 1) When the text data in each target text area is displayed in landscape orientation, the first cell and the second cell are located in the same column, the first cell is located in row m1 of the initial table, and the second cell is located in row m2 of the initial table. The second cell is located next to the first cell in the initial table if m1 is less than m2, the first cell and the second cell both have corresponding target text areas, and m1 and m2 are both positive integers;

[0207] 2) When the text data in each target text area is displayed vertically, the first cell and the second cell are located in the same row, the first cell is located in the n1th column of the initial table, and the second cell is located in the n2th column of the initial table. The second cell is located in the cell after the first cell in the initial table if n1 is less than n2, the first cell and the second cell both have corresponding target text areas, and n1 and n2 are both positive integers.

[0208] S1503, determining the display mode, if the display mode is vertical display, executing step S1504; if the display mode is horizontal display, executing step S1505;

[0209] S1504: If the sum of the third number of target text areas corresponding to the cells in the column where the second cell is located meets the second preset condition, the column where the second cell is located is merged with the column where the first cell is located to obtain a target table;

[0210] S1505: If the sum of the first number of target text areas corresponding to the cells in the row where the second cell is located meets a first preset condition, the row where the second cell is located is merged with the row where the first cell is located to obtain a target table.

[0211] S1506 , based on the correspondence between each target text area and each cell in the initial table, fill the target table with text data in the target text area corresponding to each cell in the target table to obtain table data.

[0212] Among them, the specific implementation of steps S1501-S1506 is the same as or similar to the specific implementation of steps S201-204, steps 701-706 and steps S1002-S1010. Please refer to the specific implementation of steps S201-204, steps 701-706 and steps S1002-S1010, which will not be repeated here.

[0213] In an embodiment of the present application, an initial table of the target image, a correspondence between each target text area of ​​the target image and each cell in the initial table, and a display mode of the text data in each target text area are first obtained, and then a target cell group is determined in the initial table; if the display mode is vertical display, and the third quantity sum of the target text areas corresponding to each cell included in the column where the second cell is located meets the second preset condition, then the column where the second cell is located is merged with the column where the first cell is located to obtain a target table; if the display mode is horizontal display, and the first quantity sum of the target text areas corresponding to each cell included in the row where the second cell is located meets the first preset condition, then the row where the second cell is located is merged with the row where the first cell is located to obtain a target table; finally, based on the correspondence between each target text area and each cell in the initial table, the text data in the target text area corresponding to each cell in the target table is filled into the target table to obtain table data. In an embodiment of the present application, by obtaining the display mode of the text data in each target text area, it can be determined whether the cells in the initial table should be merged in columns or rows, and then the target cell group containing the first cell and the second cell with different positional relationships can be determined according to different display modes, and the cells that need to be merged can be determined according to the first preset condition or the second preset condition, thereby obtaining a target table, wherein the target text areas in the two cells in the target table that should originally belong to the same cell but are corresponding to different rows or columns are merged. Therefore, this method avoids the situation where the text data that wraps lines or columns is filled into different cells due to the lack of clear table lines, resulting in confusion in the text data in the extracted table, thereby achieving the goal of not relying on tables with clear table lines such as full-line tables to determine which cells in the table certain target text areas are located, thereby achieving table structuring of images containing non-full-line tables and extracting table data of images containing non-full-line tables.

[0214] Based on the above table information extraction method embodiment, the present application embodiment provides a table information extraction device. Figure 16, is a structural diagram of a table information extraction device provided in an embodiment of the present application. The table information extraction device may include an acquisition unit 1601, a determination unit 1602, a merging unit 1603 and a filling unit 1604. Figure 11 The table information extraction device shown can run the following units:

[0215] The acquisition unit 1601 is configured to acquire an initial table of a target image and a correspondence between each target text region of the target image and each cell in the initial table, wherein each target text region is obtained by performing text detection processing on the target image;

[0216] The determining unit 1602 is configured to determine a target cell group in the initial table, where the cell group includes a first cell and a second cell, the first cell and the second cell are located in the same column, the first cell is located in the m1th row of the initial table, the second cell is located in the m2th row of the initial table, m1 and m2 are both positive integers, m1 is less than m2, and the first cell and the second cell both have corresponding target text areas;

[0217] The merging unit 1603 is configured to merge the m2th row of the initial table with the m1th row of the initial table to obtain a target table if the sum of the first number of target text areas corresponding to the individual cells in the m2th row satisfies a first preset condition, wherein the target text areas corresponding to the individual cells in the m1th row of the target table include the target text areas corresponding to the corresponding cells in the m1th row of the initial table and the target text areas corresponding to the corresponding cells in the m2th row of the initial table;

[0218] The filling unit 1604 is configured to fill the target table with text data in the target text area corresponding to each cell in the target table based on the correspondence between each target text area and each cell in the initial table, thereby obtaining table data.

[0219] In one embodiment, the merging unit 1603 is also used to determine the first data length of the text data of the target text area corresponding to the first cell, and the second data length of the text data of the target text area corresponding to the second cell; if the sum of the first quantities of the target text areas corresponding to the cells in the m2th row meets the first preset condition, and the first data length is greater than the second data length, then the m2th row and the m1th row of the initial table are merged to obtain the target table.

[0220] In one embodiment, the merging unit 1603 is further used to perform line detection processing on the target image to obtain at least one line; if the sum of the first number of target text areas corresponding to the cells in the m2th row meets the first preset condition, and there is no line in the target image located between the target text area corresponding to the first cell and the target text area corresponding to the second cell, then the m2th row and the m1th row of the initial table are merged to obtain the target table.

[0221] In one embodiment, the acquisition unit 1601 is further used to obtain the position information of each target text area in the target image; based on the position information of each target text area, an initial table of the target image is determined, the position information includes a horizontal coordinate and a vertical coordinate, and the initial table includes m×n cells, m is determined based on the position information of the target text area with the minimum vertical coordinate and the target text area with the maximum vertical coordinate in the target image, and n is determined based on the position information of the target text area with the minimum horizontal coordinate and the target text area with the maximum horizontal coordinate in the target image, and both m and n are positive integers.

[0222] In one embodiment, the acquisition unit 1601 is further used to arrange each target text area based on the position information of each target text area to obtain the arranged target text area; based on the arranged target text area, an initial table is determined, m is determined based on the target text area with the smallest vertical coordinate and the target text area with the largest vertical coordinate in the arranged target text area, and n is determined based on the target text area with the smallest horizontal coordinate and the target text area with the largest horizontal coordinate in the arranged target text area.

[0223] In one embodiment, the acquisition unit 1601 is further configured to acquire position information of area center points of each target text area in the target image; sort the target text areas based on the position information of the area center points of each target text area to obtain sorted target text areas, wherein the horizontal coordinate of the area center point of the xth target text area in the sorted target text areas is less than or equal to the horizontal coordinate of the area center point of the x+1th target text area, where x is a positive integer; acquire position information of each area border of the first target text area in the sorted target text areas in the target image; arrange the sorted target text areas based on the vertical coordinates of the first and second area borders of the first target text area and the vertical coordinates of the area center points of each target text area after the first target text area to obtain initially arranged target text areas; and rearrange the initially arranged target text areas based on the horizontal coordinates of the third and fourth area borders of the first target text area in the initially arranged target text areas and the horizontal coordinates of the area center points of other target text areas in the initially arranged target text areas to obtain arranged target text areas.

[0224] In one embodiment, the acquiring unit 1601 is further configured to traverse each target text region in the sorted target text region, determine a target text region in the sorted target text region whose region center point has a longitudinal coordinate greater than the longitudinal coordinate of the first region border of the first target text region and less than the longitudinal coordinate of the second region border of the first target text region, and the determined target text region and the first target text region are located in the same row in the target text region after the initial arrangement; if the longitudinal coordinate of the region center point of the next target text region determined in the sorted target text region is greater than the longitudinal coordinate of the first region border of the first target text region and greater than the longitudinal coordinate of the second region border of the first target text region, then determine that the next target text region is located in the next row of the first target text region in the target text region after the initial arrangement; use the next target text region as the first target text region, and trigger the execution of determining a target text region in the sorted target text region whose region center point has a longitudinal coordinate greater than the longitudinal coordinate of the first region border of the first target text region and less than the longitudinal coordinate of the second region border of the first target text region, so as to obtain the target text region after the initial arrangement; wherein the target text regions located in the same row in the target text region after the initial arrangement have the same ranking as the target text regions located in the same row in the sorted target text region.

[0225] In one embodiment, the acquisition unit 1601 is further configured to determine, in the target text areas after the initial arrangement, target text areas located in the same column in the arranged target text areas, wherein the horizontal coordinate of the area center point of the target text areas in the same column is greater than the horizontal coordinate of the third area border of the target text areas located in the first row in the target text areas in the same column, and is smaller than the horizontal coordinate of the fourth area border of the target text areas located in the first row in the target text areas in the same column; and based on the horizontal coordinate of the area center point of each column of the target text areas, determine that each column of the target text areas is located in the nth area of ​​the arranged target text areas. i Columns to obtain the arranged target text area; wherein, if each target text area is located in the mth target text area after the initial arrangement i row, then each target text area is located in the mth target text area after arrangement i OK, m i 、n i is a positive integer, m i Less than or equal to m, n i Less than or equal to n.

[0226] In one embodiment, the acquisition unit 1601 is further used to perform text detection processing on the target image to obtain at least one initial text area; obtain the segmentation position of each initial text area; perform segmentation processing on each initial text area based on the segmentation position to obtain a first initial text sub-area and a second initial text sub-area, and the horizontal coordinate of the first initial text sub-area in the target image is smaller than the horizontal coordinate of the second initial text sub-area in the target image; perform character recognition processing on the first initial text sub-area to determine the first character in the first initial text sub-area, and perform character recognition processing on the second initial text sub-area to determine the second character in the second initial text sub-area, the first character is the last character of the text data of the first initial text sub-area, and the second character is the first character of the text data of the second initial text sub-area; if the character type of the first character is not a preset type, and the character type of the second character is not a preset type, then it is determined that the first initial text sub-area and the second initial text sub-area are different target text areas.

[0227] In one embodiment, the determination unit 1602 is further used to determine at least one cell group in the initial table, each cell group includes a first cell and a second cell, the first cell and the second cell are located in the same column, the first cell is located in the m1th row in the initial table, the second cell is located in the m2th row in the initial table, m1 and m2 are both positive integers, m1 is less than m2, and the first cell and the second cell both have corresponding target text areas; for each cell group, obtain the target text area corresponding to the first cell in each cell group, and the distance between the target text area corresponding to the second cell in each cell group in the target image; determine the cell group with the smallest distance as the target cell group.

[0228] In one embodiment, the merging unit 1603 is further used to merge the m2th row and the m1th row of the initial table to obtain an updated table if the sum of the first number of target text areas corresponding to the cells in the m2th row meets the first preset condition; use the updated table as the initial table to trigger the execution of determining the target cell group in the initial table until the sum of the first number of target text areas corresponding to the cells in the m2th row does not meet the first preset condition, then determine the initial table to which the cells corresponding to the target text areas that do not meet the first preset condition belong as the target table, and the target table includes p×q cells, p is less than or equal to m, and p and q are positive integers.

[0229] In one embodiment, the acquisition unit 1601 is further configured to acquire a second total number of target text areas corresponding to cells in the m1-th row; if the second total number is greater than the first total number, it is determined that the first total number meets a first preset condition.

[0230] According to one embodiment of the present application, Figure 2 、 Figure 7 、 Figure 10 、 Figure 13 as well as Figure 15 The steps involved in the table information extraction method shown can be performed by Figure 16 The table information extraction device shown is executed by each unit.

[0231] According to another embodiment of the present application, Figure 16The various units in the table information extraction device shown can be separately or all merged into one or several other units to constitute, or one (or some) of the units can also be split into multiple smaller units in function to constitute, which can achieve the same operation without affecting the realization of the technical effects of the embodiments of the present application. The above-mentioned units are divided based on logical functions. In actual applications, the functions of one unit can also be implemented by multiple units, or the functions of multiple units can be implemented by one unit. In other embodiments of the present application, the table information extraction device divided based on logical functions can also include other units. In actual applications, these functions can also be implemented with the assistance of other units, and can be implemented by the collaboration of multiple units.

[0232] According to another embodiment of the present application, the program can be executed by running on a general computing device such as a computer including a central processing unit (CPU), a random access memory (RAM), a read-only memory (ROM) and other processing elements and storage elements. Figure 2 、 Figure 7 、 Figure 10 、 Figure 13 as well as Figure 16 A computer program (including program code) for each step of the corresponding method shown in FIG. Figure 16 The table information extraction device shown in and the table information extraction method of the embodiment of the present application are implemented. The computer program can be recorded on, for example, a computer readable storage medium, and loaded into the above-mentioned computing device through the computer readable storage medium and run therein.

[0233] In an embodiment of the present application, an initial table of the target image and the correspondence between each target text area of ​​the target image and each cell in the initial table are first obtained, and then a target cell group is determined in the initial table, wherein the target cell group includes a first cell and a second cell, the first cell and the second cell are located in the same column, the first cell is located in the m1th row in the initial table, and the second cell is located in the m2th row in the initial table; then, it is determined whether the sum of the first number of target text areas corresponding to each cell in the m2th row meets a first preset condition, and if so, the m2th row and the m1th row of the initial table are merged to obtain a target table; finally, based on the correspondence between each target text area and each cell in the initial table, the text data in the target text area corresponding to each cell in the target table is filled into the target table to obtain table data. In an embodiment of the present application, the initial position of each target text area in the initial table can be determined by obtaining the initial table and the correspondence between each target text area of ​​the target image and each cell in the initial table; then, by determining the target cell group in the initial table and judging whether the sum of the first quantity meets the first preset condition, the cells that should be merged in the initial table are determined, thereby obtaining a target table, wherein the target text areas in two cells in the target table that should originally belong to the same cell but are corresponding to different rows are merged. Therefore, this method avoids the situation where the text data of the line breaks is filled into different cells due to the lack of clear table lines, resulting in confusion of text data in the extracted table, thereby achieving the goal of not relying on tables with clear table lines such as full-line tables to determine which cells in the table certain target text areas are located, thereby achieving table structuring of images containing non-full-line tables and extracting table data of images containing non-full-line tables.

[0234] Based on the above method embodiment and device embodiment, the present application also provides an electronic device. Figure 17 , is a structural diagram of an electronic device provided in an embodiment of the present application. Figure 17 The electronic device shown may include at least a processor 1701, an input interface 1702, an output interface 1703, and a computer storage medium 1704. The processor 1701, the input interface 1702, the output interface 1703, and the computer storage medium 1704 may be connected via a bus or other means.

[0235] Computer storage medium 1704 may be stored in the memory of the electronic device. Computer storage medium 1704 is used to store a computer program, which includes program instructions. Processor 1701 is used to execute the program instructions stored in computer storage medium 1704. Processor 1701 (or CPU (Central Processing Unit)) is the computing core and control core of the electronic device. It is suitable for implementing one or more instructions, and is specifically suitable for loading and executing one or more instructions to implement the above-mentioned table information extraction method process or corresponding functions.

[0236] The embodiment of the present application also provides a computer storage medium (Memory), which is a memory device in an electronic device for storing programs and data. It is understandable that the computer storage medium here can include both the built-in storage medium in the terminal and, of course, the extended storage medium supported by the terminal. The computer storage medium provides a storage space that stores the operating system of the terminal. In addition, one or more instructions suitable for being loaded and executed by the processor 1701 are also stored in the storage space. These instructions can be one or more computer programs (including program codes). It should be noted that the computer storage medium here can be a high-speed random access memory (RAM) memory, or a non-volatile memory (non-volatile memory), such as at least one disk storage; optionally, it can also be at least one computer storage medium located away from the aforementioned processor.

[0237] In one embodiment, the processor 1701 may load and execute one or more instructions stored in a computer storage medium to implement the above-mentioned Figure 2 、 Figure 7 as well as Figure 10 In the corresponding steps of the method in the embodiment of the table information extraction method, in a specific implementation, one or more instructions in the computer storage medium are loaded by the processor 1701 and execute the following steps:

[0238] The processor 1701 obtains an initial table of the target image and a correspondence between each target text area of ​​the target image and each cell in the initial table, wherein each target text area is obtained by performing text detection processing on the target image;

[0239] Processor 1701 determines a target cell group in the initial table, where the target cell group includes a first cell and a second cell, the first cell and the second cell are located in the same column, the first cell is located in row m1 of the initial table, the second cell is located in row m2 of the initial table, m1 and m2 are both positive integers, m1 is less than m2, and the first cell and the second cell both have corresponding target text areas;

[0240] If the processor 1701 determines that the first total number of target text areas corresponding to the cells in the m2-th row meets the first preset condition, the processor 1701 merges the m2-th row and the m1-th row of the initial table to obtain a target table, where the target text areas corresponding to the cells in the m1-th row of the target table include the target text areas corresponding to the corresponding cells in the m1-th row of the initial table and the target text areas corresponding to the corresponding cells in the m2-th row.

[0241] Based on the correspondence between each target text area and each cell in the initial table, the processor 1701 fills the text data in the target text area corresponding to each cell in the target table into the target table to obtain table data.

[0242] In one embodiment, if the processor 1701 determines that the total first quantity of target text areas corresponding to each cell in the m2th row meets the first preset condition, the m2th row and the m1th row of the initial table are merged to obtain the target table, including: determining the first data length of the text data of the target text area corresponding to the first cell, and the second data length of the text data of the target text area corresponding to the second cell; if it is determined that the total first quantity of target text areas corresponding to each cell in the m2th row meets the first preset condition, and the first data length is greater than the second data length, the m2th row and the m1th row of the initial table are merged to obtain the target table.

[0243] In one embodiment, if the processor 1701 determines that the first total number of target text areas corresponding to the cells in the m2th row meets the first preset condition, the m2th row and the m1th row of the initial table are merged to obtain the target table, including: performing line detection processing on the target image to obtain at least one line; if the first total number of target text areas corresponding to the cells in the m2th row meets the first preset condition, and there is no line in the target image between the target text area corresponding to the first cell and the target text area corresponding to the second cell, then the m2th row and the m1th row of the initial table are merged to obtain the target table.

[0244] In one embodiment, the processor 1701 obtains an initial table of the target image, including: obtaining position information of each target text area in the target image; determining the initial table of the target image based on the position information of each target text area, the position information including horizontal coordinates and vertical coordinates, the initial table including m×n cells, m is determined based on the position information of the target text area with the minimum vertical coordinate and the target text area with the maximum vertical coordinate in the target image, and n is determined based on the position information of the target text area with the minimum horizontal coordinate and the target text area with the maximum horizontal coordinate in the target image, and both m and n are positive integers.

[0245] In one embodiment, the processor 1701 determines the initial table of the target image based on the position information of each target text area, including: arranging each target text area based on the position information of each target text area to obtain the arranged target text area; determining the initial table based on the arranged target text area, m is determined based on the target text area with the smallest vertical coordinate and the target text area with the largest vertical coordinate in the arranged target text area, and n is determined based on the target text area with the smallest horizontal coordinate and the target text area with the largest horizontal coordinate in the arranged target text area.

[0246] In one embodiment, the processor 1701 arranges the target text areas based on the position information of the target text areas to obtain arranged target text areas, including: obtaining position information of area center points of the target text areas in the target image; sorting the target text areas based on the position information of the area center points of the target text areas to obtain sorted target text areas, wherein the horizontal coordinate of the area center point of the xth target text area in the sorted target text areas is less than or equal to the horizontal coordinate of the area center point of the x+1th target text area, where x is a positive integer; obtaining position information of each area border of the first target text area in the sorted target text areas in the target image; arranging the target text areas in the sorted target text areas based on the vertical coordinates of the first and second area borders of the first target text area and the vertical coordinates of the area center points of each target text area after the first target text area to obtain initially arranged target text areas; and rearranging the initially arranged target text areas based on the horizontal coordinates of the third and fourth area borders of the first target text area in the initially arranged target text areas and the horizontal coordinates of the area center points of other target text areas in the initially arranged target text areas to obtain arranged target text areas.

[0247] In one embodiment, the processor 1701 arranges the target text areas after sorting based on the vertical coordinates of the first area border and the second area border of the first target text area, and the vertical coordinates of the area center points of each target text area after the first target text area, to obtain the target text areas after the initial arrangement, including: traversing each target text area in the sorted target text area, determining a target text area in which the vertical coordinate of the area center point in the sorted target text area is greater than the vertical coordinate of the first area border of the first target text area and less than the vertical coordinate of the second area border of the first target text area, and the determined target text area and the first target text area are located in the same row in the target text area after the initial arrangement; if the next target text area determined in the sorted target text area If the vertical coordinate of the center point of the text area is greater than the vertical coordinate of the first area border of the first target text area, and greater than the vertical coordinate of the second area border of the first target text area, then the next target text area is determined to be located in the next row of the first target text area in the target text area after the initial arrangement; the next target text area is taken as the first target text area, and the target text area whose vertical coordinate of the center point in the sorted target text area is greater than the vertical coordinate of the first area border of the first target text area, and smaller than the vertical coordinate of the second area border of the first target text area, is triggered to obtain the target text area after the initial arrangement; wherein, the order of the target text areas located in the same row in the target text area after the initial arrangement is the same as the order of the target text areas located in the same row in the sorted target text area.

[0248] In one embodiment, the processor 1701 rearranges the target text areas after the initial arrangement based on the horizontal coordinates of the third area border and the fourth area border of the first target text area after the initial arrangement, and the horizontal coordinates of the area center points of other target text areas in the target text area after the initial arrangement, to obtain the arranged target text areas, including: determining, in the target text areas after the initial arrangement, the target text areas located in the same column in the arranged target text areas, the horizontal coordinates of the area center points of the target text areas in the same column being greater than the horizontal coordinates of the third area border of the target text areas located in the first row in the target text areas in the same column, and being less than the horizontal coordinates of the fourth area border of the target text areas located in the first row in the target text areas in the same column; determining, based on the horizontal coordinates of the area center points of the target text areas in each column, that the target text areas in each column are located in the nth row in the arranged target text areas. i Columns to obtain the arranged target text area; wherein, if each target text area is located in the mth target text area after the initial arrangement irow, then each target text area is located in the mth target text area after arrangement i OK, m i 、n i is a positive integer, m i Less than or equal to m, n i Less than or equal to n.

[0249] In one embodiment, the processor 1701 is also used to perform text detection processing on the target image to obtain at least one initial text area; obtain the segmentation position of each initial text area; perform segmentation processing on each initial text area based on the segmentation position to obtain a first initial text sub-area and a second initial text sub-area, and the horizontal coordinate of the first initial text sub-area in the target image is smaller than the horizontal coordinate of the second initial text sub-area in the target image; perform character recognition processing on the first initial text sub-area to determine the first character in the first initial text sub-area, and perform character recognition processing on the second initial text sub-area to determine the second character in the second initial text sub-area, the first character is the last character of the text data of the first initial text sub-area, and the second character is the first character of the text data of the second initial text sub-area; if the character type of the first character is not a preset type, and the character type of the second character is not a preset type, then it is determined that the first initial text sub-area and the second initial text sub-area are different target text areas.

[0250] In one embodiment, the processor 1701 determines a target cell group in an initial table, including: determining at least one cell group in the initial table, each cell group including a first cell and a second cell, the first cell and the second cell being in the same column, the first cell being in the m1th row in the initial table, the second cell being in the m2th row in the initial table, m1 and m2 being positive integers, m1 being less than m2, and the first cell and the second cell both having corresponding target text areas; obtaining the target text area corresponding to the first cell in each cell group, and the distance between the target text area corresponding to the second cell in each cell group in the target image; and determining the cell group with the smallest distance as the target cell group.

[0251] In one embodiment, if the processor 1701 determines that the sum of the first quantities of target text areas corresponding to the cells in the m2th row meets the first preset condition, the m2th row and the m1th row of the initial table are merged to obtain a target table, including: if the sum of the first quantities of target text areas corresponding to the cells in the m2th row meets the first preset condition, the m2th row and the m1th row of the initial table are merged to obtain an updated table; the updated table is used as the initial table, and the target cell group is triggered to be determined in the initial table until the sum of the first quantities of target text areas corresponding to the cells in the m2th row does not meet the first preset condition, then the initial table to which the cells corresponding to the target text areas that do not meet the first preset condition belong is determined as the target table, and the target table includes p×q cells, p is less than or equal to m, and p and q are positive integers.

[0252] In one embodiment, the processor 1701 is further configured to obtain a second total number of target text areas corresponding to cells in the m1th row; if the second total number is greater than the first total number, it is determined that the first total number meets a first preset condition.

[0253] The embodiment of the present application provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the electronic device performs the above-mentioned Figure 2 、 Figure 7 as well as Figure 10 The computer-readable storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0254] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A table information extraction method, characterized in that: include: Acquire an initial table of a target image, and a correspondence between each target text area of ​​the target image and each cell in the initial table, wherein each target text area is obtained by performing text detection processing on the target image; determining at least one cell group in the initial table, each cell group including a first cell and a second cell; Obtaining a distance between a target text area corresponding to a first cell in each cell group and a target text area corresponding to a second cell in each cell group in the target image; The cell group with the smallest distance is determined as the target cell group, wherein the target cell group includes a first cell and a second cell, the first cell and the second cell are located in the same column, and the first cell is located in the first column of the initial table. row, the second cell is located in the first row of the initial table. OK, 、 are all positive integers, Less than , the first cell and the second cell both have corresponding target text areas; If the If the sum of the first number of target text areas corresponding to the cells in the row satisfies the first preset condition, the first Row and The rows are merged to obtain the target table, the first The target text area corresponding to each cell in the row includes the first The target text area corresponding to the corresponding cell in the row and the The target text area corresponding to the corresponding cell in the row; Based on the correspondence between each target text area and each cell in the initial table, the text data in the target text area corresponding to each cell in the target table is filled into the target table to obtain table data.

2. The method according to claim 1, characterized in that If the If the sum of the first number of target text areas corresponding to the cells in the row satisfies the first preset condition, the first Row and Merge the rows to get the target table, including: Determining a first data length of text data in a target text area corresponding to the first cell, and a second data length of text data in a target text area corresponding to the second cell; If the If the sum of the first number of target text areas corresponding to the cells in the row meets the first preset condition and the first data length is greater than the second data length, the first data length of the initial table is Row and The rows are merged to obtain the target table.

3. The method according to claim 1, characterized in that If the If the sum of the first number of target text areas corresponding to the cells in the row satisfies the first preset condition, the first Row and Merge the rows to get the target table, including: Performing line detection processing on the target image to obtain at least one line; If the The sum of the first number of target text areas corresponding to the cells in the row satisfies the first preset condition, and no line in the target image is located between the target text area corresponding to the first cell and the target text area corresponding to the second cell, then the first cell of the initial table is Row and The rows are merged to obtain the target table.

4. The method according to claim 1, wherein The initial table of the target image is obtained, including: Obtaining position information of each target text area in the target image; Based on the position information of each target text area, an initial table of the target image is determined, wherein the position information includes a horizontal coordinate and a vertical coordinate, and the initial table includes cells, m is determined based on the position information of the target text area with the minimum vertical coordinate and the target text area with the maximum vertical coordinate in the target image, and n is determined based on the position information of the target text area with the minimum horizontal coordinate and the target text area with the maximum horizontal coordinate in the target image, and both m and n are positive integers.

5. The method according to claim 4, characterized in that The determining of the initial form of the target image based on the position information of each target text area includes: Arranging the target text regions based on the position information of the target text regions to obtain arranged target text regions; Based on the arranged target text area, the initial table is determined, m is determined according to the target text area with the smallest vertical coordinate and the target text area with the largest vertical coordinate in the arranged target text area, and n is determined according to the target text area with the smallest horizontal coordinate and the target text area with the largest horizontal coordinate in the arranged target text area.

6. The method according to claim 5, characterized in that Arranging the target text regions based on the position information of the target text regions to obtain arranged target text regions includes: Obtaining position information of the center point of each target text area in the target image; sorting the target text areas based on the position information of the area center points of the target text areas to obtain sorted target text areas, wherein the abscissa of the area center point of an x-th target text area in the sorted target text areas is less than or equal to the abscissa of the area center point of an x+1-th target text area, where x is a positive integer; Obtaining position information of each region border of the first target text region in the sorted target text regions in the target image; Arranging the sorted target text regions based on the ordinates of the first region border and the second region border of the first target text region and the ordinates of the region center points of each target text region subsequent to the first target text region to obtain initially arranged target text regions; Based on the horizontal coordinates of the third area border and the fourth area border of the first target text area after the initial arrangement, and the horizontal coordinates of the area center points of other target text areas in the target text area after the initial arrangement, the target text areas after the initial arrangement are rearranged to obtain the arranged target text areas.

7. The method according to claim 6, characterized in that Arranging the sorted target text areas based on the ordinates of the first area border and the second area border of the first target text area and the ordinates of the area center points of each target text area after the first target text area to obtain the initially arranged target text areas includes: Traversing each target text area in the sorted target text areas, determining a target text area in which the vertical coordinate of the center point of the sorted target text area is greater than the vertical coordinate of the first area border of the first target text area and smaller than the vertical coordinate of the second area border of the first target text area, wherein the determined target text area and the first target text area are located in the same row in the target text area after the initial arrangement; If the ordinate of the area center point of the next target text area after the determined target text area in the sorted target text areas is greater than the ordinate of the first area border of the first target text area, and greater than the ordinate of the second area border of the first target text area, then it is determined that the next target text area is located in the next row of the first target text area in the target text areas after the initial arrangement; The next target text area is used as the first target text area, and the step of determining a target text area having a center point with a ordinate greater than the ordinate of a first area border of the first target text area and less than the ordinate of a second area border of the first target text area among the sorted target text areas is triggered, so as to obtain the target text areas after the initial arrangement; The order of the target text regions located in the same row in the target text regions after the initial arrangement is the same as the order of the target text regions located in the same row in the target text regions after the sorting.

8. The method according to claim 6, characterized in that The method of rearranging the initially arranged target text areas based on the abscissas of the third area border and the fourth area border of the first target text area after the initial arrangement and the abscissas of the area center points of the other target text areas after the initial arrangement to obtain the arranged target text areas includes: determining, from the target text areas after the initial arrangement, target text areas located in the same column of the arranged target text areas, wherein the abscissa of a region center point of the target text areas in the same column is greater than the abscissa of a third region border of the target text areas located in the first row of the target text areas in the same column, and is less than the abscissa of a fourth region border of the target text areas located in the first row of the target text areas in the same column; Based on the horizontal coordinates of the center points of the target text areas in each column, it is determined that the target text areas in each column are located in the first position in the arranged target text area. columns to obtain the arranged target text area; Wherein, if each target text area is located in the first target text area after the initial arrangement, row, then each target text area is located in the first row of the arranged target text area. OK, 、 is a positive integer, Less than or equal to m, Less than or equal to n.

9. The method according to claim 1, characterized in that The method further comprises: Performing text detection processing on the target image to obtain at least one initial text area; Get the segmentation position of each initial text area; Segmenting each of the initial text regions based on the segmentation positions to obtain a first initial text subregion and a second initial text subregion, wherein the horizontal coordinate of the first initial text subregion in the target image is smaller than the horizontal coordinate of the second initial text subregion in the target image; performing character recognition processing on the first initial text subregion to determine a first character in the first initial text subregion, and performing character recognition processing on the second initial text subregion to determine a second character in the second initial text subregion, where the first character is the last character of text data in the first initial text subregion and the second character is the first character of text data in the second initial text subregion; If the character type of the first character is not a preset type, and the character type of the second character is not a preset type, the first initial text sub-region and the second initial text sub-region are determined to be different target text regions.

10. The method according to any one of claims 1 to 9, characterized in that If the If the sum of the first number of target text areas corresponding to the cells in the row satisfies the first preset condition, the first Row and Merge the rows to get the target table, including: If the If the sum of the first number of target text areas corresponding to the cells in the row satisfies the first preset condition, the first Row and Merge the rows to get the updated table; The updated table is used as the initial table, triggering the execution of determining the target cell group in the initial table until the first If the sum of the first number of target text areas corresponding to the cells in the row does not meet the first preset condition, the initial table to which the cells corresponding to the target text areas that do not meet the first preset condition belong is determined as the target table, and the target table includes cells, p is less than or equal to m, 、 Is a positive integer.

11. The method according to any one of claims 1 to 9, characterized in that The method further comprises: Get the The sum of the second quantities of the target text areas corresponding to the cells in the row; If the second total is greater than the first total, it is determined that the first total satisfies a first preset condition.

12. An image-based table information extraction device, characterized in that: The image-based table information extraction device includes an acquisition unit, a determination unit, a merging unit, and a filling unit, wherein: The acquisition unit is configured to acquire an initial table of a target image, and a correspondence between each target text area of ​​the target image and each cell in the initial table, wherein each target text area is obtained by performing text detection processing on the target image; The determining unit is used to determine at least one cell group in the initial table, each cell group including a first cell and a second cell; obtain the distance between the target text area corresponding to the first cell in each cell group and the target text area corresponding to the second cell in each cell group in the target image; determine the cell group with the smallest distance as the target cell group, the cell group including the first cell and the second cell, the first cell and the second cell being located in the same column, and the first cell being located in the first column of the initial table. row, the second cell is located in the first row of the initial table. OK, 、 are all positive integers, Less than , the first cell and the second cell both have corresponding target text areas; The merging unit is used for If the sum of the first number of target text areas corresponding to the cells in the row satisfies the first preset condition, the first Row and Merge the rows to get the target table, where The target text area corresponding to each cell in the row includes the first The target text area corresponding to the corresponding cell in the row and the The target text area corresponding to the corresponding cell in the row; The filling unit is configured to fill the target table with text data in the target text area corresponding to each cell in the target table based on the correspondence between each target text area and each cell in the initial table, so as to obtain table data.

13. A computer storage medium, characterized in that The computer storage medium stores computer program instructions, which, when executed by a processor, are used to execute the table information extraction method according to any one of claims 1 to 11.

14. An electronic device, characterized in that: The electronic device includes a processor and a computer storage medium; The processor is adapted to implement one or more instructions; and The computer storage medium stores one or more instructions, and the one or more instructions are suitable for being loaded by the processor and executed by the table information extraction method according to any one of claims 1 to 11.

15. A computer program product, characterized in that The computer program product includes computer instructions, which are stored in a computer-readable storage medium; a processor of an electronic device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions. When the computer instructions are executed by the processor, they are used to execute the table information extraction method according to any one of claims 1 to 11.

Images