Table parsing method, device, computer equipment and storage medium

Abstract

The present application relates to a table parsing method, apparatus, computer equipment, storage medium and computer program product. The method includes: identifying text in an image to be processed, and the position of the text in the image to be processed; the image to be processed contains a table to be parsed; determining the header information and span information of the table to be parsed based on the text and the position; determining a target area from the image to be processed based on the header information and the span information; the target area is the smallest area of ​​the image to be processed that contains the table to be parsed; identifying target text from the target area; the target text is text belonging to the same cell; matching the target text with the header information to obtain a table parsing result of the table to be parsed in the image to be processed. The use of this method can improve the accuracy of table parsing.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a table parsing method, apparatus, computer equipment, storage medium, and computer program product. Background Art

[0002] In international document business, manual review of paper documents provided by customers is usually required. To improve business processing efficiency, technologies such as OCR (Optical Character Recognition) can be used to extract tables from images of paper documents, forming structured data, and thus realizing intelligent document review.

[0003] In the existing technology, the analysis of tables in images is mostly based on table borders. By identifying the table border lines, the original table is divided into fragments according to the table border lines, and then OCR recognition is performed on each fragment. Finally, the recognition results are reorganized according to the spatial relationship between the fragments to restore the structure and content of the original table, thereby achieving structuring of the recognition results.

[0004] However, in actual business scenarios, document styles vary greatly, and the table styles in documents may be non-standardized, which can easily affect the effectiveness of OCR recognition results and, in turn, affect subsequent intelligent document review. For example, there may be open tables that lack table borders. Open tables may also lack internal borders, left borders, or right borders. In this case, table parsing based on borders can easily identify multiple cells as one, causing confusion in the table content. There may also be cases where text content spans the table borders and exceeds the cell range. In this case, dividing the recognition results based on the table borders can easily lead to inaccurate cell recognition.

[0005] Therefore, the current table extraction technology has the problem of inaccurate table parsing results. Summary of the Invention

[0006] Based on this, it is necessary to provide a table parsing method, apparatus, computer equipment, computer-readable storage medium and computer program product that can improve accuracy in order to address the above technical issues.

[0007] In a first aspect, the present application provides a table parsing method. The method comprises:

[0008] Identifying text in an image to be processed and the position of the text in the image to be processed; the image to be processed contains a table to be parsed;

[0009] Determining header information and span information of the table to be parsed according to the text and the position;

[0010] Determine a target area from the image to be processed according to the header information and the span information; the target area is the smallest area in the image to be processed that contains the table to be parsed;

[0011] Identifying target text from the target area; the target text is text belonging to the same cell;

[0012] The target text is matched with the header information to obtain a table parsing result of the table to be parsed in the image to be processed.

[0013] In one embodiment, determining the header information and span information of the table to be parsed based on the text and the position includes:

[0014] Get the header keywords of the table to be parsed;

[0015] Obtaining the header of the table to be parsed by searching the header keyword in the text;

[0016] Determine header information of the header; the header information includes header vertices;

[0017] According to the table header vertex, the span information of the table to be parsed is obtained.

[0018] In one embodiment, the header information further includes a header position; and determining the target area from the image to be processed based on the header information and the span information includes:

[0019] Obtain the upper frame line of the table to be parsed by searching for a horizontal line above the table header that is closest to the table header and has a length not less than the span information;

[0020] Obtain the lower border of the table to be parsed by searching below the table header for a horizontal line that is farthest from the table header and has a length not less than the span information;

[0021] The target area is determined from the image to be processed according to the upper frame line and the lower frame line.

[0022] In one embodiment, identifying the target text from the target area includes:

[0023] Acquire spatial features of at least two texts in the target area;

[0024] Determining whether the spatial features of the at least two texts meet a preset condition;

[0025] If they match, determining whether the contents of the at least two texts are continuous;

[0026] If they are continuous, it is determined that the at least two texts are the target texts.

[0027] In one embodiment, the spatial features include horizontal spacing and vertical spacing; and determining whether the spatial features of the at least two texts meet a preset condition includes:

[0028] It is determined whether the horizontal distance between the at least two texts does not exceed a preset first distance, and / or it is determined whether the vertical distance between the at least two texts does not exceed a preset second distance.

[0029] In one embodiment, matching the target text with the header information to obtain a table parsing result of the table to be parsed in the image to be processed includes:

[0030] When the target text matches at least two of the table headers, respectively obtaining the distance between the target text and the at least two of the table headers;

[0031] The minimum distance is obtained by finding the minimum value among the distances;

[0032] Obtaining a target header that matches the target text according to the header corresponding to the minimum distance;

[0033] A table parsing result of the table to be parsed is obtained according to the target text and the target table header.

[0034] In a second aspect, the present application further provides a table parsing device. The device comprises:

[0035] An image text recognition module, configured to recognize text in an image to be processed and the position of the text in the image to be processed; the image to be processed includes a table to be parsed;

[0036] a table parameter determination module, configured to determine the header information and span information of the table to be parsed according to the text and the position;

[0037] a table area determination module, configured to determine a target area from the image to be processed based on the table header information and the span information; the target area being the smallest area in the image to be processed that contains the table to be parsed;

[0038] A table text recognition module is used to recognize target text from the target area; the target text is the text belonging to the same cell;

[0039] The table parsing module is used to match the target text with the table header information to obtain a table parsing result of the table to be parsed in the image to be processed.

[0040] In a third aspect, the present application further provides a computer device. The computer device includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are performed:

[0041] Identifying text in an image to be processed and the position of the text in the image to be processed; the image to be processed contains a table to be parsed;

[0042] Determining header information and span information of the table to be parsed according to the text and the position;

[0043] Determine a target area from the image to be processed according to the header information and the span information; the target area is the smallest area in the image to be processed that contains the table to be parsed;

[0044] Identifying target text from the target area; the target text is text belonging to the same cell;

[0045] The target text is matched with the header information to obtain a table parsing result of the table to be parsed in the image to be processed.

[0046] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the following steps:

[0047] Identifying text in an image to be processed and the position of the text in the image to be processed; the image to be processed contains a table to be parsed;

[0048] Determining header information and span information of the table to be parsed according to the text and the position;

[0049] Determine a target area from the image to be processed according to the header information and the span information; the target area is the smallest area in the image to be processed that contains the table to be parsed;

[0050] Identifying target text from the target area; the target text is text belonging to the same cell;

[0051] The target text is matched with the header information to obtain a table parsing result of the table to be parsed in the image to be processed.

[0052] In a fifth aspect, the present application further provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the following steps:

[0053] Identifying text in an image to be processed and the position of the text in the image to be processed; the image to be processed contains a table to be parsed;

[0054] Determining header information and span information of the table to be parsed according to the text and the position;

[0055] Determine a target area from the image to be processed according to the header information and the span information; the target area is the smallest area in the image to be processed that contains the table to be parsed;

[0056] Identifying target text from the target area; the target text is text belonging to the same cell;

[0057] The target text is matched with the header information to obtain a table parsing result of the table to be parsed in the image to be processed.

[0058] The above-mentioned table parsing method, device, computer equipment, storage medium and computer program product, by identifying the text in the image to be processed and the position of the text in the image to be processed, determine the header information and span information of the table to be parsed based on the text and the position, and determine the target area from the image to be processed based on the header information and the span information. When the table in the image to be processed is an open table, the area where the table is located can be accurately extracted, the target text can be identified from the target area, the target text can be matched with the header information, and the table parsing result of the table to be parsed in the image to be processed can be obtained. For the area where the table is located, the cells in the open table can be accurately matched to the corresponding columns, thereby improving the accuracy of open table parsing.

[0059] Moreover, the above-mentioned table parsing method, device, computer equipment, storage medium and computer program product are not only applicable to open tables, but also to structured tables. Since no standardized table border lines are required, accurate table parsing results can be obtained when the table border lines of the structured table are not standardized or are obscured by text content. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 Schematic diagram of a flow chart of a table parsing method in one embodiment;

[0061] Figure 2 is a schematic diagram of an image to be recognized in one embodiment;

[0062] Figure 3 A schematic diagram of extracting a table area from an image to be recognized in one embodiment;

[0063] Figure 4 A schematic diagram of determining text space features in one embodiment;

[0064] Figure 5 A schematic diagram of text space feature determination in another embodiment;

[0065] Figure 6 is a schematic diagram of matching target text with a table header in one embodiment;

[0066] Figure 7 A structural block diagram of a table parsing system in one embodiment;

[0067] Figure 8 It is a structural block diagram of a text positioning and recognition device of a table parsing system in one embodiment;

[0068] Figure 9 It is a structural block diagram of a table field extraction device of a table parsing system in one embodiment;

[0069] Figure 10 It is a structural block diagram of a cell aggregation device of a table parsing system in one embodiment;

[0070] Figure 11 A structural block diagram of a target column matching device of a table parsing system in one embodiment;

[0071] Figure 12 Schematic diagram of a table structured extraction method according to an embodiment;

[0072] Figure 13 is a flowchart of a table parsing method in another embodiment;

[0073] Figure 14 It is a structural block diagram of a table parsing device in one embodiment;

[0074] Figure 15 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0075] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0076] The table parsing method provided in the embodiments of the present application can be applied to a terminal or server. The terminal may include, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices may include smart speakers, smart TVs, smart air conditioners, smart car devices, etc. Portable wearable devices may include smart watches, smart bracelets, head-mounted devices, etc. The server may be implemented as a standalone server or a server cluster consisting of multiple servers.

[0077] In one embodiment, Figure 1 As shown, a table parsing method is provided, which is described by taking the application of the method to a server as an example, and includes the following steps:

[0078] Step S110 , identifying text in the image to be processed and the position of the text in the image to be processed; the image to be processed includes a table to be parsed.

[0079] The image to be processed may be an image of a paper document.

[0080] The form to be parsed may be a form on a paper document, an open form, or a structured form.

[0081] In a specific implementation, the image to be processed can be collected by, but is not limited to, photographing, scanning, and screenshotting. The collected image to be processed can be input into a server, and the server uses OCR technology to recognize the text in the image to be processed. During the text recognition process, OCR can also obtain the position of the text in the image to be processed.

[0082] For example, Figure 2 Provides a schematic diagram of the image to be identified. Figure 2 , you can choose Tesseract (an optical character recognition software) for optical character recognition and get Figure 2 The recognition results of some texts are as follows:

[0083]

[0084]

[0085] Here, Description represents the description, Net Weight represents the net weight, Unit Price represents the unit price, and Amount represents the quantity. Here, words represents the text content, position represents the text position, right_bottom represents the coordinates of the lower right corner of the text, and left_top represents the coordinates of the upper left corner of the text. This shows that Tesseract's recognition results include both the text in the image to be recognized and the location of the text.

[0086] Step S120 : determining the header information and span information of the table to be parsed according to the text and the position.

[0087] The header information may include the vertex position of the header part.

[0088] The span information may be the minimum width of the table to be parsed.

[0089] In a specific implementation, header keywords can be pre-set in the server. After identifying the text in the image to be processed, the server can select at least one header keyword and search for it in the identified text. When all header keywords are found in the text, the area containing all header keywords is determined as the header area of ​​the table to be parsed. The header area can be set to a rectangle, and the positions of the four vertices of the header area are obtained and stored as header information. According to the relative positions between the vertices, the four vertices of the header area can be named as the upper left vertex, the upper right vertex, the lower left vertex, and the lower right vertex. The upper left vertex and the upper right vertex can be connected by a line to obtain the span of the table to be parsed, and the lower left vertex and the lower right vertex can be connected by a line to obtain the span of the table to be parsed. The span can be the minimum width of the table to be parsed.

[0090] For example, Figure 3 Provides a schematic diagram of extracting the table area from the image to be recognized. Figure 3 , you can set the header keywords "Description", "Net Weight", "Unit Price" and "Amount (USD)", and search for the above keywords in the recognized text. After all keywords are found, the area containing all keywords can be determined as the header area. For example, Figure 3 The gray shaded area is identified as the header area. Get the coordinates of the upper left vertex A and the upper right vertex B of the header area respectively. Connect A and B to get the table span. You can also connect the lower left and lower right vertices of the header area to get the table span.

[0091] Step S130 : determining a target area from the image to be processed according to the header information and the span information; the target area is the smallest area in the image to be processed that contains the table to be parsed.

[0092] The header information may also include the location of the header.

[0093] In the specific implementation, you can select any point in the header area as the header position. Taking the header position as the reference, search upward in the image to be processed for the horizontal line that is closest to the header position and whose length is not less than the table span, as the upper frame line of the table to be parsed. You can also search downward in the image to be processed for the horizontal line that is farthest from the header position and whose length is not less than the table span, as the lower frame line of the table to be parsed. After finding the upper and lower frame lines, you can use the upper frame line as the upper boundary and the lower frame line as the lower boundary to intercept the target area from the image to be processed. The target area is the minimum area where the table to be parsed is located.

[0094] For example, for Figure 3, any point on the AB line (including point A and point B) can be used as the header position. When point A is used as the header position, a horizontal line that is closest to point A and whose length is not less than the length of the AB line can be found above point A to obtain the upper frame line CD of the table. A horizontal line that is farthest from point A and whose length is not less than the length of the AB line can be found below point A to obtain the lower frame line EF. Then the rectangle CEFD can be cut out from the image as the target area.

[0095] Step S140 , identifying target text from the target area; the target text is the text belonging to the same cell.

[0096] In a specific implementation, at least two texts can be randomly selected in the target area, and the spatial features of each text can be obtained. The spatial features can include the relative position relationship between any two texts, and the semantic relevance between any two texts. Based on the relative position relationship and the semantic relevance, it can be preliminarily determined whether the at least two texts belong to the same cell, and the at least two texts preliminarily determined to belong to the same cell can be connected to form a potential target text. A word order judgment model can also be pre-trained to determine whether the word order in the potential target text is continuous and valid. The potential target text is input into the word order judgment model. If the word order judgment model determines that the word order in the potential target text is continuous and valid, it can be determined that the text in the potential target text belongs to the same cell, and the potential target text can be used as the target text. Otherwise, if the word order judgment model determines that the word order in the potential target text is discontinuous, the text in the potential target text does not belong to the same cell, and the potential target text cannot be used as the target text.

[0097] In practical applications, we can preliminarily determine whether at least two texts belong to the same cell based on the following spatial features:

[0098] Feature 1: Text areas should be basically on the same horizontal line, and the horizontal distance between them should be close enough, for example, not exceeding the width of a single character. Figure 4 Provides a schematic diagram of text space feature judgment, according to Figure 4 The horizontal spacing between the G and H regions is smaller than that of a single character and they should belong to the same cell, while the spacing between the H and I regions is larger and they should not belong to the same cell.

[0099] Feature 2: The vertical distance between text areas should be close enough, for example, not exceeding the height of a single character.

[0100] Feature 3: judge based on special phrases. For example, the unit must include elements such as price. Figure 5 Provides another schematic diagram of text space feature judgment, according to Figure 5, USD 860.00 / MT is taken as a whole. Although the horizontal distance between the "USD" text area J and the "860.00 / MT" text area K is relatively far, as fixed expressions, they should be determined to belong to the same cell.

[0101] Based on the above spatial features, the texts in the target area that may belong to the same cell are connected to obtain the potential target text. The word order judgment model is used to determine whether the potential target text is continuous and valid. If it is continuous and valid, a new text area is formed. The new text area is treated as a separate text and further determined to determine whether it belongs to the same cell as other texts. Through continuous aggregation, all potential target texts can no longer form a continuous and valid expression. For example, there are texts L, M, N, and O in the target area. Based on the spatial features, L and M are connected to obtain the potential target text LM. The word order judgment model determines that LM is continuous and valid, and a new text can be formed. Then, the spatial features and word order judgment model can be used to determine whether LM and N can continue to be connected to form a new text. If so, a new text LMN is formed. If not, it can be determined whether LM and O can be connected to form a new text. If so, a new text LMO is formed. If not, LM is used as the target text, and N and O can be further aggregated.

[0102] Step S150 , matching the target text with the header information to obtain a table parsing result of the table to be parsed in the image to be processed.

[0103] In a specific implementation, a text classification model can be pre-trained to match target text to corresponding table headers. The target text is input into the text classification model, which then outputs the table headers that match the target text. If the text classification model identifies a target text that matches two or more table headers, the table header with the shortest distance to the target text is selected as the target header, and the target text is then matched against the target headers.

[0104] Figure 6 Provides a schematic diagram for matching target text with table headers, according to Figure 6 , the target text L matches both “Unit Price” and “Amount(USD)”. Since L is closer to “Unit Price”, the target text L should belong to the “Unit Price” column.

[0105] The above-mentioned table parsing method identifies the text in the image to be processed and the position of the text in the image to be processed, determines the header information and span information of the table to be parsed according to the text and the position, and determines the target area from the image to be processed according to the header information and the span information. When the table in the image to be processed is an open table, the area where the table is located can be accurately extracted, the target text can be identified from the target area, and the target text can be matched with the header information to obtain the table parsing result of the table to be parsed in the image to be processed. According to the area where the table is located, the cells in the open table can be accurately matched to the corresponding columns, thereby improving the accuracy of open table parsing.

[0106] Moreover, the above-mentioned table parsing method, device, computer equipment, storage medium and computer program product are not only applicable to open tables, but also to structured tables. Since no standardized table border lines are required, accurate table parsing results can be obtained when the table border lines of the structured table are not standardized or are obscured by text content.

[0107] In one embodiment, the above step S120 may specifically include: obtaining a header keyword of the table to be parsed; obtaining the header of the table to be parsed by searching for the header keyword in the text; determining the header information of the header; the header information includes header vertices; and obtaining the span information of the table to be parsed based on the header vertices.

[0108] In a specific implementation, header keywords can be pre-set in the server. After identifying the text in the image to be processed, the server can select at least one header keyword and search for it in the identified text. When all header keywords are found in the text, the area containing all header keywords is determined as the header of the table to be parsed. The header area can be set to a rectangle, and the positions of the four vertices of the header area are obtained and stored as header information. According to the relative positions between the vertices, the four vertices of the header area can be named as the upper left vertex, the upper right vertex, the lower left vertex and the lower right vertex. The upper left vertex and the upper right vertex can be connected by a line to obtain the span information of the table to be parsed. The lower left vertex and the lower right vertex can also be connected by a line to obtain the span information of the table to be parsed. The span information can be the minimum width of the table to be parsed.

[0109] For example, according to Figure 3 , you can set the header keywords "Description", "Net Weight", "UnitPrice" and "Amount(USD)", and search for the above keywords in the recognized text. After all keywords are found, the area containing all keywords can be determined as the header area. For example, Figure 3The gray shaded area is identified as the header area. Get the coordinates of the upper left vertex A and the upper right vertex B of the header area respectively. Connect A and B to get the table span. You can also connect the lower left and lower right vertices of the header area to get the table span.

[0110] In this embodiment, by obtaining the header keyword of the table to be parsed, obtaining the header of the table to be parsed by searching the header keyword in the text, determining the header information of the header, and obtaining the span information of the table to be parsed based on the header vertex, the header and span of the table to be parsed can be automatically obtained without human intervention, thereby improving the efficiency of table parsing.

[0111] In one embodiment, the above step S130 may specifically include: obtaining the upper frame line of the table to be parsed by searching for a horizontal line above the table header that is closest to the table header position and whose length is not less than the span information; obtaining the lower frame line of the table to be parsed by searching for a horizontal line below the table header that is farthest from the table header position and whose length is not less than the span information; and determining the target area from the image to be processed based on the upper frame line and the lower frame line.

[0112] In the specific implementation, you can select any point in the header area as the header position. Taking the header position as the reference, search upward in the image to be processed for the horizontal line that is closest to the header position and whose length is not less than the table span, as the upper frame line of the table to be parsed. You can also search downward in the image to be processed for the horizontal line that is farthest from the header position and whose length is not less than the table span, as the lower frame line of the table to be parsed. After finding the upper and lower frame lines, you can use the upper frame line as the upper boundary and the lower frame line as the lower boundary to intercept the target area from the image to be processed. The target area is the minimum area where the table to be parsed is located.

[0113] For example, for Figure 3 , any point on the AB line (including point A and point B) can be used as the header position. When point A is used as the header position, a horizontal line that is closest to point A and whose length is not less than the length of the AB line can be found above point A to obtain the upper frame line CD of the table. A horizontal line that is farthest from point A and whose length is not less than the length of the AB line can be found below point A to obtain the lower frame line EF. Then the rectangle CEFD can be cut out from the image as the target area.

[0114] In this embodiment, the upper frame line of the table to be parsed is obtained by searching for a horizontal line above the table header that is closest to the table header position and whose length is not less than the span information. The lower frame line of the table to be parsed is obtained by searching for a horizontal line below the table header that is farthest from the table header position and whose length is not less than the span information. Based on the upper frame line and the lower frame line, the target area is determined from the image to be processed. When the table in the image to be processed is an open table, the area where the table is located can be efficiently extracted, thereby improving the efficiency of table parsing.

[0115] In one embodiment, the above step S140 may specifically include:

[0116] Step S141, obtaining spatial features of at least two texts in the target area;

[0117] Step S142, determining whether the spatial features of at least two texts meet a preset condition;

[0118] Step S143: If yes, determine whether the contents of at least two texts are continuous;

[0119] Step S144: If they are continuous, it is determined that at least two texts are target texts.

[0120] In a specific implementation, at least two texts can be randomly selected in the target area, and the spatial features of each text can be obtained. The spatial features can include the relative position relationship between any two texts, and the semantic relevance between any two texts. Based on the relative position relationship and the semantic relevance, it can be preliminarily determined whether the at least two texts belong to the same cell, and the at least two texts preliminarily determined to belong to the same cell can be connected to form a potential target text. A word order judgment model can also be pre-trained to determine whether the word order in the potential target text is continuous and valid. The potential target text is input into the word order judgment model. If the word order judgment model determines that the word order in the potential target text is continuous and valid, it can be determined that the text in the potential target text belongs to the same cell, and the potential target text can be used as the target text. Otherwise, if the word order judgment model determines that the word order in the potential target text is discontinuous, the text in the potential target text does not belong to the same cell, and the potential target text cannot be used as the target text.

[0121] In practical applications, we can preliminarily determine whether at least two texts belong to the same cell based on the following spatial features:

[0122] Feature 1: The text areas should be basically on the same horizontal line, and the horizontal distance between them should be close enough, for example, not exceeding the width of a single character. Figure 4 The horizontal spacing between the G and H regions is smaller than that of a single character and they should belong to the same cell, while the spacing between the H and I regions is larger and they should not belong to the same cell.

[0123] Feature 2: The vertical distance between text areas should be close enough, for example, not exceeding the height of a single character.

[0124] Feature 3, judging based on special phrases, for example, the unit must include elements such as price. Figure 5, USD860.00 / MT is taken as a whole. Although the horizontal distance between the "USD" text area J and the "860.00 / MT" text area K is relatively far, as fixed expressions, they should be determined to belong to the same cell.

[0125] Based on the above spatial features, the texts in the target area that may belong to the same cell are connected to obtain the potential target text. The word order judgment model is used to determine whether the potential target text is continuous and valid. If it is continuous and valid, a new text area is formed. The new text area is treated as a separate text and further determined to determine whether it belongs to the same cell as other texts. Through continuous aggregation, all potential target texts can no longer form a continuous and valid expression. For example, there are texts L, M, N, and O in the target area. Based on the spatial features, L and M are connected to obtain the potential target text LM. The word order judgment model determines that LM is continuous and valid, and a new text can be formed. Then, the spatial features and word order judgment model can be used to determine whether LM and N can continue to be connected to form a new text. If so, a new text LMN is formed. If not, it can be determined whether LM and O can be connected to form a new text. If so, a new text LMO is formed. If not, LM is used as the target text, and N and O can be further aggregated.

[0126] In this embodiment, by obtaining the spatial features of at least two texts in the target area, it is determined whether the spatial features of the at least two texts meet the preset conditions. If they do, it is determined whether the contents of the at least two texts are continuous. If they are continuous, the at least two texts are determined to be target texts. The table can be parsed from the two aspects of spatial analysis and natural language understanding, with high fault tolerance, so that the parsing results have high reliability.

[0127] In one embodiment, the above step S142 may specifically include: determining whether the horizontal distance between at least two texts does not exceed a preset first distance, and / or determining whether the vertical distance between at least two texts does not exceed a preset second distance.

[0128] In a specific implementation, the spatial features of at least two texts in the target area may include the horizontal spacing between the texts. The position coordinates of the two texts are obtained. If the difference in the vertical coordinates is less than a preset threshold, it can be determined that the two texts are on the same horizontal line. The horizontal spacing between the two texts is calculated, and it is determined whether the horizontal spacing does not exceed a preset first spacing. For example, the width of a single text can be used as the first spacing. If it does not exceed the first spacing, the two texts are determined to belong to the same cell. Otherwise, if it exceeds the first spacing, it is determined that they do not belong to the same cell. The spatial features of at least two texts in the target area may also include the vertical spacing between the texts. If the difference in the horizontal coordinates in the position coordinates of the two texts is less than a preset threshold, it can be determined that the two texts are on the same vertical line. The vertical spacing between the two texts is calculated, and it is determined whether the vertical spacing does not exceed a preset second spacing. For example, the height of a single text can be used as the second spacing. If it does not exceed the second spacing, the two texts are determined to belong to the same cell. Otherwise, if it exceeds the second spacing, it is determined that they do not belong to the same cell.

[0129] In this embodiment, by determining whether the horizontal spacing between at least two texts does not exceed a preset first spacing, and / or determining whether the vertical spacing between at least two texts does not exceed a preset second spacing, the text in the target area can be automatically aggregated to form the cells in the table to be parsed, without the need for manual operation, thereby improving the efficiency of table parsing.

[0130] In one embodiment, the above step S150 may specifically include: when the target text matches at least two table headers, respectively obtaining the distances between the target text and the at least two table headers; obtaining the minimum distance by searching for the minimum value among the distances; obtaining the target table header that matches the target text based on the table header corresponding to the minimum distance; and obtaining the table parsing result of the table to be parsed based on the target text and the target table header.

[0131] In the specific implementation, if a target text is identified by the text classification model and matches two or more headers at the same time, the distance between the target text and each header can be calculated, and the minimum distance can be selected. The header corresponding to the minimum distance is closest to the target text. This header can be used as the target header, and the target text can be attributed to the target header.

[0132] For example, according to Figure 6 , the target text L matches both “Unit Price” and “Amount(USD)”. Since L is closer to “Unit Price”, the target text L should belong to the “Unit Price” column.

[0133] In this embodiment, when the target text matches at least two table headers, the distances between the target text and at least two table headers are obtained respectively, the minimum distance is obtained by finding the minimum value in each distance, the target table header matching the target text is obtained according to the table header corresponding to the minimum distance, and the table parsing result of the table to be parsed is obtained according to the target text and the target table header. This can improve the accuracy of matching the target text with the table header and reduce the probability of matching misalignment.

[0134] In one embodiment, Figure 7 As shown, a table parsing system is provided, comprising: a text positioning and recognition device 710, a table field extraction device 720, a cell aggregation device 730, and a target column matching device 740, wherein the text positioning and recognition device 710 is connected to the table field extraction device 720, the text positioning and recognition device 710 is further connected to the cell aggregation device 730, and the cell aggregation device 730 is connected to the target column matching device 740.

[0135] The text location recognition device 710 mainly uses OCR technology to recognize text in an image and the position information of the text in the image.

[0136] The table field extraction device 720 uses keyword matching within the OCR results to locate the table header and obtain its location within the image. Even the most open tables retain their top and bottom edges. By analyzing the table lines that cross the header, the upper and lower boundaries of the table are located, ultimately allowing the table to be extracted from the original image.

[0137] The cell aggregation device 730 connects the recognition results by analyzing the relative position relationship between the recognition results to preliminarily form candidate paragraphs, and then uses the semantic continuity discrimination model to determine whether the connected text areas actually belong to the same paragraph, forming paragraphs that are semantically continuous in their respective fields, thereby aggregating the originally scattered recognition results into candidate cells.

[0138] The target column matching device 740 uses a text classification method to determine which column of the table header the candidate cell should belong to, and then completes the structured extraction of the table based on the relative position relationship between the cells.

[0139] Figure 8 A structural block diagram of a text positioning and recognition device 710 is provided. Figure 8 The text positioning and recognition device 710 includes an OCR recognition unit 711, wherein:

[0140] The OCR recognition unit 711 uses OCR technology to recognize the text in the image and obtain the coordinate position of the text in the image. The OCR technology can be selected from Tesseract. Figure 2The image shown can identify the text and the corresponding position coordinates of the text area in the image. Figure 2 Taking the header in as an example, the recognition results are as follows:

[0141]

[0142]

[0143] Figure 9 A structural block diagram of a table field extraction device 720 is provided. Figure 9 The table field extraction device 720 includes a target header identification unit 721 and a table boundary identification and interception unit 722, wherein:

[0144] The target table header recognition unit 721 finds the position of the target table header in the image in the OCR recognition result by a keyword search method, and preliminarily locates the start and span of the table.

[0145] For example, for Figure 2 In the image shown, the table headers "Description", "Net Weight", "Unit Price" and "Amount(USD)" are found through keyword matching. At the same time, the upper left vertex A of the text area "Description" and the upper right vertex B of the text area "Amount(USD)" can be determined. The line AB can be preliminarily regarded as the table span.

[0146] Table boundary recognition and extraction unit 722: In an open table, the middle table line may be lost, but the top and bottom edges will be retained. After extracting the table header position and the initial table span, the table header position is used as a reference. The horizontal line closest to the table header position that is large enough to span the entire table is used as the upper boundary of the table; the horizontal line farthest from the table header position that is large enough to span the entire table is used as the lower boundary. The detection and extraction of horizontal lines in the image can use OpenCV's straight line detection technology. After finding the upper and lower boundaries of the table, the table image is separately cut out from the image based on this.

[0147] For example, using the above method in Figure 3 Find CD as the upper boundary of the table and EF as the lower boundary of the table, and then use this to cut out the CEFD rectangular image containing the table from the original image.

[0148] Figure 10 A block diagram of the structure of a cell aggregation device 730 is provided. Figure 10 The cell aggregation device 730 includes a candidate region identification unit 731, a word order judgment model generation unit 732, and a word order judgment execution unit 733, wherein:

[0149] The candidate region identification unit 731 uses spatial analysis and relative positional relationships to determine whether two separate text regions are continuous and belong to the same cell in the output of the table image after passing through the text positioning and recognition device. For example, it determines whether the two identified regions are close enough and are in the same row. These spatial features are used to find potential text pairs belonging to the same cell. Specific examples of spatial features are as follows:

[0150] Feature 1: The text areas should be basically on the same horizontal line, and the horizontal distance between them should be close enough, for example, not exceeding the width of a single character. Figure 4 The horizontal spacing between the G and H regions is smaller than that of a single character and they should belong to the same cell; while the spacing between the H and I regions is larger and they should not belong to the same cell.

[0151] Feature 2: The vertical distance between text areas should be close enough, for example, not higher than the height of a single character.

[0152] Feature 3, special phrase judgment, for example, the unit must include price and other elements. Figure 5 As shown, USD 860.00 / MT is taken as a whole. Although the horizontal distance between the "USD" text area J and the "860.00 / MT" text area K is relatively far, they should be judged as belonging to the same cell as fixed expressions.

[0153] The word order judgment model generation unit 732 utilizes existing or collected training text containing sufficient valid expressions in the given field, and uses statistical learning or deep learning techniques to train a language model that determines whether word order is continuous. The model takes a piece of text as input and outputs a judgment result, indicating whether the input text is a continuous, valid expression in the given field. This model generation unit is executed once at system startup and is executed as appropriate during system operation as training samples are accumulated.

[0154] The word order determination execution unit 733 connects the text that may belong to the same cell, as output by the candidate region recognition unit. The word order determination model then determines whether the connected text is continuous and valid. If so, it forms a new text region, which also participates in the subsequent iterative aggregation process. This text region recognition and aggregation process continues until all potential text regions cannot form a continuous and valid expression, completing the aggregation of all cells.

[0155] Figure 11 A structural block diagram of a target column matching device 740 is provided. Figure 11 The target column matching device 740 includes a candidate target column classification model generating unit 741 and a target column classification matching unit 742, wherein:

[0156] The target column classification model generation unit 741 uses the cell content as the value and the corresponding column name as the key. Based on valid historical data, a sufficient number of such key-value pairs are collected for the table type being processed to train a text classification model. The model input is the cell content and the output is the column name. The model can use a deep learning model such as Fasttext, Transformer, or BERT. This model generation unit is executed once at system startup and is executed as appropriate during system operation as valid samples accumulate.

[0157] The target column classification matching unit 742 passes the cell content found by the cell aggregation device through the target column classification model, and the model outputs which column the cell belongs to. If there is a cell that matches multiple columns at the same time, further judgment should be made based on the principle of proximity, that is, the closest distance in space. For example, Figure 6 Region L matches both the "Unit Price" and "Amount(USD)" columns, but because region L is closer to "Unit Price", region L should belong to the "UnitPrice" column.

[0158] In one embodiment, Figure 12 As shown in the figure, based on the above table parsing system, a method for extracting structured content from open-ended image tables based on relative spatial position analysis and NLP natural language processing is proposed. The method mainly includes the following steps:

[0159] Step S1210: Applying a text location recognition device to the image to be analyzed to obtain the recognition result text and the position information of the recognition result in the image;

[0160] Step S1220: The recognition result of step S1210 and the original image are passed through a table field extraction device to first obtain the position information of the target table header in the image, and then the entire table image is cut out from the original image based on the upper and lower characteristic edges of the table;

[0161] Step S1230: Apply the text positioning and recognition device to the table image to be analyzed again to obtain the recognition result text and the position information of the recognition result in the table image;

[0162] Step S1240 , the OCR recognition result including the positioning information outputted from step S1230 is passed through a cell aggregation device to combine the originally scattered text regions into semantically continuous paragraphs, each of which is a candidate cell;

[0163] In step S1250, the output result of step S1240 is passed through the target column matching device to complete the matching of candidate cells and table headers, and then the cells are sorted according to the spatial order to complete the structuring of the table recognition result.

[0164] Figure 13 A flowchart of a table parsing method is provided. Figure 13 Taking the application to the server as an example, the table parsing method may include the following steps:

[0165] Step S1301, application system initialization;

[0166] Step S1302, generating an initial word order discrimination model;

[0167] Step S1303, generating an initial target column classification model;

[0168] Step S1304, the application system starts to start;

[0169] Step S1305, obtaining an image to be processed;

[0170] Step S1306: Perform text location recognition on the image to obtain the text content recognized by OCR and the location information of the recognition area;

[0171] Step S1307, determining the positions of the target table header and upper and lower edges through keyword and relative spatial position analysis;

[0172] Step S1308 , extracting a table image from the original image using the table boundaries and spans;

[0173] Step S1309: Position and recognize text in the table image separately to obtain recognition results and location information;

[0174] Step S1310 , by analyzing the spatial relationship between text regions, a group of candidate text regions that potentially constitute the same cell are found, and these groups are connected to form candidate cell regions;

[0175] Step S1311: The text in the candidate cell regions is passed through a word order judgment model to determine whether the connected text is coherent and valid. If not, the process returns to step S1310 to identify new candidate regions.

[0176] Step S1312: If valid, the text is assigned to the same cell and forms a new text region. The new text region continues to participate in the next round of recognition and aggregation. Iterate steps S1310 to S1312 until no more text regions meeting the conditions are found, forming a series of valid cell texts.

[0177] Step S1313, using the text classification model, find the column to which the candidate cell should belong;

[0178] Step S1314: For candidate cells that may belong to multiple target columns, further determine their belonging through spatial distance analysis;

[0179] Step S1315: Iterate step S1313 to step S1315 until all target key-value pairs are matched. Rearrange the recognition results according to the relative spatial position relationship between cells to complete the structuring of the table recognition results.

[0180] The above table parsing method iteratively completes open tables in images based on spatial analysis and word order judgment. It is applicable not only to open tables but also to structured tables. Since standardized table lines are not required, only the upper and lower edges of the table boundary need to be located. By identifying the table header and upper and lower boundaries, the table can be cut out from the original image. Further, OCR and NLP technologies are used to complete the structured extraction of the table image. Therefore, the above table parsing method has the following beneficial effects:

[0181] The quality requirements for the original images are low, and the analysis is performed through spatial analysis and natural language understanding, rather than relying solely on spatial analysis, and has high fault tolerance.

[0182] The reliability of the parsing results is higher. The results are judged based on semantic understanding, and cells and columns are matched, so the probability of matching misalignment is lower.

[0183] The word order judgment model and text classification model are adopted. The model is evolutionary and learnable. By continuously accumulating data and iteratively updating the model, the accuracy can be continuously improved.

[0184] It should be understood that, although the various steps in the flowcharts involved in the various embodiments described above are displayed in sequence according to the instructions of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the various embodiments described above can include multiple steps or multiple stages, and these steps or stages are not necessarily executed and completed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0185] Based on the same inventive concept, the present application also provides a table parsing device for implementing the table parsing method described above. The solution provided by this device is similar to the solution described in the method described above. Therefore, the specific limitations of one or more table parsing device embodiments provided below can be found in the above-mentioned limitations of the table parsing method and will not be repeated here.

[0186] In one embodiment, Figure 14 As shown, a table parsing device 1400 is provided, comprising: an image text recognition module 1410, a table parameter determination module 1420, a table area determination module 1430, a table text recognition module 1440 and a table parsing module 1450, wherein:

[0187] The image text recognition module 1410 is used to recognize text in an image to be processed and the position of the text in the image to be processed; the image to be processed contains a table to be parsed;

[0188] A table parameter determination module 1420 is configured to determine the header information and span information of the table to be parsed based on the text and the position;

[0189] The table area determination module 1430 is configured to determine a target area from the image to be processed based on the table header information and the span information; the target area is the smallest area in the image to be processed that contains the table to be parsed;

[0190] The table text recognition module 1440 is used to recognize target text from the target area; the target text is the text belonging to the same cell;

[0191] The table parsing module 1450 is configured to match the target text with the table header information to obtain a table parsing result of the table to be parsed in the image to be processed.

[0192] In one embodiment, the above-mentioned table parameter determination module 1420 is also used to obtain the header keyword of the table to be parsed; obtain the header of the table to be parsed by searching the header keyword in the text; determine the header information of the header; the header information includes header vertices; and obtain the span information of the table to be parsed based on the header vertices.

[0193] In one embodiment, the table area determination module 1430 is further used to obtain the upper frame line of the table to be parsed by searching for a horizontal line above the table header that is closest to the table header position and whose length is not less than the span information; to obtain the lower frame line of the table to be parsed by searching for a horizontal line below the table header that is farthest from the table header position and whose length is not less than the span information; and to determine the target area from the image to be processed based on the upper frame line and the lower frame line.

[0194] In one embodiment, the table text recognition module 1440 further includes:

[0195] An acquisition module, configured to acquire spatial features of at least two texts in the target area;

[0196] A first judging module is configured to judge whether the spatial features of the at least two texts meet a preset condition;

[0197] A second judgment module is used to judge whether the contents of the at least two texts are continuous if the conditions are met;

[0198] A determination module is configured to determine that the at least two texts are the target texts if they are continuous.

[0199] In one embodiment, the first judgment module is further configured to judge whether the horizontal distance between the at least two texts does not exceed a preset first distance, and / or judge whether the vertical distance between the at least two texts does not exceed a preset second distance.

[0200] In one embodiment, the above-mentioned table parsing module 1450 is also used to obtain the distance between the target text and at least two of the table headers respectively when the target text matches at least two of the table headers; obtain the minimum distance by finding the minimum value in each of the distances; obtain the target table header that matches the target text based on the table header corresponding to the minimum distance; and obtain the table parsing result of the table to be parsed based on the target text and the target table header.

[0201] Each module in the table parsing device described above may be implemented in whole or in part through software, hardware, or a combination thereof. Each module may be embedded in or independent of a processor in a computer device in the form of hardware, or may be stored in a memory in the computer device in the form of software, so that the processor can call and execute the corresponding operations of each module.

[0202] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 15 As shown. The computer device includes a processor, a memory, and a network interface connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The database of the computer device is used to store table parsing data. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a table parsing method is implemented.

[0203] Those skilled in the art will understand that Figure 15The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0204] In one embodiment, a computer device is further provided, including a memory and a processor. The memory stores a computer program, and the processor implements the steps in the above method embodiments when executing the computer program.

[0205] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0206] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0207] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0208] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processor involved in the various embodiments provided herein may be, but are not limited to, a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic unit, a data processing logic unit based on quantum computing, and the like.

[0209] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0210] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A table parsing method, characterized in that: The method comprises: Identifying text in an image to be processed and the position of the text in the image to be processed; the image to be processed contains a table to be parsed; Determining header information and span information of the table to be parsed according to the text and the position; Determine a target area from the image to be processed according to the header information and the span information; the target area is the smallest area in the image to be processed that contains the table to be parsed; Identifying target text from the target area; the target text is text belonging to the same cell; Matching the target text with the header information to obtain a table parsing result of the table to be parsed in the image to be processed; The identifying target text from the target area includes: Acquire spatial features of at least two texts in the target area; Determining whether the spatial features of the at least two texts meet a preset condition; If they match, determining whether the contents of the at least two texts are continuous; If they are continuous, it is determined that the at least two texts are the target texts.

2. The method according to claim 1, characterized in that The step of determining the header information and span information of the table to be parsed according to the text and the position includes: Get the header keywords of the table to be parsed; Obtaining the header of the table to be parsed by searching the header keyword in the text; Determine header information of the header; the header information includes header vertices; According to the table header vertex, the span information of the table to be parsed is obtained.

3. The method according to claim 2, characterized in that The header information also includes a header position; and determining a target area from the image to be processed based on the header information and the span information includes: Obtain the upper frame line of the table to be parsed by searching for a horizontal line above the table header that is closest to the table header and has a length not less than the span information; Obtain the lower border of the table to be parsed by searching below the table header for a horizontal line that is farthest from the table header and has a length not less than the span information; The target area is determined from the image to be processed according to the upper frame line and the lower frame line.

4. The method according to claim 1, wherein The spatial features include horizontal spacing and vertical spacing; and determining whether the spatial features of the at least two texts meet a preset condition includes: It is determined whether the horizontal distance between the at least two texts does not exceed a preset first distance, and / or it is determined whether the vertical distance between the at least two texts does not exceed a preset second distance.

5. The method according to claim 2, characterized in that The step of matching the target text with the header information to obtain a table parsing result of the table to be parsed in the image to be processed includes: When the target text matches at least two of the table headers, respectively obtaining the distance between the target text and the at least two of the table headers; The minimum distance is obtained by finding the minimum value among the distances; Obtaining a target header that matches the target text according to the header corresponding to the minimum distance; A table parsing result of the table to be parsed is obtained according to the target text and the target table header.

6. A table parsing device, characterized in that: The device comprises: An image text recognition module, configured to recognize text in an image to be processed and the position of the text in the image to be processed; the image to be processed includes a table to be parsed; a table parameter determination module, configured to determine the header information and span information of the table to be parsed according to the text and the position; a table area determination module, configured to determine a target area from the image to be processed based on the table header information and the span information; the target area being the smallest area in the image to be processed that contains the table to be parsed; A table text recognition module is used to recognize target text from the target area; the target text is the text belonging to the same cell; a table parsing module, configured to match the target text with the table header information to obtain a table parsing result of the table to be parsed in the image to be processed; The table text recognition module further includes: An acquisition module, configured to acquire spatial features of at least two texts in the target area; A first judging module is configured to judge whether the spatial features of the at least two texts meet a preset condition; The second judgment module is configured to judge whether the contents of the at least two texts are continuous if the two texts match; if so, determine that the at least two texts are the target texts.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 5 are implemented.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

9. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images