Table structure recognition and correction method and device, equipment and storage medium

Abstract

The invention relates to a table structure recognition and correction method and device, equipment and a storage medium, and the method comprises the steps: obtaining a to-be-recognized table, and dividing the to-be-recognized table into a plurality of to-be-recognized regions according to the cell content of each cell in the to-be-recognized table; for each to-be-identified area, executing the following processing: extracting attribute features of the to-be-identified area, and obtaining an initial structure category identification result of the to-be-identified area according to the attribute features and a first classification model; obtaining attribute features of a context region of the to-be-identified region; wherein the context region comprises at least one other to-be-recognized region adjacent to the to-be-recognized region; and correcting the initial structure category recognition result in combination with the attribute features of the context region of the to-be-recognized region to obtain a final structure category recognition result of the to-be-recognized region. Therefore, the accuracy of a final table structure recognition result can be improved.

Description

Technical Field

[0001] The present application relates to the field of computers, and in particular to a method, device, equipment and storage medium for identifying and correcting a table structure. Background Art

[0002] A spreadsheet consists of multiple rows (or columns). The rows can be divided into different structural categories according to the content of each row, such as: title, table field, table content, notes, subtotal, etc. The structural categories of all rows in the table constitute the structural category of the table. Identifying the structural category of a table is conducive to generating data analysis for the table.

[0003] In the related art, a classification model is used to determine the structural category of a row in a table, and in order to increase the accuracy of the structural category recognition result, a binary classification model is set for each structural category to respectively identify whether the row in the table is a structural category corresponding to the binary classification model. For example, the classification result of the title model is a decimal between 0 and 1, and the closer to 1, the structural category of the corresponding row is a title, and the closer to 0, the structural category of the corresponding row is not a title.

[0004] This classification method may cause a row of cells to have similar classification results in binary classification models of different structural categories, resulting in inaccurate recognition results of the structural category of the row. Summary of the invention

[0005] The present application provides a method, device, equipment and storage medium for identifying and correcting a table structure, so as to solve the technical problem in the prior art that the structural category of a row in a table cannot be accurately identified.

[0006] In a first aspect, the present application provides a method for identifying and correcting a table structure, the method comprising:

[0007] Acquire a table to be identified, and divide the table to be identified into a plurality of areas to be identified according to the cell content of each cell in the table to be identified, wherein the area to be identified includes at least one row or at least one column in the table to be identified;

[0008] The following processing is performed for each of the areas to be identified:

[0009] Extracting attribute features of the area to be identified, and obtaining an initial structural category identification result of the area to be identified based on the attribute features and a first classification model;

[0010] Acquire attribute features of a context area of ​​the area to be identified; wherein the context area includes at least one other area to be identified that is adjacent to the area to be identified;

[0011] The initial structure category recognition result is corrected in combination with the attribute features of the context area of ​​the to-be-recognized area to obtain a final structure category recognition result of the to-be-recognized area.

[0012] In a possible implementation, the correcting process of the initial structure category recognition result in combination with the attribute features of the context area of ​​the area to be recognized to obtain the final structure category recognition result of the area to be recognized includes:

[0013] Generate context features of the area to be identified based on the attribute features of the area to be identified and its context area;

[0014] Determining the confidence of the initial structure category recognition result according to the context feature;

[0015] When it is determined that the initial structure category recognition result is not credible according to the confidence level, the initial structure category recognition result is corrected according to the context feature to obtain a final structure category recognition result of the to-be-recognized area.

[0016] In a possible implementation manner, determining the confidence level of the initial structural category recognition result of the to-be-recognized area according to the context feature includes:

[0017] The context features are input into a prediction model to obtain the confidence level of the initial structural class recognition result.

[0018] In a possible implementation manner, the correcting the initial structure category recognition result according to the context feature to obtain the final structure category recognition result of the to-be-recognized area includes:

[0019] The context feature is input into a second classification model, and a final structural category recognition result of the to-be-recognized area is determined according to output information of the second classification model.

[0020] In a possible implementation manner, generating the context feature of the area to be identified according to the attribute features of the area to be identified and its context area respectively includes:

[0021] splicing the attribute features of the to-be-recognized area and its context area in a set order, and determining the splicing result as the context feature of the to-be-recognized area;

[0022] Alternatively, the attribute features of the to-be-recognized region and its context region and the initial structure category recognition results are spliced ​​in a set order, and the splicing result is determined as the context feature of the to-be-recognized region.

[0023] In a possible implementation manner, before inputting the context feature and the initial structure category recognition result into the second classification model, the method further includes:

[0024] The following processing is performed on the initial structural category recognition results of the to-be-recognized area and its context area respectively:

[0025] Determining the weight of the initial structure category recognition result according to the output information of the first classification model;

[0026] Using the weights to perform weighted processing on the initial structure category recognition results;

[0027] The step of inputting the context features and the initial structure category recognition results into the second classification model is performed using the initial structure category recognition results after each weighted process.

[0028] In a possible implementation manner, dividing the table to be identified into a plurality of areas to be identified according to the cell content of each cell in the table to be identified includes:

[0029] Acquire the cell content of each cell in the table to be identified, and generate characteristic information of the cell according to the cell content;

[0030] Determine the similarity between any two adjacent rows or columns in the table to be identified based on the feature information of each cell in the table to be identified;

[0031] Two adjacent rows or columns whose similarity reaches a set similarity threshold are divided into a region to be identified.

[0032] In a possible implementation, when the table to be identified includes a merged cell, the similarity between the row occupied by the merged cell and its adjacent rows is determined in the following manner, wherein the adjacent rows occupied by the merged cell refer to: adjacent rows of a row area formed by the rows occupied by the merged cell:

[0033] The following processing is performed for each row occupied by the merged cell:

[0034] Determining a characteristic vector of the occupied row according to characteristic information of each minimum cell in the occupied row;

[0035] Determining the similarity between the occupied row and its adjacent rows according to the feature vector of the occupied row;

[0036] Two adjacent rows whose similarity reaches the set similarity threshold are divided into a to-be-recognized area in the following way:

[0037] When the number of similarities between each occupied row of the merged cell and its adjacent rows reaches a set similarity threshold and reaches a first threshold, each occupied row of the merged cell and its adjacent rows are divided into a to-be-identified area.

[0038] In a possible implementation, when the table to be identified includes a merged cell, the similarity between the row occupied by the merged cell and its adjacent rows is determined in the following manner:

[0039] The following processing is performed for each row occupied by the merged cell:

[0040] Determining a characteristic vector of the occupied row according to characteristic information of each minimum cell in the occupied row;

[0041] Determining the similarity between the occupied row and its adjacent rows according to the feature vector of the occupied row;

[0042] Two adjacent rows whose similarity reaches the set similarity threshold are divided into a to-be-recognized area in the following way:

[0043] When the similarity between the occupied row and its adjacent rows reaches a set similarity threshold, the occupied row and its adjacent rows are divided into a to-be-recognized area.

[0044] In a possible implementation, the attribute feature includes at least two of the following attribute dimension features: a position dimension, a cell merging information dimension, and a cell content type dimension.

[0045] In a second aspect, the present application provides a device for identifying and correcting a table structure, the device comprising:

[0046] A region division module, used for obtaining a table to be identified, and dividing the table to be identified into a plurality of regions to be identified according to the cell content of each cell in the table to be identified, wherein the region to be identified includes at least one row or at least one column in the table to be identified;

[0047] A feature extraction module is used to perform the following processing for each of the regions to be identified: extracting attribute features of the region to be identified, and obtaining an initial structural category recognition result of the region to be identified based on the attribute features and a first classification model;

[0048] The feature extraction module is further used to obtain attribute features of a context area of ​​each of the to-be-recognized areas; wherein the context area includes at least one other to-be-recognized area adjacent to the to-be-recognized area;

[0049] The correction module is used to correct the initial structure category recognition result for each of the to-be-recognized regions in combination with the attribute features of the context region of the to-be-recognized region to obtain a final structure category recognition result for the to-be-recognized region.

[0050] In a possible implementation, the correction module includes:

[0051] A context feature extraction unit, used for generating context features of the area to be identified based on the attribute features of the area to be identified and its context area;

[0052] A confidence prediction unit, used to determine the confidence of the initial structure category recognition result according to the context feature;

[0053] A correction unit is used to correct the initial structure category recognition result according to the context feature when it is determined that the initial structure category recognition result is not credible according to the confidence level, so as to obtain a final structure category recognition result of the area to be recognized.

[0054] In a possible implementation, the confidence prediction unit is specifically used to:

[0055] The context features are input into a prediction model to obtain the confidence level of the initial structural class recognition result.

[0056] In a possible implementation manner, the correction unit is specifically configured to:

[0057] The context feature is input into a second classification model, and a final structural category recognition result of the to-be-recognized area is determined according to output information of the second classification model.

[0058] In a possible implementation, the context feature extraction unit is used to:

[0059] splicing the attribute features of the to-be-recognized area and its context area in a set order, and determining the splicing result as the context feature of the to-be-recognized area;

[0060] Alternatively, the attribute features of the to-be-recognized region and its context region and the initial structure category recognition results are spliced ​​in a set order, and the splicing result is determined as the context feature of the to-be-recognized region.

[0061] In a possible implementation, the device further includes:

[0062] A weighted processing module is used to perform the following processing on the initial structural category recognition results of the to-be-recognized area and its context area respectively before inputting the context features and the initial structural category recognition results into the second classification model:

[0063] Determining the weight of the initial structure category recognition result according to the output information of the first classification model;

[0064] Using the weights to perform weighted processing on the initial structure category recognition results;

[0065] The step of inputting the context features and the initial structure category recognition results into the second classification model is performed using the initial structure category recognition results after each weighted process.

[0066] In a possible implementation manner, the region division module includes:

[0067] A cell feature generating unit, used for acquiring the cell content of each cell in the table to be identified, and generating feature information of the cell according to the cell content;

[0068] A similarity determination unit, configured to determine the similarity between any two adjacent rows or columns in the table to be identified based on the characteristic information of each cell in the table to be identified;

[0069] The division unit is used to divide two adjacent rows or two adjacent columns whose similarity reaches a set similarity threshold into a to-be-recognized area.

[0070] In a possible implementation, when the table to be identified includes a merged cell, the similarity between the row occupied by the merged cell and its adjacent rows is determined in the following manner, wherein the adjacent rows occupied by the merged cell refer to: adjacent rows of a row area formed by the rows occupied by the merged cell:

[0071] The following processing is performed for each row occupied by the merged cell:

[0072] Determining a characteristic vector of the occupied row according to characteristic information of each minimum cell in the occupied row;

[0073] Determining the similarity between the occupied row and its adjacent rows according to the feature vector of the occupied row;

[0074] Two adjacent rows whose similarity reaches the set similarity threshold are divided into a to-be-recognized area in the following way:

[0075] When the number of similarities between each occupied row of the merged cell and its adjacent rows reaches a set similarity threshold and reaches a first threshold, each occupied row of the merged cell and its adjacent rows are divided into a to-be-identified area.

[0076] In a possible implementation, when the table to be identified includes a merged cell, the similarity between the row occupied by the merged cell and its adjacent rows is determined in the following manner:

[0077] The following processing is performed for each row occupied by the merged cell:

[0078] Determining a characteristic vector of the occupied row according to characteristic information of each minimum cell in the occupied row;

[0079] Determining the similarity between the occupied row and its adjacent rows according to the feature vector of the occupied row;

[0080] Two adjacent rows whose similarity reaches the set similarity threshold are divided into a to-be-recognized area in the following way:

[0081] When the similarity between the occupied row and its adjacent rows reaches a set similarity threshold, the occupied row and its adjacent rows are divided into a to-be-recognized area.

[0082] In a possible implementation manner, the attribute features include at least two of the following attribute dimension features: position dimension, layout style dimension, and content type dimension.

[0083] In a third aspect, the present application provides a device comprising: a processor and a memory, wherein the processor is used to execute a recognition and correction program of a table structure stored in the memory to implement the recognition and correction method of the table structure described in any one of the first aspects.

[0084] In a fourth aspect, the present application provides a storage medium storing one or more programs, which can be executed by one or more processors to implement the method for identifying and correcting the table structure described in any one of the first aspects.

[0085] The above technical solution provided by the embodiment of the present application has the following advantages over the prior art: the method provided by the embodiment of the present application divides the table to be identified into multiple areas to be identified according to the cell content of each cell in the table to be identified, extracts the attribute features of each area to be identified, and obtains the initial structural category recognition result of each area to be identified according to the attribute features and the first classification model; obtains the attribute features of the context area of ​​the area to be identified; wherein the context area includes at least one other area to be identified adjacent to the area to be identified, and the initial structural category recognition result is corrected according to the attribute features of the context area of ​​the area to be identified to obtain the final structural category recognition result of the area to be identified, thereby realizing the comprehensive consideration of the information of the area to be identified and its context area to determine the final structural category recognition result of the area to be identified. This processing method utilizes the overall layout of the table and the consistency of the data flow, which helps to accurately identify the structure of the table, and thus can ensure the accuracy of the final table structure recognition result. BRIEF DESCRIPTION OF THE DRAWINGS

[0086] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0087] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0088] One or more embodiments are exemplarily described by pictures in the corresponding drawings, and these exemplified descriptions do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings represent similar elements, and unless otherwise stated, the figures in the drawings do not constitute proportional limitations.

[0089] Figure 1 A flowchart of an embodiment of a method for identifying and correcting a table structure provided in an embodiment of the present application;

[0090] Figure 2 is an example of a table to be identified;

[0091] Figure 3 A flowchart of another method for identifying and correcting a table structure provided in an embodiment of the present application;

[0092] Figure 4 A flowchart of another method for identifying and correcting a table structure provided in an embodiment of the present application;

[0093] Figure 5A block diagram of an embodiment of a table structure recognition and correction device provided in an embodiment of the present application;

[0094] Figure 6 A schematic diagram of the structure of a device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0095] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0096] The disclosure below provides many different embodiments or examples to realize the different structures of the present application. In order to simplify the disclosure of the present application, the parts and settings of specific examples are described below. Of course, they are only examples, and the purpose is not to limit the present application. In addition, the present application can repeat reference numbers and / or letters in different examples. This repetition is for the purpose of simplification and clarity, and does not itself indicate the relationship between the various embodiments and / or settings discussed.

[0097] In order to solve the technical problem in the prior art that the structural category of rows in a table cannot be accurately identified, the present application provides a table structure identification and correction method, device, equipment and storage medium, which can improve the accuracy of the final table structure recognition result.

[0098] Figure 1 The following is a flow chart of an embodiment of a method for identifying and correcting a table structure provided in an embodiment of the present application. Figure 1 As shown, the process includes the following steps:

[0099] Step 101: obtain a table to be identified, and divide the table to be identified into a plurality of regions to be identified according to the cell content of each cell in the table to be identified, wherein the region to be identified includes at least one row or at least one column in the table to be identified.

[0100] The table to be identified refers to a table whose table structure is to be identified, which can be a table in a traditional format, such as a table in a .xls, .xlsx, etc. format, or a new generation of smart tables, online tables, etc. The embodiments of the present application do not limit the specific type of the table to be identified. The table to be identified is composed of multiple rows (or multiple columns), and the structure of the table rows (or table columns) can be divided into different categories, such as headlines, row headings, table contents, notes, subtotals, etc. The structural categories of each table row in the table to be identified together constitute the table structure of the table to be identified.

[0101] Each cell in the table to be identified refers to each independent cell in the table to be identified. Here, an independent cell refers to a cell in the table that can be regarded as an independent logical entity. These cells are independent in data, logic and operation, allowing users to operate freely without affecting the content or structure of other cells in the table. At present, there are two common types of independent cells in tables: merged cells and minimum cells. A merged cell is a cell obtained by merging at least two minimum cells. In this case, the minimum cell is a subcell in the merged cell. An independent minimum cell is a cell that cannot be split and does not participate in the merging of any cells. It can be seen that each cell in the table to be identified is either a merged cell or an independent minimum cell.

[0102] In step 101, the table to be identified is divided into a plurality of areas to be identified, each of which includes at least one row or at least one column in the table to be identified. Wherein, when the table to be identified is a vertical table (a vertical table means that the data is arranged mainly vertically, and the data items are usually located in the rows of the table, while the titles or field names are located in the columns; wherein, the title is located in the column means that the column where the title is located contains at least 2 columns of cells, or the title is located in the column means that the column where the title is located contains 1 column of cells, and the column of cells contains at least 2 rows, and at least 1 row contains data items except the row where the title is located; the field name is located in the column means that at least 2 attributes are located in a row, or the field name is located in the column means that 1 attribute is located in a row, and the column where the field name is located contains at least 1 row, and at least 1 row contains data items except the row where the field name is located), the area to be identified includes at least one row in the table to be identified, When the table to be identified is a horizontal table (a horizontal table means that the data is arranged mainly horizontally, and the data items are usually located in the columns of the table, while the headers or field names are located in the rows; wherein, a header is located in a row means that a row in which a header is located contains at least 2 rows of cells, or, a header is located in a row means that a row in which a header is located contains 1 row of cells, and the row of cells contains at least 2 columns, and at least 1 column contains data items except the column where the header is located; a field name is located in a row means that at least 2 field names are located in one column, or, a field name is located in a row means that 1 field name is located in one column, and the row in which the field name is located contains at least 1 column, and at least 1 column contains data items except the column where the field name is located), the area to be identified includes at least one column in the table to be identified.

[0103] It should be noted that, no matter it is a horizontal table or a vertical table, different to-be-recognized areas do not overlap with each other, that is, any row (or any column) in the to-be-recognized table can only be divided into one to-be-recognized area.

[0104] For example, see Figure 2, is an example of a table to be identified. Figure 2 The table to be identified in the example is a vertical table with 10 rows, of which the first row is a table field row (including multiple field names), and the second to tenth rows are content rows. Figure 2 The example table to be identified executes step 101, and divides the table to be identified into 8 areas to be identified, wherein the first area to be identified includes the first row of the table to be identified (row number 12), the second area to be identified includes the second row of the table to be identified (row number 13), the third area to be identified includes the third row of the table to be identified (row number 14), the fourth area to be identified includes the fourth and fifth rows of the table to be identified (row numbers 15 and 16), the fifth area to be identified includes the sixth row of the table to be identified (row number 17), the sixth area to be identified includes the seventh and eighth rows of the table to be identified (row numbers 18 and 19), the seventh area to be identified includes the ninth row of the table to be identified (row number 20), and the eighth area to be identified includes the tenth row of the table to be identified (row number 21).

[0105] As for how to divide the table to be identified into multiple areas to be identified according to the cell contents of each cell in the table to be identified, the following will be described. Figure 4 The illustrated embodiments are only exemplary explanations and will not be described in detail herein.

[0106] Step 102: for each region to be identified, extract the attribute features of the region to be identified, and obtain the initial structural category recognition result of the region to be identified based on the attribute features and the first classification model.

[0107] The attribute features of the area to be identified can reflect the characteristics of the area to be identified itself. In the technical solution of the embodiment of the present application, these features are mainly auxiliary features for identifying its structural category, including but not limited to the position characteristics of the area to be identified in the table, layout style characteristics, content type characteristics, etc.

[0108] Accordingly, in one embodiment, the attribute features of the area to be identified include features of at least two of the following attribute dimensions: a location dimension, a layout style dimension, and a content type dimension.

[0109] Among them, the attribute characteristics of the position dimension provide the relative position or absolute position information of the area to be identified in the table. In practical applications, the areas of certain structural categories may be located in specific positions in the table. For example, the table field row and the large title are usually located at the top of the table, the content row is usually located below the table field row, and the notes or footnotes are usually located at the bottom of the table or in a specific position. It can be seen that the attribute characteristics of the position dimension are helpful for the identification of structural categories.

[0110] Optionally, the attribute characteristics of the location dimension include but are not limited to one or more of the following characteristics:

[0111] (1) The starting position of the non-empty column on the left

[0112] The starting position of the non-empty column on the left refers to the column number of the first non-empty column traversed from the left side of the area to be identified to the right.

[0113] by Figure 2 Taking the table shown as an example, the starting position of the non-empty column on the left side of each area to be identified is column number 6.

[0114] (2) The starting position of the non-empty column on the right

[0115] The starting position of the non-empty column on the right refers to the column number of the first non-empty column traversed from the right side of the area to be identified to the left.

[0116] by Figure 2 Taking the table shown as an example, the starting position of the non-empty column on the right side of each area to be identified is column number 9.

[0117] (3) Number of columns in the area to be identified

[0118] The number of columns of the to-be-recognized area refers to the number of columns of the to-be-recognized area along the horizontal direction (for example, from left to right).

[0119] by Figure 2 Take the table shown as an example, where each area to be identified has 4 columns.

[0120] The attribute characteristics of the layout style dimension provide layout style information of the area to be identified, usually the cell merging involved in the area to be identified. In practical applications, there may be some correlation between layout style and structural category. For example, the header row is usually a merged cell. It can be seen that the attribute characteristics of the layout style dimension are helpful for the identification of structural categories.

[0121] Optionally, the attribute characteristics of the layout style dimension include but are not limited to the following characteristics:

[0122] (1) Number of blank columns

[0123] The number of blank columns refers to the number of columns in the area to be identified that are all blank.

[0124] by Figure 2 Taking the table shown as an example, the number of blank columns in each area to be identified is 0, that is, there is no blank column.

[0125] (2) The number of cells in the area to be identified

[0126] The cells here refer to independent cells, including merged cells and independent minimum cells.

[0127] (3) The number of merged cells in the area to be identified

[0128] (4) The ratio of the number of merged cells in the area to be identified to the number of all minimum cells contained in the area to be identified.

[0129] Among them, all the minimum cells contained in the area to be identified include independent minimum cells in the table and sub-cells in the merged cells.

[0130] For example, the area to be identified includes two rows of cells, the first row has 1 merged cell and 4 independent minimum cells, wherein the merged cell is obtained by merging 2 minimum cells, so the first row has a total of 6 minimum cells. The second row has 2 merged cells and 2 independent minimum cells, wherein both of the merged cells are obtained by merging 2 minimum cells, so the second row also has a total of 6 minimum cells, so the area to be identified has a total of 3 merged cells and 12 minimum cells, therefore, the ratio of the number of merged cells to the number of minimum cells in the area to be identified is 3 / 12.

[0131] (5) Maximum number of merged columns

[0132] The maximum number of merged columns refers to the maximum value of the number of merged columns corresponding to each merged cell in the area to be identified.

[0133] As shown in Table 1 below, the maximum number of merged columns in the second row (one of the areas to be identified) is 2, because there is only one merged cell in the row, and the merged cell is formed by merging cells in two adjacent columns.

[0134] Table 1

[0135]

[0136] (6) Maximum number of merged rows

[0137] The maximum number of merged rows refers to the maximum value of the number of merged rows corresponding to each merged cell in the area to be identified.

[0138] As shown in Table 2 below, the maximum number of merged rows in the first area to be identified (including the first row and the second row) is 2, because there is only one merged cell in the row, and the merged cell is formed by merging cells in two adjacent rows.

[0139] Table 2

[0140]

[0141] The attribute characteristics of the content type dimension provide information about the data type of the data content in the area to be identified, such as text, numbers, dates, etc. In practical applications, there may be some association between data types and structural categories. For example, the text in the title row is usually a description or summary of the data rows below, usually in text type, while the numbers in the content row may be specific measurements or statistical data. It can be seen that the attribute characteristics of the content type dimension help identify structural categories.

[0142] Optionally, the attribute characteristics of the content type dimension include but are not limited to one or more of the following characteristics:

[0143] (1) The type of the cell content of each cell.

[0144] Optionally, divide the cell contents into Chinese, English, numbers, dates, times, blanks, etc.

[0145] (2) The number of merged cells and non-merged cells including Chinese content

[0146] (3) Including the number of merged cells and the number of non-merged cells with English content

[0147] (4) Includes the number of merged cells and the number of non-merged cells with numeric content

[0148] (5) The number of merged cells including date content and the number of non-merged cells

[0149] (6) The ratio of the number of non-merged cells containing Chinese content to the number of non-merged cells with content in the area to be identified

[0150] (7) The ratio of the number of non-merged cells containing digital content to the number of non-merged cells with content in the area to be identified

[0151] (8) Number of cells with colons

[0152] (9) The difference between the font size of the cell content and the font size of the cell content in the adjacent area to be identified

[0153] In the technical solution provided by the above embodiment, by combining the attribute features of the position dimension, the layout style dimension and the content type dimension to identify the structural category of the area to be identified, the accuracy and reliability of table structure recognition can be significantly improved.

[0154] One additional point to be explained is that the above is merely an exemplary enumeration of the attribute characteristics of the area to be identified from the dimensions of location, layout style, and content type. In actual applications, the attribute characteristics of the area to be identified may also include other feature information. Those skilled in the art may flexibly set them according to actual scenarios and business needs, and the embodiments of the present application do not impose any restrictions on this.

[0155] Furthermore, the above explanation of the attribute characteristics of the area to be identified is based on a vertical table as an example. If the table to be identified is a horizontal table, the corresponding attribute characteristics can be adjusted in rows and columns, for example, the start position of the non-empty column on the left is adjusted to the start position of the non-empty row above, and the maximum number of merged columns is adjusted to the maximum number of merged rows. This will not be repeated here.

[0156] In one embodiment, the first classification model is a binary classification model corresponding to a preset structural category, which is used to determine whether the area to be identified belongs to its corresponding structural category. That is, the first classification model is a binary classification model for determining whether the area to be identified belongs to the corresponding structural category. Among them, the structural category may include one or more of the main title, table field row, table content and other structural categories (such as notes, subtotals, etc.). Accordingly, four first classification models are pre-set, namely, the first classification model corresponding to the main title, used to determine whether the area to be identified belongs to the main title; the first classification model corresponding to the row title, used to determine whether the area to be identified belongs to the row title; the first classification model corresponding to the table content, used to determine whether the area to be identified belongs to the table content; and the first classification model corresponding to other structural categories, used to determine whether the area to be identified belongs to other structural categories.

[0157] The output information of the first classification model is, for example, 0 or 1, or a number between 0 and 1, which is used to indicate whether the area to be identified belongs to the structural category corresponding to the first classification model. For example, 1 indicates that the area to be identified belongs to the structural category corresponding to the first classification model, and 0 indicates that the area to be identified does not belong to the structural category corresponding to the first classification model. For another example, a value greater than or equal to 0.7 (or other values ​​greater than 0.5) indicates that the area to be identified belongs to the structural category corresponding to the first classification model, and a value less than 0.7 indicates that the area to be identified does not belong to the structural category corresponding to the first classification model.

[0158] Based on this, in step 102, the attribute features of the area to be identified are respectively input into each first classification model to obtain the output information of each first classification model, indicating whether the area to be identified belongs to its corresponding structural category, and then the structural category identification result of the area to be identified (here referred to as the initial structural category identification result) can be obtained based on the output information of each first classification model.

[0159] For example, assuming that the attribute features of the area to be identified are respectively input into the above four first classification models, the output information obtained is: 1, 0, 0, 0, then it can be determined that the initial structural category recognition result of the area to be identified is: headline.

[0160] Step 103: For each area to be identified, obtain attribute features of a context area of ​​the area to be identified, wherein the context area includes at least one other area to be identified that is adjacent to the area to be identified.

[0161] The context area includes the upper context area and / or the lower context area of ​​the area to be identified. Optionally, for the first area to be identified, its context area includes the lower context area, for the last area to be identified, its context area includes the upper context area, and for other areas to be identified (areas to be identified except the first and last areas to be identified), its context area may include the upper context area and / or the lower context area. It will be understood by those skilled in the art that the upper context and the lower context mentioned here refer to the upper and lower relationship in physical position.

[0162] Taking a vertical table as an example, the upper area of ​​the area to be identified includes one or more areas to be identified located above the area to be identified, including at least an area to be identified located above and adjacent to the area to be identified, and in the case where the upper area includes multiple areas to be identified, these areas to be identified are usually continuous. Similarly, the lower area of ​​the area to be identified includes one or more areas to be identified located below the area to be identified, including at least an area to be identified located below and adjacent to the area to be identified, and in the case where the lower area includes multiple areas to be identified, these areas to be identified are usually continuous.

[0163] For example, the upper area of ​​the area to be identified includes the three areas to be identified that are located above the area to be identified and are closest to the area to be identified. The lower area of ​​the area to be identified includes the two areas to be identified that are located below the area to be identified and are closest to the area to be identified (when there are three areas to be identified).

[0164] For each to-be-recognized region in the context region, step 102 is used to extract its attribute features and obtain the corresponding initial structure category recognition result.

[0165] Step 104 : for each region to be identified, the initial structural category identification result is corrected in combination with the attribute features of the context region of the region to be identified, so as to obtain the final structural category identification result of the region to be identified.

[0166] From the description of the above steps, it can be known that the initial structural category recognition result of the area to be recognized is the recognition result of structural category recognition based on the attribute characteristics of the area to be recognized itself, that is, the initial structural category recognition result only depends on the attribute characteristics of the area to be recognized itself, which may be inaccurate. In this regard, by combining the attribute characteristics of the context area of ​​the area to be recognized, the initial structural category recognition result of the area to be recognized is corrected, and the consistency of the overall layout of the table and the data flow is utilized, which helps to accurately recognize the structure of the table.

[0167] In one embodiment of the present application, the initial structure category recognition result is corrected in combination with the attribute features of the context area of ​​the area to be recognized to obtain the final structure category recognition result of the area to be recognized, including: generating the context features of the area to be recognized based on the attribute features of the area to be recognized and its context area; or generating the context features of the area to be recognized based on the attribute features of the area to be recognized and its context area and the initial structure category recognition result. As for how to correct the initial structure category recognition result of each area to be recognized in combination with the attribute features of the context area of ​​the area to be recognized to obtain the final structure category recognition result of the area to be recognized, it will be described below. Figure 3 The illustrated embodiments are only exemplary explanations and will not be described in detail herein.

[0168] The technical solution provided by the embodiment of the present application divides the table to be identified into multiple areas to be identified according to the cell content of each cell in the table to be identified, extracts the attribute features of each area to be identified, and obtains the initial structural category recognition result of each area to be identified according to the first classification model of the attribute features; obtains the attribute features of the context area of ​​each area to be identified, and corrects the initial structural category recognition result in combination with the attribute features of the context area of ​​each area to be identified to obtain the final structural category recognition result of the area to be identified, thereby comprehensively considering the information of the area to be identified and its context area to determine the final structural category recognition result of the area to be identified. This processing method utilizes the overall layout of the table and the consistency of the data flow, which helps to accurately identify the structure of the table, and thus can improve the accuracy of the final table structure recognition result.

[0169] Figure 3 A flowchart of another embodiment of a method for identifying and correcting a table structure provided in an embodiment of the present application. Figure 3 The process shown in Figure 1 Based on the process shown in the figure, the following focuses on the specific implementation of correcting the initial structural category recognition result of each region to be recognized by combining the attribute characteristics of the context region of the region to be recognized to obtain the final structural category recognition result of the region to be recognized. Figure 3As shown, the process includes the following steps:

[0170] Step 301: Obtain a table to be identified, and divide the table to be identified into a plurality of regions to be identified according to the cell content of each cell in the table to be identified, wherein the region to be identified includes at least one row or at least one column in the table to be identified.

[0171] Step 302: for each region to be identified, extract the attribute features of the region to be identified, and obtain an initial structural category identification result of the region to be identified based on the attribute features and the first classification model.

[0172] Step 303: For each area to be identified, obtain attribute features of a context area of ​​the area to be identified; wherein the context area includes at least one other area to be identified that is adjacent to the area to be identified.

[0173] For detailed description of steps 301 to 303, please refer to Figure 1 The relevant descriptions in the process shown will not be repeated here.

[0174] Step 304: For each region to be identified, a context feature of the region to be identified is generated according to the attribute features of the region to be identified and its context region.

[0175] It can be seen from the description of step 304 that the context features of the region to be identified incorporate multiple aspects of information, including the attribute features of the region to be identified itself and the attribute features of its context region. This processing method enables the context features of the region to be identified to reflect the logical coherence between the region to be identified and its context region, which helps to accurately grasp the overall layout and data flow of the table, thereby helping to accurately identify the structure of the table.

[0176] In one embodiment, the specific implementation of generating the context feature of the area to be identified based on the attribute features of the area to be identified and its context area includes: splicing the attribute features of the area to be identified and its context area in a set order, and determining the splicing result as the context feature of the area to be identified.

[0177] Optionally, the setting order is, for example: attribute features of the region to be identified, attribute features of the i-th region to be identified in the context region, wherein i is an integer in the interval [1, s], and s is the number of regions to be identified contained in the context region.

[0178] This embodiment obtains the context features of the area to be identified by splicing the attribute features of the area to be identified and its context area. While satisfying the requirement that the context features of the area to be identified can reflect the logical coherence between the area to be identified and its context area, the complexity of the context features can be reduced because no additional processing steps are required.

[0179] In another embodiment, the specific implementation of generating the context features of the area to be identified based on the respective attribute features of the area to be identified and its context area includes: splicing the respective attribute features of the area to be identified and its context area and the initial structure category recognition results in a set order, and determining the splicing result as the context features of the area to be identified.

[0180] Optionally, the setting order is, for example: attribute features of the area to be identified, initial structural category recognition result of the area to be identified, attribute features of the i-th area to be identified in the context area, initial structural category recognition result of the i-th area to be identified in the context area, where i is an integer in the interval [1, s], and s is the number of areas to be identified contained in the context area.

[0181] This embodiment obtains the context features of the area to be identified by splicing the attribute features of the area to be identified and its context area and the corresponding initial structure category recognition results. While satisfying the requirement that the context features of the area to be identified can reflect the logical coherence between the area to be identified and its context area, the information content of the context features is increased by introducing the initial structure category recognition results. More information helps to improve the accuracy and robustness of recognition. Therefore, it has strong adaptability and stability in complex and changeable scenarios.

[0182] In addition, in order to improve the effective utilization of context features in the recognition task, thereby improving the accuracy of the final recognition result, the following embodiments are provided:

[0183] In one embodiment provided in the present application, before the attribute features of the area to be identified and its context area and the initial structure category recognition results are spliced ​​in a set order, the following processing is performed on the initial structure category recognition results of the area to be identified and its context area respectively: the weight of the initial structure category recognition result is determined according to the output information of the first classification model; the initial structure category recognition result is weighted using the weight; and the initial structure category recognition results after each weighted processing are used to perform the step of splicing the attribute features of the area to be identified and its context area and the initial structure category recognition results in a set order.

[0184] The weight of the initial structure category recognition result can be a value in the form of probability, which can be used to represent the confidence of the initial structure category recognition result. The weighting process can be to combine the weight with the initial structure category recognition result through a multiplication operation to obtain the initial structure category recognition result after weighting process. Afterwards, using the initial structure category recognition result after weighting process to generate context features can make the initial structure category recognition result with high reliability occupy a larger proportion in the context features, thereby improving the effective utilization of the context features in the recognition task and improving the accuracy of the final recognition result.

[0185] Among them, as an optional implementation method, when determining the weight of the initial structure category recognition result based on the output information of the first classification model, the correspondence between the preset numerical range and the weight can be searched according to the output information of the first classification model, and the weight corresponding to the numerical range within which the output information falls can be determined as the weight corresponding to the initial structure category recognition result.

[0186] As another optional implementation, when determining the weight of the initial structure category recognition result according to the output information of the first classification model, the output information of the first classification model is substituted into a preset weight calculation formula to obtain the weight corresponding to the initial structure category recognition result. The weight calculation formula here is used to reveal the correlation between the output information of the first classification model and the weight, and the embodiment of the present application does not limit the specific form of the weight calculation formula.

[0187] Exemplarily, in the above two implementations, the output information of the first classification model follows a certain relationship with the weight. For example, the closer the output value of the first classification model is to the judgment threshold, such as 0.7 in the above example (the output value of the first classification model is greater than or equal to 0.7, indicating that the area to be identified belongs to the structural category corresponding to the first classification model, and less than 0.7, indicating that the area to be identified does not belong to the structural category corresponding to the first classification model), the smaller the weight is, which means that the initial structural category recognition result determined according to the output value of the first classification model is less reliable. The farther the output value of the first classification model is from the judgment threshold, the larger the weight is, which means that the initial structural category recognition result determined according to the output value of the first classification model is more reliable.

[0188] In another embodiment provided by the present application, when the attribute features of the area to be identified and its context area are spliced ​​in a set order to obtain the context features of the area to be identified, before the context features are input into the second classification model, it also includes: determining the weight of the initial structure category recognition result according to the output information of the first classification model, weighting the initial structure category recognition result using the weight, and using each weighted initial structure category recognition result to perform the step of inputting the context features into the second classification model. Here, using each weighted initial structure category recognition result to perform the step of inputting the context features into the second classification model means: inputting the weighted initial structure category recognition result and the context features into the second classification model together. This processing method is similar to the above-mentioned embodiment, and can make the initial structure category recognition result with high reliability occupy a larger proportion when the initial structure category recognition result is corrected, thereby improving the effective utilization of the initial structure category recognition result in the recognition task and improving the accuracy of the final recognition result.

[0189] Step 305: For each to-be-recognized region, determine the confidence level of the corresponding initial structure category recognition result according to its contextual features.

[0190] The confidence of the initial structure category recognition result is used to measure the reliability and credibility of the initial structure category recognition result. Optionally, the confidence of the initial structure category recognition result has a value range of [0,1], where the closer the confidence is to 0, the less credible the initial structure category recognition result is, and the closer the confidence is to 1, the more credible the initial structure category recognition result is; or, the closer the confidence is to 1, the less credible the initial structure category recognition result is, and the closer the confidence is to 0, the more credible the initial structure category recognition result is. The embodiments of the present application do not limit this.

[0191] In step 305, for each region to be identified, the confidence of the corresponding initial structural category identification result is determined according to its context features. Since the context features of the region to be identified integrate various information, including the attribute features of the region to be identified itself, the attribute features of its context area, and the corresponding initial structural category identification result, the context features of the region to be identified can reflect the logical coherence between the region to be identified and its context area, and this coherence helps to accurately grasp the overall layout and data flow of the table, thereby helping to accurately identify the confidence of the initial structural category identification result.

[0192] In one embodiment, the specific implementation of determining the confidence of the initial structural category recognition result of the area to be recognized based on the contextual features of the area to be recognized includes: inputting the contextual features of the area to be recognized and the initial structural category recognition result into a trained prediction model to obtain the confidence of the initial structural category recognition result of the area to be recognized. Here, in the case where the contextual features of the area to be recognized are spliced ​​with the initial structural category recognition result of the area to be recognized, only the contextual features of the area to be recognized can be input into the trained prediction model to obtain the confidence of the initial structural category recognition result of the area to be recognized.

[0193] Optionally, the prediction model adopts a random forest machine learning model, which is a classifier containing multiple decision trees. Exemplarily, a large amount of labeled data is obtained, which are context features labeled with confidence, and then the initial random forest machine learning model is trained using these labeled data until the model converges to obtain a trained prediction model. The input of the prediction model is the context features corresponding to the initial structure category recognition result, or includes the context features and the initial structure category recognition result, and the output is the confidence of the initial structure category recognition result.

[0194] Step 306: for each area to be identified, if the confidence level of the initial structural category identification result corresponding to the area to be identified determines that the initial structural category identification result is unreliable, the initial structural category identification result is corrected according to its contextual features to obtain the final structural category identification result of the area to be identified.

[0195] As an optional implementation, the confidence of the initial structure category recognition result is compared with a preset confidence threshold, and whether the initial structure category recognition result is credible is determined based on the comparison result. For example, when the confidence is set to be closer to 0, the initial structure category recognition result is less credible, and the confidence is closer to 1, the initial structure category recognition result is more credible, the confidence threshold is set to 0.7, and when the confidence of the initial structure category recognition result is greater than or equal to 0.7, the initial structure category recognition result is determined to be credible, and when the confidence of the initial structure category recognition result is less than 0.7, the initial structure category recognition result is determined to be uncredible.

[0196] For another example, when the confidence is set to be closer to 1, the less credible the initial structure category recognition result is, and the confidence is closer to 0, the more credible the initial structure category recognition result is, the confidence threshold is set to 0.4. When the confidence of the initial structure category recognition result is less than or equal to 0, the initial structure category recognition result is determined to be credible. When the confidence of the initial structure category recognition result is greater than 0.4, the initial structure category recognition result is determined to be uncredible.

[0197] Those skilled in the art will appreciate that the confidence thresholds described above are merely exemplary descriptions, and in actual applications, users can flexibly set or adjust the confidence thresholds according to business scenarios and business needs. By adjusting the confidence thresholds, the prediction sensitivity of the prediction model can be adjusted.

[0198] It can be seen from the description of step 306 that when the initial structure category recognition result of the area to be identified is determined to be unreliable based on the confidence level of the initial structure category recognition result, the initial structure category recognition result is corrected based on its contextual features to obtain the final structure category recognition result of the area to be identified.

[0199] In one embodiment, the initial structural category recognition result of the area to be identified is corrected according to its contextual features, and the specific implementation of obtaining the final structural category recognition result of the area to be identified includes: inputting the contextual features of the area to be identified into a trained second classification model, and determining the final structural category recognition result of the area to be identified based on the output information of the second classification model. Optionally, in the case where the contextual features of the area to be identified are composed of the splicing of the attribute features of the area to be identified and its context area, the initial structural category recognition results of the area to be identified and its context area can also be input into the second classification model. This processing method increases the amount of information input to the second classification model by introducing the initial structural category recognition result. The increase in the amount of information helps the second classification model to make more accurate judgments, thereby improving the accuracy and robustness of recognition.

[0200] It can be seen from this that when the initial structural category recognition result is unreliable, the structural category of the area to be identified is re-identified based on the contextual features of the area to be identified. Under this processing method, since the contextual features of the area to be identified are combined with the attribute features of its context area in addition to the attribute features of the area to be identified itself, the contextual features of the area to be identified can reflect the logical coherence between the area to be identified and its context area. This coherence helps to accurately grasp the overall layout and data flow of the table, thereby helping to accurately identify the structure of the table.

[0201] Figure 3 The process shown first determines the confidence of the initial structure category recognition result of the area to be identified. When it is determined that the initial structure category recognition result of the area to be identified is unreliable based on the confidence, the initial structure category recognition result is corrected according to its context features to obtain the final structure category recognition result of the area to be identified. On the one hand, the accuracy of the final structure category recognition result is ensured by utilizing the logical coherence between the area to be identified and its context area. On the other hand, unnecessary correction of recognition results can be avoided, computing resources can be saved, and recognition efficiency can be improved.

[0202] Figure 4 A flowchart of an embodiment of another method for identifying and correcting a table structure provided in an embodiment of the present application. Figure 4 The process shown is based on any of the above embodiments and describes a specific implementation of dividing the table to be identified into multiple areas to be identified according to the cell content of each cell in the table to be identified. Figure 4 As shown, the process includes the following steps:

[0203] Step 401: Obtain the cell content of each cell in the table to be identified, and generate cell feature information according to the cell content.

[0204] The table to be identified may include multiple cells, and each cell may contain its own cell content. The cell content may reflect the characteristics of the cell itself, and these characteristics together constitute the characteristic information of the cell. In one embodiment, the characteristic information of the cell represents the type of the cell content of the cell, and the type of the cell content is of auxiliary significance for dividing the area to be identified in the table to be identified. Specifically, the content of each cell in the area to be identified can be divided into Chinese, English, mathematics, date, time, blank and other types as the characteristic information of each cell in the area to be identified.

[0205] The cell content of each cell in the table to be identified refers to the content information of each independent cell in the table to be identified. Figure 1 From the description of step 101, it can be seen that the two common types of independent cells in a table are merged cells and minimum cells. A merged cell is a cell obtained by merging at least two minimum cells, and a minimum cell is a cell that cannot be split and does not participate in the merging of any cells. It can be seen that the cell content and feature information of each cell in the table to be identified can also include two situations: when a cell is a merged cell, the feature information of each minimum cell that constitutes the merged cell is used as the feature information of the merged cell, wherein when extracting the feature vector of the minimum cell (that is, the sub-cell) in the merged cell, the content in the merged cell is regarded as the cell content of the minimum cell in the merged cell; when a cell is an independent minimum cell, the feature information of the cell is the feature information of the independent minimum cell.

[0206] As a possible implementation, the correspondence between the characteristic information and the value can be preset to facilitate the description of the characteristics of each cell and generate the characteristic information of the cell. Exemplarily, the correspondence between the characteristic information and the preset value can be shown in Table 3 below:

[0207] Table 3

[0208] Feature information Preset value Chinese 1 English 2 number 3 date 4 time 5 blank 0

[0209] For example, if the content of cell A is Chinese, it means that the characteristic information of cell A is Chinese, that is, the characteristic information of cell A is 1. Similarly, if the content of cell B is blank, it means that the characteristic information of cell B is blank, that is, the characteristic information of cell B is 0. It can be seen that, in one embodiment, the characteristic information of the cell is generated according to the cell content of each cell in the table to be identified and the preset value in the above Table 2.

[0210] Step 402: Determine the similarity between any two adjacent rows or columns in the table to be identified based on the feature information of each cell in the table to be identified.

[0211] Based on the characteristic information of the cell generated in step 401, in a possible implementation, the similarity between any two adjacent rows or columns in the table to be identified can be calculated using the characteristic information of the cell. Wherein, if the table to be identified is a vertical table, the similarity between any two adjacent rows in the table to be identified is calculated; if the table to be identified is a horizontal table, the similarity between any two adjacent columns in the table to be identified is calculated.

[0212] In one embodiment, taking the calculation of the similarity between any two adjacent rows in the table to be identified as an example, the feature vector of each row can be obtained based on the feature information of the cells in each row of the table to be identified, and the similarity between any two adjacent rows or columns in the table to be identified can be determined based on the feature vector.

[0213] Optionally, the feature vector of each row of cells can be obtained based on the feature information of each minimum cell in each row. For example, assuming that there are 4 minimum cells in the first row of the table to be identified, and the feature information of the 4 minimum cells is 3, 4, 1, and 0 respectively, then the feature vector of the first row of the table to be identified is (3, 4, 1, 0). Similarly, assuming that there are 3 minimum cells in the first column of the table to be identified, and the feature information of the 3 minimum cells is 0, 3, and 5 respectively, then the feature vector of the first column of the table to be identified is (0, 3, 5).

[0214] Optionally, the amount of feature information of each minimum cell in each row may be made equal, which is beneficial to calculating the similarity between every two adjacent rows.

[0215] Based on the feature vectors of two adjacent rows or columns, the similarity of the two adjacent rows can be determined by calculating the distance between the feature vectors. The larger the distance, the greater the similarity, and the smaller the distance, the smaller the similarity. The above distance can be Euclidean distance or other distances. Of course, the similarity of each two adjacent rows or columns can also be calculated by other methods, and this application does not limit this.

[0216] For example, if the distance between the feature vectors of the first row and the second row in the table to be identified is 1, then the value of the similarity between the two adjacent rows is determined to be 1.

[0217] Step 403: divide two adjacent rows or two adjacent columns whose similarity reaches a set similarity threshold into a to-be-recognized area.

[0218] As a possible implementation, a similarity threshold may be preset, and then the area to be identified may be divided by judging the similarity between two adjacent rows or columns and the size of the similarity threshold. For example, the preset similarity threshold may be 0.6. When the similarity between two adjacent rows or columns is less than or equal to the similarity threshold, for example, less than or equal to 0.6, it indicates that the two adjacent rows or columns are similar; when the similarity between two adjacent rows or columns is greater than the similarity threshold, for example, greater than 0.6, it indicates that the two adjacent rows or columns are not similar.

[0219] Afterwards, the consecutive similar rows are divided into the same area to be identified. Figure 2 For example, in the table to be identified, if the similarity between any two adjacent rows is calculated to determine that the first row and the second row are not similar, and the second row and the third row are similar, the second row and the third row are divided into the same area to be identified. The first row is the first area to be identified in the table to be identified, and the area formed by the second row and the third row is the second area to be identified in the table to be identified.

[0220] Here is another point. In one embodiment provided by the present application, when a table to be identified includes a merged cell, the similarity between the row occupied by the merged cell and its adjacent rows is determined in the following manner, wherein the adjacent rows occupied by the merged cell refer to: the adjacent rows of the row area formed by the rows occupied by the merged cell: the following processing is performed for each occupied row of the merged cell: the feature vector of the occupied row is determined according to the feature information of each minimum cell in the occupied row; the similarity between the occupied row and its adjacent rows is determined according to the feature vector of the occupied row. Further, two adjacent rows whose similarity reaches a set similarity threshold are divided into an area to be identified, including: when the number of similarities between each occupied row of the merged cell and its adjacent rows reaches the set similarity threshold and reaches a first threshold, each occupied row of the merged cell and its adjacent rows are divided into an area to be identified. Wherein, the first threshold can be the number of sub-cells contained in the merged cell, or half of the number of sub-cells contained in the merged cell, or 1, which is not limited here.

[0221] For example, see Table 1 below, which is an example of a table to be identified:

[0222] Table 1

[0223]

[0224] The table to be identified shown in Table 1 includes a merged cell A. The rows occupied by the merged cell A include two rows, namely row 2 and row 3. Then the row area formed by the rows occupied by the merged cell A is the area composed of row 2 and row 3. According to the above description, the adjacent rows of the row occupied by the merged cell A include two rows, namely row 1 and row 4.

[0225] Assuming that the merged cell A is formed by merging the minimum cell A1 and the minimum cell A2, then, according to the feature information of the minimum cell A1, the minimum cell B1 and the minimum cell C1, the feature vector of the second row is determined, and according to the feature information of the minimum cell A2, the minimum cell B2 and the minimum cell C2, the feature vector of the third row is determined. In one embodiment provided by the present application, if the merged cell contains a minimum cell that does not contain content, such as A1 contains content and A2 does not contain content, the feature information of the cell containing content in the merged cell, such as the feature information of A1, is used as the feature information of the minimum cell contained in the merged cell. If the minimum cells contained in the merged cell all contain content, the feature information of the minimum cells contained in the merged cell is the feature information of each minimum cell containing content. Then, for the first row in the adjacent rows, the similarity between the second row and the first row is calculated according to the feature vector of the second row, and the similarity between the third row and the first row is calculated according to the feature vector of the third row. In the case where the first threshold is the number of sub-cells contained in the merged cell, if both similarities reach the set similarity threshold, the 1st row, the 2nd row, and the 3rd row are divided into one region to be identified. Similarly, for the 4th row in the adjacent rows, the similarity between the 2nd row and the 4th row is calculated based on the feature vector of the 2nd row, and the similarity between the 3rd row and the 4th row is calculated based on the feature vector of the 3rd row. If both similarities reach the set similarity threshold, the 4th row, the 2nd row, and the 3rd row are divided into one region to be identified.

[0226] In the case where a merged cell is formed by merging cells in at least two columns, the similarity between the column occupied by the merged cell and its adjacent columns is determined in the following manner, wherein the adjacent columns of the column occupied by the merged cell refer to: adjacent columns of the column area formed by the columns occupied by the merged cell: the following processing is performed for each occupied column of the merged cell: the characteristic vector of the occupied column is determined according to the characteristic information of each minimum cell in the occupied column; the similarity between the occupied column and its adjacent columns is determined according to the characteristic vector of the occupied column. Further, two adjacent columns whose similarity reaches a set similarity threshold are divided into a region to be identified, including: when the number of similarities between each occupied column of the merged cell and its adjacent columns reaches the set similarity threshold reaches a first threshold, each occupied column of the merged cell and its adjacent columns are divided into a region to be identified.

[0227] For specific explanations, please refer to the above-mentioned specific description of determining the similarity between the rows occupied by the merged cells and their adjacent rows, which will not be repeated here.

[0228] In another embodiment provided by the present application, in the case where a merged cell is included in a table to be identified, the similarity between the row occupied by the merged cell and its adjacent rows is determined in the following manner: the following processing is performed for each occupied row of the merged cell: the feature vector of the occupied row is determined according to the feature information of each minimum cell in the occupied row, wherein, in one embodiment provided by the present application, if the merged cell contains a minimum cell that does not contain content, such as A1 contains content and A2 does not contain content, then the feature information of the cell containing content in the merged cell, such as the feature information of A1, is used as the feature information of the minimum cell contained in the merged cell. According to the feature vector of the occupied row, the similarity between the occupied row and its adjacent rows is determined. Further, two adjacent rows whose similarity reaches a set similarity threshold are divided into a region to be identified, including: when the similarity between the occupied row and its adjacent rows reaches a set similarity threshold, the occupied row and its adjacent rows are divided into a region to be identified. If the minimum cells contained in the merged cell all contain content, the feature information of the minimum cells contained in the merged cell is the feature information of each minimum cell containing content.

[0229] Taking the table to be identified as illustrated in Table 1 above as an example, the table to be identified shown in Table 1 includes a merged cell A. The rows occupied by the merged cell A include two rows, namely row 2 and row 3. For row 2, its adjacent rows include two rows, namely row 1 and row 3. For row 3, its adjacent rows also include two rows, namely row 2 and row 3.

[0230] Then, for the second row, the similarity between the second row and the first row, and the similarity between the second row and the third row are calculated according to the feature vector of the second row. For the third row, the similarity between the third row and the second row (it is not necessary to repeat the calculation here), and the similarity between the third row and the fourth row are calculated according to the feature vector of the third row.

[0231] Assuming that it is calculated that the similarity between the second row and the first row reaches the set similarity threshold, and at the same time, it is calculated that the similarity between the third row and the fourth row reaches the set similarity threshold, then the second row and the first row are divided into one area to be identified, and the third row and the fourth row are divided into one area to be identified.

[0232] It can be seen from the above examples that different occupied rows of the same merged cell can also be divided into different areas to be identified, and there is no limitation here.

[0233] Figure 4 The process shown calculates the similarity between any two adjacent rows or columns in the table to be identified based on the cell content of each cell in the table to be identified, and divides the table to be identified into multiple areas to be identified according to the similarity. This makes it possible to make the cell content in each area to be identified have high internal similarity, while there are large differences between different areas to be identified, so that each area to be identified has a clear content boundary, which can facilitate subsequent data processing and identification of the structural category of the area to be identified.

[0234] Figure 5 This is a block diagram of an embodiment of a table structure recognition and correction device provided in an embodiment of the present application. Figure 5 As shown, the device comprises:

[0235] The region division module 51 is used to obtain a table to be identified, and divide the table to be identified into a plurality of regions to be identified according to the cell content of each cell in the table to be identified, wherein the region to be identified includes at least one row or at least one column in the table to be identified;

[0236] The feature extraction module 52 is used to perform the following processing for each of the regions to be identified: extracting the attribute features of the region to be identified, and obtaining the initial structural category identification result of the region to be identified based on the attribute features and the first classification model;

[0237] The feature extraction module 52 is further used to obtain attribute features of a context area of ​​each of the to-be-recognized areas; wherein the context area includes at least one other to-be-recognized area adjacent to the to-be-recognized area;

[0238] The correction module 53 is used to correct the initial structure category recognition result for each of the regions to be recognized in combination with the attribute features of the context region of the region to be recognized, so as to obtain a final structure category recognition result of the region to be recognized.

[0239] In a possible implementation, the correction module 53 includes:

[0240] A context feature extraction unit, used for generating context features of the area to be identified based on the attribute features of the area to be identified and its context area;

[0241] A confidence prediction unit, used to determine the confidence of the initial structure category recognition result according to the context feature;

[0242] A correction unit is used to correct the initial structure category recognition result according to the context feature when it is determined that the initial structure category recognition result is not credible according to the confidence level, so as to obtain a final structure category recognition result of the area to be recognized.

[0243] In a possible implementation, the confidence prediction unit is specifically used to:

[0244] The context features are input into a prediction model to obtain the confidence level of the initial structural class recognition result.

[0245] In a possible implementation manner, the correction unit is specifically configured to:

[0246] The context feature is input into a second classification model, and a final structural category recognition result of the to-be-recognized area is determined according to output information of the second classification model.

[0247] In a possible implementation, the context feature extraction unit is used to:

[0248] splicing the attribute features of the to-be-recognized area and its context area in a set order, and determining the splicing result as the context feature of the to-be-recognized area;

[0249] Alternatively, the attribute features of the to-be-recognized region and its context region and the initial structure category recognition results are spliced ​​in a set order, and the splicing result is determined as the context feature of the to-be-recognized region.

[0250] In a possible implementation, the device further includes:

[0251] A weighted processing module is used to perform the following processing on the initial structural category recognition results of the to-be-recognized area and its context area respectively before inputting the context features and the initial structural category recognition results into the second classification model:

[0252] Determining the weight of the initial structure category recognition result according to the output information of the first classification model;

[0253] Using the weights to perform weighted processing on the initial structure category recognition results;

[0254] The step of inputting the context features and the initial structure category recognition results into the second classification model is performed using the initial structure category recognition results after each weighted process.

[0255] In a possible implementation, the region division module 51 includes:

[0256] A cell feature generating unit, used for acquiring the cell content of each cell in the table to be identified, and generating feature information of the cell according to the cell content;

[0257] A similarity determination unit, configured to determine the similarity between any two adjacent rows or columns in the table to be identified based on the characteristic information of each cell in the table to be identified;

[0258] The division unit is used to divide two adjacent rows or two adjacent columns whose similarity reaches a set similarity threshold into a to-be-recognized area.

[0259] In a possible implementation, when the table to be identified includes a merged cell, the similarity between the row occupied by the merged cell and its adjacent rows is determined in the following manner, wherein the adjacent rows occupied by the merged cell refer to: adjacent rows of a row area formed by the rows occupied by the merged cell:

[0260] The following processing is performed for each row occupied by the merged cell:

[0261] Determining a characteristic vector of the occupied row according to characteristic information of each minimum cell in the occupied row;

[0262] Determining the similarity between the occupied row and its adjacent rows according to the feature vector of the occupied row;

[0263] Two adjacent rows whose similarity reaches the set similarity threshold are divided into a to-be-recognized area in the following way:

[0264] When the number of similarities between each occupied row of the merged cell and its adjacent rows reaches a set similarity threshold and reaches a first threshold, each occupied row of the merged cell and its adjacent rows are divided into a to-be-identified area.

[0265] In a possible implementation, when the table to be identified includes a merged cell, the similarity between the row occupied by the merged cell and its adjacent rows is determined in the following manner:

[0266] The following processing is performed for each row occupied by the merged cell:

[0267] Determining a characteristic vector of the occupied row according to characteristic information of each minimum cell in the occupied row;

[0268] Determining the similarity between the occupied row and its adjacent rows according to the feature vector of the occupied row;

[0269] Two adjacent rows whose similarity reaches the set similarity threshold are divided into a to-be-recognized area in the following way:

[0270] When the similarity between the occupied row and its adjacent rows reaches a set similarity threshold, the occupied row and its adjacent rows are divided into a to-be-recognized area.

[0271] In a possible implementation manner, the attribute features include at least two of the following attribute dimension features: position dimension, layout style dimension, and content type dimension.

[0272] like Figure 6 As shown, an embodiment of the present application provides a device, including a processor 111, a communication interface 112, a memory 113 and a communication bus 114, wherein the processor 111, the communication interface 112, and the memory 113 communicate with each other through the communication bus 114.

[0273] Memory 113, used for storing computer programs;

[0274] In one embodiment of the present application, the processor 111 is used to implement the table structure recognition and correction method provided by any one of the above method embodiments when executing the program stored in the memory 113, including:

[0275] Acquire a table to be identified, and divide the table to be identified into a plurality of areas to be identified according to the cell content of each cell in the table to be identified, wherein the area to be identified includes at least one row or at least one column in the table to be identified;

[0276] The following processing is performed for each of the areas to be identified:

[0277] Extracting attribute features of the area to be identified, and obtaining an initial structural category identification result of the area to be identified based on the attribute features and a first classification model;

[0278] Acquire attribute features of a context area of ​​the area to be identified; wherein the context area includes at least one other area to be identified that is adjacent to the area to be identified;

[0279] The initial structure category recognition result is corrected in combination with the attribute features of the context area of ​​the to-be-recognized area to obtain a final structure category recognition result of the to-be-recognized area.

[0280] An embodiment of the present application further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the method for identifying and correcting a table structure provided in any one of the aforementioned method embodiments are implemented.

[0281] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0282] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a general hardware platform, and of course, by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the relevant technology can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0283] It should be understood that the terms used herein are only for the purpose of describing specific example embodiments and are not intended to be limiting. Unless the context clearly indicates otherwise, the singular forms "one", "an" and "said" as used herein may also be meant to include plural forms. The terms "include", "comprise", "contain", and "have" are inclusive, and therefore specify the existence of stated features, steps, operations, elements and / or parts, but do not exclude the existence or addition of one or more other features, steps, operations, elements, parts, and / or combinations thereof. The method steps, processes, and operations described herein are not interpreted as necessarily requiring them to be performed in the specific order described or illustrated, unless the execution order is clearly indicated. It should also be understood that additional or alternative steps may be used.

[0284] The above description is only a specific implementation of the present application, so that those skilled in the art can understand or implement the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest range consistent with the principles and novel features applied for herein.

Claims

1. A method for identifying and correcting a table structure, characterized in that: The method comprises: Acquire a table to be identified, and divide the table to be identified into a plurality of areas to be identified according to the cell content of each cell in the table to be identified, wherein the area to be identified includes at least one row or at least one column in the table to be identified; The following processing is performed for each of the areas to be identified: Extracting attribute features of the area to be identified, and obtaining an initial structural category identification result of the area to be identified based on the attribute features and a first classification model; Acquire attribute features of a context area of ​​the area to be identified; wherein the context area includes at least one other area to be identified that is adjacent to the area to be identified; The initial structure category recognition result is corrected in combination with the attribute features of the context area of ​​the to-be-recognized area to obtain a final structure category recognition result of the to-be-recognized area.

2. The method according to claim 1, characterized in that The correcting the initial structure category recognition result by combining the attribute features of the context area of ​​the area to be recognized to obtain the final structure category recognition result of the area to be recognized includes: Generate context features of the area to be identified based on the attribute features of the area to be identified and its context area; Determining the confidence of the initial structure category recognition result according to the context feature; When it is determined that the initial structure category recognition result is not credible according to the confidence level, the initial structure category recognition result is corrected according to the context feature to obtain a final structure category recognition result of the to-be-recognized area.

3. The method according to claim 2, characterized in that The step of determining the confidence level of the initial structural category recognition result of the to-be-recognized area according to the context feature includes: The context features are input into a prediction model to obtain the confidence level of the initial structural class recognition result.

4. The method according to claim 2, characterized in that: The correcting the initial structure category recognition result according to the context feature to obtain the final structure category recognition result of the to-be-recognized area includes: The context feature is input into a second classification model, and a final structural category recognition result of the to-be-recognized area is determined according to output information of the second classification model.

5. The method according to claim 2, characterized in that: The step of generating the context feature of the area to be identified based on the attribute features of the area to be identified and its context feature includes: splicing the attribute features of the to-be-recognized area and its context area in a set order, and determining the splicing result as the context feature of the to-be-recognized area; Alternatively, the attribute features of the to-be-recognized region and its context region and the initial structure category recognition results are spliced ​​in a set order, and the splicing result is determined as the context feature of the to-be-recognized region.

6. The method according to claim 4, characterized in that Before inputting the context feature into the second classification model, the method further includes: The following processing is performed on the initial structural category recognition results of the to-be-recognized area and its context area respectively: Determining the weight of the initial structure category recognition result according to the output information of the first classification model; Using the weights to perform weighted processing on the initial structure category recognition results; The step of inputting the context features into the second classification model is performed using the initial structure category recognition results after each weighted process.

7. The method according to claim 1, characterized in that The step of dividing the table to be identified into a plurality of areas to be identified according to the cell content of each cell in the table to be identified includes: Acquire the cell content of each cell in the table to be identified, and generate characteristic information of the cell according to the cell content; Determine the similarity between any two adjacent rows or columns in the table to be identified based on the feature information of each cell in the table to be identified; Two adjacent rows or columns whose similarity reaches a set similarity threshold are divided into a region to be identified.

8. The method according to claim 7, characterized in that In the case where the table to be identified contains merged cells, the similarity between the row occupied by the merged cell and its adjacent rows is determined in the following manner, wherein the adjacent rows occupied by the merged cell refer to: adjacent rows of a row area formed by the rows occupied by the merged cell: The following processing is performed for each row occupied by the merged cell: Determining a characteristic vector of the occupied row according to characteristic information of each minimum cell in the occupied row; Determining the similarity between the occupied row and its adjacent rows according to the feature vector of the occupied row; Two adjacent rows whose similarity reaches the set similarity threshold are divided into a to-be-recognized area in the following way: When the number of similarities between each occupied row of the merged cell and its adjacent rows reaches a set similarity threshold and reaches a first threshold, each occupied row of the merged cell and its adjacent rows are divided into a to-be-identified area.

9. The method according to claim 7, characterized in that: When the table to be identified contains merged cells, the similarity between the row occupied by the merged cell and its adjacent rows is determined in the following manner: The following processing is performed for each row occupied by the merged cell: Determining a characteristic vector of the occupied row according to characteristic information of each minimum cell in the occupied row; Determining the similarity between the occupied row and its adjacent rows according to the feature vector of the occupied row; Two adjacent rows whose similarity reaches the set similarity threshold are divided into a to-be-recognized area in the following way: When the similarity between the occupied row and its adjacent rows reaches a set similarity threshold, the occupied row and its adjacent rows are divided into a to-be-recognized area.

10. The method according to claim 1, characterized in that The attribute features include at least two of the following attribute dimension features: position dimension, layout style dimension, and content type dimension.

11. A table structure recognition and correction device, characterized in that: The device comprises: A region division module, used for obtaining a table to be identified, and dividing the table to be identified into a plurality of regions to be identified according to the cell content of each cell in the table to be identified, wherein the region to be identified includes at least one row or at least one column in the table to be identified; A feature extraction module is used to perform the following processing for each of the regions to be identified: extracting attribute features of the region to be identified, and obtaining an initial structural category recognition result of the region to be identified based on the attribute features and a first classification model; The feature extraction module is further used to obtain attribute features of a context area of ​​each of the to-be-recognized areas; wherein the context area includes at least one other to-be-recognized area adjacent to the to-be-recognized area; The correction module is used to correct the initial structure category recognition result for each of the to-be-recognized regions in combination with the attribute features of the context region of the to-be-recognized region to obtain a final structure category recognition result for the to-be-recognized region.

12. A storage medium, characterized in that: The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the table structure recognition and correction method according to any one of claims 1 to 10.

Images