Data annotation, table information extraction method, device and electronic equipment

Abstract

The present disclosure provides a method, device and electronic device for data annotation and table information extraction, which relates to the fields of artificial intelligence and big data, especially to the fields of machine learning, deep learning, knowledge graphs, intelligent search and intelligent recommendation, and can be used in intelligent extraction scenarios of table information. The specific implementation scheme is: clustering based on the features of multiple rows and / or columns of cells in a table to determine candidate header cells in the table; and based on the candidate header cells, data annotation of the table to mark the header cells and non-header cells in the table as different data.

Description

Technical Field

[0001] The present disclosure relates to the fields of artificial intelligence and big data, and in particular to the fields of machine learning, deep learning, knowledge graphs, intelligent search, and intelligent recommendation, and can be used in scenarios where tabular information is intelligently extracted. Background Art

[0002] For humans, tables are an intuitive and efficient form of data representation, and are widely used in all kinds of documents. Tables contain a lot of information, which is worth exploring in depth. In addition, unlike free text, the data in tables is organized in a semi-structured form. Therefore, compared with extracting information from free text, the information extracted from tables is more reliable and has better data quality. Summary of the invention

[0003] The present disclosure provides a data annotation method, a table information extraction method, a model training method, and corresponding devices, equipment, storage media, and computer program products.

[0004] According to one aspect of the present disclosure, a data labeling method is provided, comprising: clustering based on features of multiple rows of cells and / or multiple columns of cells in a table to determine candidate header cells in the table; and labeling the table based on the candidate header cells to label the header cells and non-header cells in the table as different data.

[0005] According to another aspect of the present disclosure, a method for extracting table information is provided, comprising: determining a table to be processed; and extracting attributes and corresponding attribute value pairs in the table using a table information extraction model, wherein the table information extraction model is trained based on labeled data, and the labeled data is obtained by data labeling using the data labeling method in an embodiment of the present disclosure.

[0006] According to another aspect of the present disclosure, a model training method is provided, comprising: acquiring labeled data; and using the labeled data to train a corresponding table information extraction model, wherein the labeled data is obtained by data labeling using the data labeling method in an embodiment of the present disclosure.

[0007] According to another aspect of the present disclosure, a device for extracting table information is provided, including: a clustering module for clustering based on features of multiple rows of cells and / or multiple columns of cells in a table to determine candidate header cells in the table; and a data labeling module for labeling data on the table based on the candidate header cells to label the header cells and non-header cells in the table as different data.

[0008] According to another aspect of the present disclosure, a device for extracting table information is provided, comprising: a determination module, used to determine a table to be processed; and an extraction module, used to extract attributes and corresponding attribute value pairs in the table using a table information extraction model, wherein the table information extraction model is obtained by training based on labeled data.

[0009] The annotated data is obtained by annotating data using the data annotating method in the embodiment of the present disclosure.

[0010] According to another aspect of the present disclosure, a model training device is provided, including: an acquisition module for acquiring labeled data; and a training module for using the labeled data to train a corresponding table information extraction model, wherein the labeled data is obtained by data labeling using the data labeling method in the embodiment of the present disclosure.

[0011] According to another aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method described in an embodiment of the present disclosure.

[0012] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause the computer to execute the method according to the embodiment of the present disclosure.

[0013] According to another aspect of the present disclosure, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the method according to the embodiment of the present disclosure is implemented.

[0014] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0016] Figure 1 The system architecture suitable for the embodiments of the present disclosure is exemplified;

[0017] Figure 2 A flowchart of a data annotation method according to an embodiment of the present disclosure is exemplified;

[0018] Figure 3 The schematic diagram of the positioning table head according to the embodiment of the present disclosure is exemplarily shown;

[0019] Figure 4 A flowchart of a method for extracting table information according to an embodiment of the present disclosure is exemplified;

[0020] Figure 5 A flowchart of a model training method according to an embodiment of the present disclosure is exemplified;

[0021] Figure 6 A block diagram of a data annotation device according to an embodiment of the present disclosure is exemplarily shown;

[0022] Figure 7 A block diagram of a device for extracting table information according to an embodiment of the present disclosure is exemplarily shown;

[0023] Figure 8 A block diagram of a model training device according to an embodiment of the present disclosure is exemplified; and

[0024] Fig. 9 A block diagram of an electronic device for implementing an embodiment of the present disclosure is exemplarily shown. DETAILED DESCRIPTION

[0025] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0026] It should be understood that the information in the table can be extracted in the following manner.

[0027] (1) Extraction through manual labeling or configuration rules, that is, extraction through direct manual labeling or manual configuration of templates by observing the common characteristics of the data.

[0028] (2) Extraction through matching with external data sources, that is, extraction by comparing with external data sources such as a predetermined knowledge base.

[0029] (3) Extraction of tables in a specific format, which requires the table header to be in a specific position in the table.

[0030] (4) Extraction through supervised machine learning methods, that is, extraction through training models with labeled data.

[0031] (5) Extraction through unsupervised machine learning methods, that is, designing manual features and extracting through unsupervised methods such as clustering.

[0032] Among them, the above table information extraction method has the following defects.

[0033] The main disadvantages of method (1) are that, first, the cost of manual labeling is high, and second, the extraction is performed through configuration templates, which has poor generalization ability, can handle very limited scenarios, and is not transferable between table data.

[0034] The main disadvantage of method (2) is that it has high requirements for external data sources and requires a high coverage of external data sources. Therefore, it is only suitable for general scenarios such as general knowledge graphs, but not for non-general scenarios such as industry knowledge graphs. It is also not suitable for application scenarios with rich and diverse table headers or table headers containing short sentences.

[0035] The main disadvantage of method (3) is that it can only process tables with specific formats, and its generalization and transferability between table data are poor.

[0036] The main disadvantage of method (4) is that it requires a lot of manpower to label the data.

[0037] The main disadvantage of method (5) is that for different business scenarios or different data sets, the feature weights need to be manually tuned.

[0038] In addition, it should be understood that although the data in the table is organized in a semi-structured form, its structure and content vary greatly. In different business scenarios, or even in different data sets, the structure and content of the table data may be very different. For example, the data in Table 1 are mostly numbers, while the data in Table 2 are mostly long texts. The difference between the two is huge.

[0039] Therefore, how to develop a universal table information extraction method for effective low-cost migration, and effectively extract table information with the lowest possible manual intervention cost in different industry scenarios and even different data set scenarios is a major difficulty currently faced by related fields.

[0040] Table 1

[0041] Annual taxable income (dinar) tax rate 0-120000 0％ 120001-360000 20％ 360001-1440000 30％ 1,440,000 and above 35％

[0042] Table 2

[0043]

[0044] In addition, it should be understood that thanks to the development of artificial intelligence technology, natural language processing related technologies have made great progress in recent years. Traditional machine learning algorithms and the deep learning algorithms that have become popular in recent years have their own application scenarios. A major advantage of traditional machine learning algorithms is that the features they use are artificial features (i.e., features manually extracted / configured), which are more interpretable, easier to intervene, and more suitable for scenarios with obvious data features. Correspondingly, a major advantage of deep learning algorithms is that there is generally no need to design manual features, and the model itself is often an excellent feature extractor, making it more suitable for scenarios where data features are difficult to discover. It can be seen that the two have their own strengths, and in some task scenarios, the strengths of the two can be combined to better complete the task.

[0045] Therefore, the embodiments of the present disclosure provide a solution, including an automatic data annotation method based on clustering (a traditional machine learning method), a method for extracting table information based on a model, and a model training method based on machine-automatically annotated data (a deep learning method). Among them, the data annotation method provided by the embodiments of the present disclosure can replace manual annotation methods with machine-automatically annotated methods based on clustering methods to achieve data annotation, and provide annotated data for subsequent model training based on deep learning, thereby saving a lot of manual annotation costs. At the same time, this solution can also combine the strengths of traditional machine learning methods and deep learning methods to better complete the task.

[0046] The present disclosure will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0047] The system architecture of the related methods and devices suitable for the embodiments of the present disclosure is introduced as follows.

[0048] Figure 1 The system architecture suitable for the embodiment of the present disclosure is exemplified. It should be noted that: Figure 1 The examples shown are merely examples of system architectures to which the embodiments of the present disclosure may be applied, in order to help those skilled in the art understand the technical content of the present disclosure, but do not mean that the embodiments of the present disclosure may not be used in other environments or scenarios.

[0049] like Figure 1 As shown, the system architecture 100 in the embodiment of the present disclosure may include: terminal device 101, terminal device 102 and terminal device 103, and database 104. Among them, terminal devices 101 to 103 may include a client or a server (such as a cloud server).

[0050] The terminal device 101 can be used to execute the data annotation method provided by the embodiment of the present disclosure, and to deploy the data annotation device provided by the embodiment of the present disclosure to complete the data annotation for model training. The terminal device 102 can be used to execute the model training method provided by the embodiment of the present disclosure, and to deploy the model training device provided by the embodiment of the present disclosure to complete the training of the table information extraction model. The terminal device 103 can be used to execute the table information extraction method provided by the embodiment of the present disclosure, and to deploy the table information extraction device provided by the embodiment of the present disclosure to realize model-based table information extraction.

[0051] In addition, in the embodiments of the present disclosure, the data annotation device can be used to implement the corresponding data annotation method. The table information extraction device can be used to implement the corresponding table information extraction method. The model training device can be used to implement the corresponding model training.

[0052] In addition, in the embodiment of the present disclosure, the database 104 may be used to store the tables to be processed, and / or store the extracted table information, such as attributes, attribute values, and mapping relationships between attributes and corresponding attribute values.

[0053] It should be understood that Figure 1 The number of databases and terminal devices in the embodiment is only for illustration. Any number of databases and terminal devices may be provided according to implementation requirements.

[0054] The application scenarios of the relevant methods and devices suitable for the embodiments of the present disclosure are introduced as follows.

[0055] It should be understood that the relevant methods and devices provided in the embodiments of the present disclosure can be used for training table information extraction models, generating corresponding annotation data, and intelligent extraction scenarios of table information.

[0056] Exemplarily, the table information extraction method provided by the embodiment of the present disclosure can extract attributes and corresponding attribute values ​​(i.e., attribute-attribute value pairs) and their constraint information from semi-structured table data without relying on external data sources and annotation data.

[0057] As shown in Table 1, the following information can be extracted: when the attribute value corresponding to the attribute "taxable income for the whole year (dinar)" is "0-120000", the attribute value corresponding to the attribute "tax rate" is "0%"; when the attribute value corresponding to the attribute "taxable income for the whole year (dinar)" is "120001-360000", the attribute value corresponding to the attribute "tax rate" is "20%" and so on.

[0058] As shown in Table 2, it can be extracted that when the attribute value corresponding to the attribute "position" is "valve components", and the attribute value corresponding to the attribute "maintenance content" is "integrity and sealing", the attribute value corresponding to the attribute "process quality requirements" is "the valve should be removed and the shaft should be checked..." The extracted information means that the process quality requirements for the integrity and sealing of the valve components are that the valve should be removed and the shaft should be checked...

[0059] As shown in Table 2, it can also be extracted that: when the attribute value of the attribute "position" is "seal", the attribute value of the attribute "maintenance content" is "leakage", the attribute value of the attribute "process quality requirement" is "a 0.15MPa pressure test should be performed after maintenance without leakage...", and the extracted information means that the process quality requirement for sealing leakage is that a 0.15MPa pressure test should be performed after maintenance without leakage...

[0060] In addition, other information can be extracted from the table shown in Table 2, which will not be listed here one by one.

[0061] It can be seen that there are significant differences in the structure and content of Table 1 and Table 2. However, the table information extraction method provided by the embodiment of the present disclosure can be adaptively adjusted for different table extraction scenarios and can be applied to new data scenarios without much manual intervention and data annotation.

[0062] Furthermore, the method for extracting tabular information provided in the embodiments of the present disclosure can effectively help downstream nodes build various knowledge graphs (such as general knowledge graphs and industry knowledge graphs), and can then effectively help more downstream nodes complete corresponding downstream tasks based on the knowledge graphs, such as intelligent retrieval of information, intelligent recommendation, and intelligent question and answer, etc., and has broad application and implementation prospects.

[0063] According to an embodiment of the present disclosure, the present disclosure provides a data annotation method for generating annotated data required for training a table information extraction model.

[0064] Figure 2 The flowchart of the data labeling method according to the embodiment of the present disclosure is exemplified.

[0065] like Figure 2 As shown, the data labeling method 200 may include: operations S210 to S220.

[0066] In operation S210 , clustering is performed based on features of multiple rows of cells and / or multiple columns of cells in a table to determine candidate header cells in the table.

[0067] In operation S220, data is annotated on the table based on the candidate header cells, so as to annotate the header cells and non-header cells in the table as different data.

[0068] In operation S210, clustering may be performed based on row features of cells in multiple rows of a table to determine candidate column header cells in the table. Alternatively, in operation S210, clustering may be performed based on column features of cells in multiple columns of a table to determine candidate row header cells in the table. Alternatively, in operation S210, clustering may be performed based on row features of cells in multiple rows of a table and column features of cells in multiple columns of a table, respectively, to determine candidate header cells in the table. In this case, the determined candidate header cells may only include candidate column header cells, or may only include candidate row header cells, or may include both candidate column header cells and candidate row header cells.

[0069] In operation S220, data annotation may be performed on the table to be processed based on the candidate header cells determined in operation S210, so that different data may be respectively annotated for the header cells and non-header cells in the table. For example, the header cells may be annotated with 1, and the non-header cells may be annotated with 0.

[0070] Through the embodiments of the present disclosure, the structural features of the table can be adapted to the table, and the table of different types can be automatically labeled through an unsupervised clustering algorithm, thereby generating corresponding labeled data. The labeled data can be used for subsequent model training tasks.

[0071] It should be understood that compared with obtaining labeled data for model training through manual labeling, which requires a lot of manpower costs, the embodiments of the present disclosure can automatically generate labeled data through cell-based clustering, thereby saving a lot of manpower costs and improving the output efficiency of the model.

[0072] As an optional embodiment, clustering is performed based on features of multiple rows of cells and / or multiple columns of cells in a table to determine candidate header cells in the table, which may include the following operations.

[0073] For each row of cells in the M rows of cells in the table, clustering is performed based on row features of each cell in the row to determine a corresponding candidate column header cell.

[0074] For each column of N columns of cells in the table, clustering is performed based on the column features of each cell in the column to determine the corresponding candidate row header cell.

[0075] Adaptively, in this embodiment, data annotation of the table based on the candidate header cells may include the following operations.

[0076] In the case where the table is determined to be a row header table based on the candidate column header cells and the candidate row header cells, a first operation is performed, wherein the first operation includes: determining at least one header row and at least one non-header row of the table based on the candidate row header cells; uniformly marking each header cell in at least one header row as label one (such as a number), and uniformly marking each non-header cell in at least one non-header row as label two (such as another number).

[0077] As an optional embodiment, the table for the above operation is a table organized in a two-dimensional matrix form. In a table organized in a two-dimensional matrix form, there are no merged cells, only basic cells, and the position of each cell can be represented by a two-dimensional array. As shown in Table 3, the table is a table organized in a two-dimensional matrix form. The table can also be expressed in HTML format or other special structures.

[0078] Table 3

[0079] Project Name Construction Fee Information Administration Management System 1000

[0080] For example, Table 3 can be expressed as:

[0081]

[0082] Project NameConstruction Fee

[0083] Information Administration Management System1000

[0084]

[0085] For example, Table 3 can also be expressed as:

[0086] [(0, 0, 'Project Name'), (0, 1, 'Construction Fee'), (1, 0, 'Information Administration Management System'), (1, 1, 1000)]

[0087] It should be noted that, in the embodiment of the present disclosure, if the table to be processed is a table organized in the form of a two-dimensional matrix, the above-mentioned operation can be directly performed to automatically distinguish and mark the header cells and non-header cells in the table. If the table to be processed is not a table organized in the form of a two-dimensional matrix, for example, the table to be processed is a table as shown in Table 4, then the table to be processed can be preprocessed first, and then the above-mentioned operation can be performed on the processed table to automatically distinguish and mark the header cells and non-header cells in the table. Among them, in the preprocessing stage, the table to be processed can be processed into a table organized in the form of a two-dimensional matrix. Specifically, if the table to be processed contains merged cells, it is necessary to split the merged cells, that is, split them into basic cells, and fill the same content in each basic cell. In addition, blank rows or columns will be cleared.

[0088] For example, the table shown in Table 4 can be processed into the table shown in Table 5 in the preprocessing stage. In the preprocessing stage, the "project name" cell in the table is split into two basic cells with the same content, and the "construction fee" cell is split into five basic cells with the same content. In addition, the blank rows in Table 4 have been cleared in Table 5.

[0089] Table 4

[0090]

[0091] Table 5

[0092]

[0093] Among them, the table shown in Table 5 can be expressed as:

[0094] ['Project Name', 'Construction Fee', 'Construction Fee', 'Construction Fee', 'Construction Fee', 'Construction Fee']

[0095] ['Project Name', 'Development Fee', 'Implementation Fee', 'Integration Fee', 'Testing Fee', 'Technical Consulting Fee']

[0096] ['Marketing and Operation Support Professional Application V1.0 Construction', '12342', '2314', '2143', '2334', '0']

[0097] ['Informationized Administrative Management System', '888', '12333', '9876', '11132', '0']

[0098] ['Total', '123', '342', '2341', '432', '0']

[0099] Exemplarily, assuming that the table to be processed is a target table with M rows and N columns of cells, then the above operations can be performed on the target table to determine whether the target table contains candidate column header cells, and when it is determined that the table contains candidate column header cells, it can be further determined which candidate column header cells are contained in the table.

[0100] Similarly, the above operations may be performed on the target table to determine whether the target table contains candidate row header cells, and if it is determined that the table contains candidate row header cells, it may be further determined which candidate row header cells are contained in the table.

[0101] Among them, if the obtained candidate column header cell set is an empty set, and the obtained candidate row header cell set is a non-empty set, it means that the target table only contains row header cells, but no column header cells. In this case, the above-mentioned first operation can be performed, that is, based on the candidate row header cells, at least one header row and at least one non-header row of the table are determined, and each header cell in the at least one header row is uniformly marked as a number (such as 1), and each non-header cell in the at least one non-header row is uniformly marked as another number (such as 0).

[0102] Further, in the disclosed embodiment, for a candidate row header cell, if and only if the proportion of cells in a row hitting the candidate row header cell is greater than a threshold, the row can be considered to be a row header cell. For example, for the above Table 5, when the proportion of cells in row 0 hitting the candidate row header cell is greater than 60%, the row can be considered to be a row header cell.

[0103] Through the embodiments of the present disclosure, based on an unsupervised clustering algorithm, the structural characteristics of the table itself can be adapted to automatically distinguish and label the header cells and non-header cells of the table, thereby reducing manual intervention as much as possible, thereby saving manual labeling costs and improving the output efficiency of the model.

[0104] As an optional embodiment, data annotation of a table based on candidate header cells may include: when the table is determined to be a column header table based on the candidate column header cells and the candidate row header cells, performing a second operation, wherein the second operation includes: determining at least one header column and at least one non-header column of the table based on the candidate column header cells; uniformly annotating each header cell in at least one header column as annotation three (such as a number), and uniformly annotating each non-header cell in at least one non-header column as annotation four (such as another number).

[0105] Alternatively, as another optional embodiment, the method may further include: when it is determined based on the candidate column header cell and the candidate row header cell that the table includes both a row header and a column header, performing at least one of the following operations.

[0106] In response to determining that the contents of the columns in the table are similar, a first operation is performed.

[0107] In response to determining that the contents of the rows in the table are similar, a second operation is performed.

[0108] In response to determining that the similarity of the content of each row in the table is consistent with the similarity of the content of each column, no data annotation is performed on the table.

[0109] In one embodiment of the present disclosure, if the calculated candidate row header cell set is an empty set, and the calculated candidate column header cell set is a non-empty set, it means that the target table only contains column header cells, and does not contain row header cells. In this case, the second operation described above can be performed, that is, based on the candidate column header cells, at least one header column and at least one non-header column of the table are determined, and each header cell in the at least one header column is uniformly marked as a number (such as 1), and each non-header cell in the at least one non-header column is uniformly marked as another number (such as 0).

[0110] In the embodiment of the present disclosure, for a candidate list header cell, if and only if the proportion of cells in a column that hit the candidate list header cell is greater than a threshold, the column can be considered to be a list header cell.

[0111] In another embodiment of the present disclosure, if the calculated candidate row header cell set is a non-empty set, and the calculated candidate column header cell set is also a non-empty set, it means that the target table contains both row header cells and column header cells. In this case, it can be determined first whether the contents in the table are more similar in columns or rows, and then the first operation or the second operation described above can be performed according to the determination result.

[0112] It should be understood that if the contents in the table are more similar in columns, it can be considered that the table should be a row header table. Similarly, if the contents in the table are more similar in rows, it can be considered that the table should be a column header table.

[0113] In addition, if it is determined that the similarity of the contents in the table in columns is consistent with the similarity in rows, it can be determined that the table header cannot be accurately selected according to the method provided in the embodiment of the present disclosure, and therefore data annotation is not performed on such a table.

[0114] Through the embodiments of the present disclosure, different operations can be performed for tables of different header types to flexibly and accurately locate the position of the header, and then data can be accurately and distinguishably marked for header cells and non-header cells.

[0115] Further, as an optional embodiment, the similarity of the contents of each row and the similarity of the contents of each column in the table may be determined through the following operations.

[0116] Determines the average distance within a table's rows.

[0117] Determine the average distance within the columns of a table.

[0118] Compare the average distance within the row and the average distance within the column to determine the similarity of the content of each row and the similarity of the content of each column in the table.

[0119] In an embodiment of the present disclosure, for a table having both row headers and column headers, the average distance within the rows and the average distance within the columns of the table can be calculated, and the sizes of the average distance within the rows and the average distance within the columns can be compared to assist in determining whether the contents in the table are more similar in columns or in rows.

[0120] For example, the average distance within a row and the average distance within a column of a table may be determined by the following operations:

[0121] In the row direction, the row features of each cell in the table can be obtained. Then, for each row of cells, k-means clustering can be performed on each row of data based on the row features of each cell in the row. If the number of clusters is set to 2, the intra-cluster distance of the cluster with the smaller average distance from the cluster element to the cluster center can be taken as the intra-row distance of the row. Repeating the above operation can obtain the intra-row distance vector corresponding to each row of cells in the entire table as

[0122] Similarly, in the column direction, the column features of each cell in the table can be obtained, and then for each column of cells, k-means clustering can be performed on each column of data based on the column features of each cell in the column. If the number of clusters is set to 2, the intra-cluster distance of the cluster with the smaller average distance from the cluster elements to the cluster center can be taken as the intra-column distance of the column. Repeating the above operation can obtain the intra-column distance vector corresponding to each column of cells in the entire table as

[0123] Furthermore, after removing the maximum and minimum values ​​of the intra-row distances in the intra-row distance vector of the table, the average value of the remaining intra-row distances can be obtained, and the average value is used as the intra-row average distance of the table, denoted as Similarly, the average distance within the column of the table can be calculated

[0124] Among them, if It can be assumed that the table contents are more similar in rows and the table should have column headers. It can be assumed that the table contents are more similar in columns and the table should have row headers. It is considered that the similarity of the table content in the rows is consistent with its similarity in the columns. In this case, the table header cannot be correctly selected using the method provided by the embodiment of the present disclosure, so the table is not labeled with data in this case.

[0125] Through the embodiments of the present disclosure, it is possible to accurately determine whether the contents in a table are more similar in columns or rows.

[0126] In addition, as an optional embodiment, clustering is performed based on the row features of each cell in the row to determine the corresponding candidate column header cell, which may include: performing the following operations for each row of cells.

[0127] Clustering is performed based on the row features of each cell in the row, and two clusters are obtained.

[0128] At least one cell corresponding to the cluster with the smaller number of elements in the two clusters is used as a candidate list header cell of the row.

[0129] Alternatively, as an optional embodiment, clustering is performed based on row features of cells in a row to determine corresponding candidate column header cells, including: performing the following operations for each row of cells.

[0130] Clustering is performed based on the row features of each cell in the row, and two clusters are obtained.

[0131] The cell corresponding to the cluster with fewer elements and no elements in the two clusters that hits the blacklist is used as the candidate header cell of this row, where the blacklist records specific cell features. For example, for Table 1, the "number + text" feature can be recorded in the blacklist to exclude the possibility that the "1440000 or more" cell is identified as the header cell.

[0132] Similarly, candidate row header cells may also be determined based on a similar method, which will not be described in detail in this embodiment.

[0133] It should be understood that if a column header is in column form, the attribute value under the attribute corresponding to a column header will be in row form. Similarly, if a row header is in row form, the attribute value under the attribute corresponding to a row header will be in column form.

[0134] Exemplarily, the column header may be calculated by the following operations.

[0135] (1-1) Calculate the row features of each cell in each row of the table and obtain the feature vector corresponding to each row of the table

[0136] (1-2) Perform k-means clustering on each element in the feature vector corresponding to each row in (1-1). If the number of clusters is set to 2, the cell corresponding to a cluster that does not hit the configuration blacklist (i.e., an exception, such as "above 1440000" in Table 1, which contains both numbers and non-numbers) and contains a small number of elements can be recorded as the candidate list header cell;

[0137] (1-3) Repeat the above two steps, that is, repeat the above (1-1) and (1-2) until all possible candidate column header cells in all rows of the table are calculated;

[0138] (1-4) Among all candidate list header cells, a column can be considered as a list header if and only if the proportion of cells in a column that hit the candidate list header cells is greater than a threshold.

[0139] Similarly, the row header can be calculated by the following operations.

[0140] (2-1) Calculate the column features of each cell in each column of the table and obtain the feature vector corresponding to each column of the table

[0141] (2-2) Perform k-means clustering on each element in the feature vector corresponding to each column in (2-1). If the number of clusters is set to 2, the cell corresponding to a cluster that does not hit the configuration blacklist and contains a small number of elements can be recorded as a candidate row header cell;

[0142] (2-3) Repeat the above two steps, that is, repeat the above (2-1) and (2-2) until all possible candidate row header cells in all columns of the table are calculated;

[0143] (2-4) Among all candidate row header cells, if and only if the proportion of cells in a row that hit candidate row header cells is greater than a threshold, the row can be considered to be a row header.

[0144] Through the embodiments of the present disclosure, the row headers and / or column headers in the table can be simply and accurately located, thereby achieving accurate data labeling.

[0145] Among them, it can be based on Figure 3The basic principle shown is used to determine the header in the table to be processed. That is, feature calculation is performed first, then feature clustering is performed, and then the candidate header cells are calculated based on a cluster with fewer elements. For example, the candidate header cell determined by the left column of the table in the figure is the cell corresponding to "Annual Taxable Income (Dinar)", that is, the cell corresponding to row 0 and column 0. For example, the candidate header cell determined by the right column of the table in the figure is the cell corresponding to "Tax Rate", that is, the cell corresponding to row 0 and column 1. Further, it can be determined that the header of the table shown in the figure is in row 0.

[0146] As an optional embodiment, the above-mentioned features (which may be row features or column features) may include: at least one of: data features, statistical features, and semantic features. Further, as an optional embodiment, the statistical features may include: word frequency statistical features.

[0147] It should be understood that since the word segmentation statistical features, part-of-speech statistical features, and character statistical features may cause the length of the feature vector to change due to different data content, and considering that model training usually requires specific vectors of fixed length, the word segmentation statistical features, part-of-speech statistical features, and character statistical features of the tabular data are not considered in this embodiment.

[0148] In one embodiment, the features used in this solution may be composed of structural features (including data features and statistical features) and semantic features, wherein the feature granularity is the cell granularity.

[0149] Among them, in this embodiment, the structural feature can be a feature that can be obtained from the data itself or from a statistical perspective without understanding the meaning of the text, such as the length of the data, the frequency of data occurrence, etc. The semantic feature can be a feature containing the semantic information of the text. The features of texts with similar semantics will be closer in the feature space.

[0150] In this embodiment, both structural features and semantic features may be flexibly added or deleted according to different usage scenarios.

[0151] Exemplarily, structural features may include but are not limited to: whether there are numbers, whether it is pure numbers, whether there is Chinese, whether it is pure Chinese, whether there is English, whether it is pure English, the length of the data, the row where the data is located, the column where the data is located, the frequency of data occurrence, etc.

[0152] Taking the table shown in Table 1 as an example, the structural characteristics of the "tax rate" cell in row 0 and column 1 can be:

[0153] [0×w 是否有数字 ,0×w 是否纯数字 , 1×w 是否有中文 , 1×w 是否纯中文 ,0×w是否有英文 ,0×w 是否纯英文 , 2×w 数据的长度 ,0×w 数据所在行 , 1×w 数据所在列 , 1×w 数据出现频次 ].

[0154] In this embodiment, each structural feature has its own weight, and each weight can be initialized using empirical values ​​during the data annotation stage.

[0155] In addition, in this embodiment, for semantic features, the pre-trained language model can be used as a semantic feature extractor to extract semantic features. Semantic features also have their own weights, and their initial values ​​can be set according to the order of magnitude of the features.

[0156] In addition, as an optional embodiment, the method may also include: before clustering based on the features of multiple rows of cells and / or multiple columns of cells in a table, for the multiple rows of cells and / or multiple columns of cells, first replacing the numbers in each cell with a unified number, and then extracting the features of each cell in the row.

[0157] It should be noted that the features used in the embodiments of the present disclosure may be a combination of data features, statistical features and semantic features. Since the embodiments use an unsupervised clustering algorithm without a training process, it is very convenient to add and delete the features used.

[0158] Furthermore, in the embodiments of the present disclosure, data features refer to features of data itself, such as data length, etc. Statistical features refer to features presented by data in a group of data, such as the frequency of occurrence of data, etc.

[0159] In addition, corresponding weights can be set for each feature to facilitate fine-tuning for different tasks. In addition, when extracting features, for cells containing numbers, regardless of the size and length of the numbers, the numbers contained in the cells can be replaced with a uniform number, such as 1. This is because it can be considered that numbers are similar to each other, and in order to facilitate calculation, the numbers in all cells can be uniformly replaced with 1.

[0160] Exemplarily, digital features may include but are not limited to: whether there are numbers, whether it is pure numbers, whether there are Chinese characters, whether it is pure Chinese characters, whether there are English characters, whether it is pure English, data length, the row where the data is located, the column where the data is located, etc. Among them, "yes" can be represented by 1, and "no" can be represented by 0.

[0161] Exemplarily, the statistical features may include, but are not limited to, statistical features such as frequency of occurrence of data that do not cause a change in the length of a feature vector.

[0162] Exemplarily, taking the above Table 1 as an example, assuming that all feature weights are 1, calculating the column features of the "tax rate" cell in row 0 and column 1 may include the following steps. Calculating its data features yields: [0 (indicating no numbers), 0 (indicating not pure numbers), 1 (indicating Chinese), 1 (indicating pure Chinese), 0 (indicating no English), 0 (indicating not pure English), 2 (indicating data length is 2), 0 (indicating in row 0), 1 (indicating in column 1)]. Calculating its statistical features yields: [1 (indicating frequency of occurrence)]. The corresponding semantic features are extracted using a pre-trained semantic model. Finally, by combining the above data features, statistical features, and semantic features, the final column features of the "tax rate" cell can be obtained.

[0163] In addition, as an optional embodiment, the method may further include: after obtaining the annotation data about the table, based on the table characteristics, filtering the annotation data to filter out erroneous annotation data and retain correct annotation data.

[0164] Further, as an optional embodiment, the above table characteristics may include: a position characteristic of a table header in the table and / or a number of layers characteristic of the table header.

[0165] In the disclosed embodiment, the generated annotation data may be screened according to table characteristics to ultimately generate highly confident annotation data.

[0166] For example, the header often appears in the table at a more left or upper position, and the number of header layers is usually not too many. Therefore, according to the characteristics of the table, the header cells with high confidence can be screened out by identifying whether the header is in the first x rows or columns and whether the number of header layers is less than y. Among them, x and y can be empirical values, and the effect of the model trained in the training phase can be controlled by controlling x and y.

[0167] It should be understood that for an m×n table, m×n pieces of annotation data can be generated, that is, for each cell, one piece of annotation data can be obtained, and each piece of annotation data is composed of the features of the corresponding cell itself and the features of all elements in the cell. The label of the annotated data can be a binary classification label system of 0 or 1, where 0 means that the cell is not in the table header and 1 means that the cell is in the table header.

[0168] Taking the cell at row 0 and column 0 in Table 1 as an example, if there are both row headers and column headers, after identifying the header as row 0 of Table 1, the following annotation data can be obtained (where f(·) refers to the mapping from cell to feature):

[0169] Data:

[0170] [f(taxable income for the whole year (dinars)), f(tax rate), f(0-120000), f(0%), f(120001-360000), f(20%), f(360001-1440000), f(30%), f(above 1440000), f(35%)].

[0171] According to an embodiment of the present disclosure, the present disclosure provides a method for extracting table information based on deep learning.

[0172] Figure 4 The flowchart of the method for extracting table information according to the embodiment of the present disclosure is exemplified.

[0173] like Figure 4 As shown, the method 400 for extracting table information may include: operations S410 to S420.

[0174] Operation S410: determining a table to be processed.

[0175] In operation S420, the table information extraction model is used to extract attributes and corresponding attribute value pairs in the table.

[0176] Among them, the table information extraction model is obtained based on the training of the labeled data, and the labeled data is obtained by automatically labeling the data through the data labeling method provided in the above embodiment of the present disclosure, which will not be repeated here.

[0177] It should be understood that in this embodiment, the table is also a table organized in the form of a two-dimensional matrix. For a specific description, reference may be made to the relevant description in the above embodiment, which will not be repeated here.

[0178] In operation S420, the table obtained in operation S410 may be input into the above-mentioned table information extraction model, so as to output the attributes in the table and the corresponding attribute value pairs.

[0179] Exemplarily, the table information extraction model can be used to identify the table header, that is, traverse the cells in the table, and use the table information extraction model to predict whether each cell is in the header. If and only if the proportion of elements in a row / column in the header is greater than a threshold, the row / column is considered to be the header. Then, attribute-attribute value mapping is performed. For example, the table can be uniformly processed as a row header table, and all cells in the non-header area are traversed. Then, for each cell, the header is searched in the direction of decreasing row number to form an attribute-attribute value pair. In the case of multi-level headers, the attribute will be a list containing multiple values. As shown in Table 4 above, for the cell "12342" in the second row and the first column, searching the header upward will obtain: [Construction fee, development fee]: 12342.

[0180] Furthermore, for multi-level headers, after extracting the attribute-attribute value pairs, you can also perform attribute fusion on the attribute-attribute value pairs in the same row. For attribute-attribute value pairs between rows, if the attributes after fusion of each row are consistent, you can directly combine them; if the attributes after fusion of each row are inconsistent, you can further perform attribute fusion.

[0181] For example, as shown in Table 4, for the attribute-attribute value pairs "[construction fee, development fee]: 12342" and "[construction fee, implementation fee]: 2314", attribute fusion can obtain "[construction fee]: [[development fee]: 12342, [implementation fee]: 2314]". By direct combination, multiple results can be directly combined into a list.

[0182] For example, for the table shown in Table 1, by performing the above operations, the following attribute-attribute value pairs (i.e., mapping of attributes and corresponding attribute values) may be output:

[0183] {'Annual taxable income (dinar)': '0-120000', 'Tax rate': '0%'}

[0184] {'Annual taxable income (dinar)': '120001-360000', 'Tax rate': '20%'}

[0185] {'Annual taxable income (dinar)': '360001-1440000', 'Tax rate': '30%'}

[0186] {'Annual taxable income (dinar)': '1,440,000 and above', 'Tax rate': '35%'}.

[0187] Similarly, for multi-level headers, after extracting the attribute-attribute value pairs, you can also perform attribute fusion on the attribute-attribute value pairs in the same column. For attribute-attribute value pairs between columns, if the attributes after fusion of each column are consistent, you can directly combine them; if the attributes after fusion of each column are inconsistent, you can further perform attribute fusion.

[0188] In addition, the table information extraction technology described in the embodiments of the present disclosure aims to extract attributes and corresponding attribute value information from semi-structured table data.

[0189] The method implemented by the present disclosure does not require too much manual intervention. It is a comprehensive method that combines the advantages of unsupervised clustering methods that do not require labeled data and supervised methods that do not require manual adjustment of feature weights. It can use the structural characteristics of the table itself to extract information in various formats without manually labeled data and without relying on any external data source. It is adaptive in different business and data scenarios and can adaptively adjust feature weights without too much manual intervention. The method provided by the present disclosure has rich applicable scenarios, strong portability, and low labor costs.

[0190] In addition, using the method provided by the embodiment of the present disclosure, for new documents, new attributes, and new expressions, there is no need for excessive manual intervention, nor is there a need to formulate new extraction rules for the data. The algorithm provided by the embodiment of the present disclosure utilizes the structural characteristics of the table itself, and there is no need to manually annotate samples or manually adjust feature weights, and the cold start cost is low. In addition, the algorithm provided by the embodiment of the present disclosure does not rely on external data sources, which lowers the threshold for using the algorithm, and can handle a variety of table header scenarios, which is suitable for both general knowledge graph construction and industry knowledge graph construction. In addition, the algorithm provided by the embodiment of the present disclosure can handle a variety of table types, is flexible and changeable, has low dependence on the format of the table, and can handle multi-level header situations. In addition, the algorithm provided by the embodiment of the present disclosure has flexible usage scenarios, and can flexibly match and select features and models according to actual application scenarios.

[0191] According to an embodiment of the present disclosure, the present disclosure provides a model training method.

[0192] Figure 5 A flowchart of a model training method according to an embodiment of the present disclosure is exemplified.

[0193] like Figure 5 As shown, the model training method 500 may include: operations S510 to S520.

[0194] Operation S510: acquiring annotation data.

[0195] Operation S520, using the annotated data, training a corresponding table information extraction model, wherein the annotated data is obtained by automatically annotating data using the data annotation method provided by the embodiment of the present disclosure, which will not be repeated here.

[0196] In the disclosed embodiment, the annotated data generated by the aforementioned embodiment can be used to train a binary classifier to obtain a corresponding table information extraction model. The binary classifier can be selected freely according to actual hardware conditions, scene requirements, effect requirements, etc., and this embodiment does not limit it here.

[0197] In addition, in this embodiment, the design of the model structure can choose traditional machine learning algorithms such as random forest and xgboost, or can choose to build a binary classification deep learning model. The input of the model is the data part (i.e., the Data part) of the labeled data obtained in the above embodiment, and the output of the model is a 0 or 1 label.

[0198] According to an embodiment of the present disclosure, the present disclosure also provides a data labeling device.

[0199] Figure 6 A block diagram of a data annotation device according to an embodiment of the present disclosure is exemplarily shown.

[0200] like Figure 6 As shown, the data annotation device 600 may include: a clustering module 610 and a data annotation module 620 .

[0201] The clustering module 610 is used to perform clustering based on the features of multiple rows of cells and / or multiple columns of cells in the table to determine candidate header cells in the table.

[0202] The data labeling module 620 is used to label the table based on the candidate header cells, so as to label the header cells and non-header cells in the table as different data.

[0203] As an optional embodiment, the clustering module includes: a first clustering unit, used to cluster each row of cells in the M rows of cells in the table based on row features of each cell in the row, so as to determine the corresponding candidate column header cells; a second clustering unit, used to cluster each column of cells in the N columns of cells in the table based on column features of each cell in the column, so as to determine the corresponding candidate row header cells.

[0204] Correspondingly, the data labeling module is also used to: when the table is determined to be a row header table based on the candidate column header cell and the candidate row header cell, perform a first operation, wherein the first operation includes: determining at least one header row and at least one non-header row of the table based on the candidate row header cell; uniformly labeling each header cell in the at least one header row as label one, and uniformly labeling each non-header cell in the at least one non-header row as label two.

[0205] As an optional embodiment, the data labeling module is also used to: when the table is determined to be a column header table based on the candidate column header cell and the candidate row header cell, perform a second operation, wherein the second operation includes: determining at least one header column and at least one non-header column of the table based on the candidate column header cell; uniformly labeling each header cell in the at least one header column as label three, and uniformly labeling each non-header cell in the at least one non-header column as label four.

[0206] As an optional embodiment, the data labeling module is also used to perform at least one of the following operations when it is determined that the table includes both a row header and a column header based on the candidate column header cell and the candidate row header cell: in response to determining that the contents of the columns in the table are similar, perform the first operation; in response to determining that the contents of the rows in the table are similar, perform the second operation; in response to determining that the similarity of the contents of the rows in the table is consistent with the similarity of the contents of the columns, do not perform data labeling on the table.

[0207] As an optional embodiment, the similarity of the contents of each row and the similarity of the contents of each column in the table are determined by the following operations: determining the average distance within the row of the table; determining the average distance within the column of the table; and comparing the average distance within the row and the average distance within the column to determine the similarity of the contents of each row and the similarity of the contents of each column in the table.

[0208] As an optional embodiment, the first clustering unit includes: a first clustering sub-unit, used to cluster each row of cells based on row characteristics of each cell in the row to obtain two clusters; and a first determination sub-unit, used to select at least one cell corresponding to a cluster with a smaller number of elements in the two clusters as a candidate list header cell for the row.

[0209] As an optional embodiment, the first clustering unit includes: for each row of cells, a second clustering subunit is used to cluster based on the row features of each cell in the row to obtain two clusters; and a second determination subunit is used to use the cell corresponding to a cluster whose elements do not hit the blacklist and have a smaller number of elements as the candidate list header cell of the row, wherein the blacklist records specific cell features. For example, for Table 1, the "number + text" feature can be recorded in the blacklist to exclude the possibility that the "1440000 or more" cell is identified as a header cell.

[0210] As an optional embodiment, the feature includes at least one of the following: a data feature, a statistical feature, and a semantic feature.

[0211] As an optional embodiment, the statistical feature includes: word frequency statistical feature.

[0212] As an optional embodiment, the table is organized in a two-dimensional matrix form.

[0213] As an optional embodiment, the device also includes: a feature extraction module, which is used to replace the numbers in each cell with a unified number for the multiple rows of cells and / or the multiple columns of cells in a table before clustering based on the features of the multiple rows of cells and / or the multiple columns of cells in the table, and then extract the features of the multiple rows of cells and / or the multiple columns of cells.

[0214] As an optional embodiment, the device further includes: a screening module, which is used to screen the marked data based on table characteristics after obtaining the marked data about the table.

[0215] As an optional embodiment, the table characteristics include: a position characteristic of a table header in the table and / or a number of levels of the table header.

[0216] According to an embodiment of the present disclosure, the present disclosure also provides a device for extracting table information.

[0217] Figure 7 The block diagram of the apparatus for extracting table information according to the embodiment of the present disclosure is exemplified.

[0218] like Figure 7 As shown, the apparatus 700 for extracting table information may include: a determination module 710 and an extraction module 720 .

[0219] The determination module 710 is used to determine the table to be processed.

[0220] The extraction module 720 is used to extract the attributes and corresponding attribute value pairs in the table by using the table information extraction model, wherein the table information extraction model is obtained based on the training of the labeled data, and the labeled data is obtained by data labeling using the data labeling method described in the embodiment of the present disclosure.

[0221] According to an embodiment of the present disclosure, the present disclosure also provides a model training device.

[0222] Figure 8 A block diagram of a model training device according to an embodiment of the present disclosure is exemplarily shown.

[0223] like Figure 8 As shown, the model training device 800 may include: an acquisition module 810 and a training module 820.

[0224] The acquisition module 810 is used to acquire the annotation data.

[0225] The training module 820 is used to use the labeled data to train a corresponding table information extraction model, wherein the labeled data is obtained by data labeling using the data labeling method described in the embodiment of the present disclosure.

[0226] It should be understood that the embodiments of the device part of the present disclosure are the same or similar to the embodiments of the method part of the present disclosure, and the technical problems solved and the technical effects achieved are also the same or similar, and the present disclosure will not elaborate on them here.

[0227] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0228] Fig. 9 A schematic block diagram of an example electronic device 900 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0229] like Fig. 9 As shown, the electronic device 900 includes a computing unit 901, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 902 or a computer program loaded from a storage unit 908 into a random access memory (RAM) 903. In the RAM 903, various programs and data required for the operation of the electronic device 900 can also be stored. The computing unit 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0230] Multiple components in the electronic device 900 are connected to the I / O interface 905, including: an input unit 906, such as a keyboard, a mouse, etc.; an output unit 907, such as various types of displays, speakers, etc.; a storage unit 908, such as a disk, an optical disk, etc.; and a communication unit 909, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 909 allows the device 900 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0231] The computing unit 901 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 901 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 901 performs the various methods and processes described above, such as a method for extracting table information. For example, in some embodiments, the method for extracting table information may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 908. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 900 via the ROM 902 and / or the communication unit 909. When the computer program is loaded into the RAM 903 and executed by the computing unit 901, one or more steps of the method for extracting table information described above may be performed. Alternatively, in other embodiments, the computing unit 901 may be configured to perform a method for extracting table information in any other appropriate manner (e.g., by means of firmware).

[0232] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0233] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0234] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0235] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0236] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0237] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs that run on the corresponding computers and have a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or a cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services ("Virtual Private Server", or "VPS" for short). The server may also be a server for a distributed system, or a server combined with a blockchain.

[0238] In the technical solution disclosed herein, the recording, storage and application of the table data involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0239] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0240] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A data annotation method, comprising: Clustering based on features of multiple rows of cells and / or multiple columns of cells in a table to determine candidate header cells in the table, including: clustering for each row of cells in the M rows of cells in the table based on row features of each cell in the row to determine a corresponding candidate column header cell; clustering for each column of cells in the N columns of cells in the table based on column features of each cell in the column to determine a corresponding candidate row header cell; and Based on the candidate header cells, data annotation is performed on the table, so that header cells and non-header cells in the table are annotated as different data; The step of labeling the table with data based on the candidate table header cells includes: In the case where the table is determined to be a row header table based on the candidate column header cells and the candidate row header cells, a first operation is performed; wherein the first operation includes: determining at least one header row and at least one non-header row of the table based on the candidate row header cells; uniformly marking each header cell in the at least one header row as label one, and uniformly marking each non-header cell in the at least one non-header row as label two; In the case where the table is determined to be a column header table based on the candidate column header cells and the candidate row header cells, a second operation is performed; wherein the second operation includes: determining at least one header column and at least one non-header column of the table based on the candidate column header cells; uniformly marking each header cell in the at least one header column as label three, and uniformly marking each non-header cell in the at least one non-header column as label four; In the case where it is determined based on the candidate column header cell and the candidate row header cell that the table includes both a row header and a column header, at least one of the following operations is performed: in response to determining that the contents of the columns in the table are similar, the first operation is performed; in response to determining that the contents of the rows in the table are similar, the second operation is performed; in response to determining that the similarity of the contents of the rows in the table is consistent with the similarity of the contents of the columns, no data annotation is performed on the table.

2. The method according to claim 1, wherein: The similarity of the contents of each row and each column in the table is determined by the following operations: determining an average distance within a row of the table; determining an average distance within a column of the table; as well as The average distance within the row and the average distance within the column are compared to determine the similarity of the contents of each row and the similarity of the contents of each column in the table.

3. The method according to claim 1, wherein: The clustering based on the row features of each cell in the row to determine the corresponding candidate column header cell includes: for each row of cells, Clustering is performed based on the row features of each cell in the row to obtain two clusters; and At least one cell corresponding to the cluster with the smaller number of elements in the two clusters is used as a candidate list header cell of the row.

4. The method according to claim 1, wherein: The clustering based on the row features of each cell in the row to determine the corresponding candidate column header cell includes: for each row of cells, Clustering is performed based on the row features of each cell in the row to obtain two clusters; and The cell corresponding to the cluster whose elements do not hit the blacklist and whose number of elements is smaller than that of the two clusters is used as the candidate list head cell of the row, wherein the blacklist records specific cell features.

5. The method according to claim 1, wherein: The feature includes at least one of the following: a data feature, a statistical feature, and a semantic feature.

6. The method according to claim 5, wherein: The statistical features include: word frequency statistical features.

7. The method according to claim 1, wherein: The table is organized in a two-dimensional matrix form.

8. The method according to claim 1, further comprising: Before clustering based on the features of multiple rows of cells and / or multiple columns of cells in a table, for the multiple rows of cells and / or the multiple columns of cells, First, the numbers in each cell are replaced with a uniform number, and then the features of each cell are extracted.

9. The method according to claim 1, further comprising: After obtaining the annotated data about the table, The annotated data is filtered based on table characteristics.

10. The method according to claim 9, wherein: The table characteristics include: a position characteristic of a table header in the table and / or a number of layers characteristic of the table header.

11. A method for extracting table information, comprising: Identify the forms to be processed; as well as Using the table information extraction model, extract the attributes and corresponding attribute value pairs in the table, where: The table information extraction model is trained based on the labeled data. The annotated data is obtained by annotating data using the method described in any one of claims 1 to 10.

12. A model training method, comprising: Get the labeled data; as well as Using the labeled data, a corresponding table information extraction model is trained. Wherein, the annotated data is obtained by data annotating using the method described in any one of claims 1 to 10.

13. A data annotation device, comprising: A clustering module, used for clustering based on the features of multiple rows of cells and / or multiple columns of cells in a table to determine candidate header cells in the table; as well as A data annotation module, used for annotating the table with data based on the candidate header cells, so as to annotate the header cells and non-header cells in the table with different data; Wherein, the clustering module includes: A first clustering unit, configured to cluster each row of cells in the M rows of cells of the table based on row features of each cell in the row, so as to determine a corresponding candidate column header cell; A second clustering unit is used to cluster each column of the N columns of cells in the table based on the column features of each cell in the column to determine a corresponding candidate row header cell; Wherein, the data annotation module is also used for: In the case where the table is determined to be a row header table based on the candidate column header cells and the candidate row header cells, a first operation is performed, wherein the first operation includes: determining at least one header row and at least one non-header row of the table based on the candidate row header cells; uniformly marking each header cell in the at least one header row as label one, and uniformly marking each non-header cell in the at least one non-header row as label two; In the case where the table is determined to be a column header table based on the candidate column header cells and the candidate row header cells, a second operation is performed; wherein the second operation includes: determining at least one header column and at least one non-header column of the table based on the candidate column header cells; uniformly marking each header cell in the at least one header column as label three, and uniformly marking each non-header cell in the at least one non-header column as label four; In the case where it is determined based on the candidate column header cell and the candidate row header cell that the table includes both a row header and a column header, at least one of the following operations is performed: in response to determining that the contents of the columns in the table are similar, the first operation is performed; in response to determining that the contents of the rows in the table are similar, the second operation is performed; in response to determining that the similarity of the contents of the rows in the table is consistent with the similarity of the contents of the columns, no data annotation is performed on the table.

14. The device according to claim 13, wherein: The similarity of the contents of each row and each column in the table is determined by the following operations: determining an average distance within a row of the table; determining an average distance within a column of the table; as well as The average distance within the row and the average distance within the column are compared to determine the similarity of the contents of each row and the similarity of the contents of each column in the table.

15. The device according to claim 13, wherein: The first clustering unit comprises: A first clustering subunit is used to cluster each row of cells based on row features of each cell in the row to obtain two clusters; and The first determining subunit is configured to use at least one cell corresponding to a cluster with a smaller number of elements in the two clusters as a candidate list header cell of the row.

16. The device according to claim 13, wherein: The first clustering unit comprises: A second clustering subunit is used to cluster each row of cells based on row features of each cell in the row to obtain two clusters; and The second determining subunit is used to select a cell corresponding to a cluster whose elements do not hit the blacklist and has a smaller number of elements in the two clusters as a candidate list head cell of the row, wherein the blacklist records specific cell features.

17. The device according to claim 13, wherein: The feature includes at least one of the following: a data feature, a statistical feature, and a semantic feature.

18. The device according to claim 17, wherein: The statistical features include: word frequency statistical features.

19. The device according to claim 13, wherein: The table is organized in a two-dimensional matrix form.

20. The apparatus of claim 13, further comprising: A feature extraction module is used for, before clustering based on the features of the multiple rows of cells and / or the multiple columns of cells in the table, First, the numbers in each cell are replaced with a uniform number, and then the features of each cell are extracted.

21. The apparatus according to claim 13, further comprising: A screening module is used to obtain the annotated data about the table. The annotated data is filtered based on table characteristics.

22. The device according to claim 21, wherein The table characteristics include: a position characteristic of a table header in the table and / or a number of layers characteristic of the table header.

23. A device for extracting table information, comprising: A determination module, used for determining the form to be processed; as well as The extraction module is used to extract the attribute and the corresponding attribute value pair in the table by using the table information extraction model, wherein: The table information extraction model is trained based on the labeled data. The annotated data is obtained by annotating data using the method described in any one of claims 1 to 10.

24. A model training device, comprising: An acquisition module is used to acquire annotation data; as well as A training module is used to train a corresponding table information extraction model using the labeled data. Wherein, the annotated data is obtained by data annotating using the method described in any one of claims 1 to 10.

25. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 12.

26. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-12.

27. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 12.

Images