Table information extraction method, device, equipment and storage medium

Abstract

The present application provides a table information extraction method, device, equipment and storage medium, wherein the method includes: performing entity classification on the cells of the table to be processed based on the pre-trained entity classification model to obtain the entity type of each cell in the table to be processed, the entity classification model obtains the entity type of the cell based on the positional relationship between each word segmentation element in the table to be processed, and according to the entity type of each cell, determining the relationship information of each cell, the relationship information is used to characterize the inclusion relationship between the cell and the associated cell, and extracting information from the table to be processed based on the relationship information of each cell to obtain the extraction result of the table to be processed. The method of the present application can be applied to complex and diverse table formats, has good versatility, and improves the accuracy of table information extraction while reducing implementation complexity.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a table information extraction method, device, equipment and storage medium. Background Art

[0002] Tabular information extraction is to extract tabular information into a form that is easy to process, such as key-value pairs or natural language. Since tables come in a variety of forms, how to efficiently extract data from tables of different forms has become an urgent problem to be solved.

[0003] The steps of extracting information from a table in the prior art include header detection and key-value pair extraction. First, the table header is detected to determine the header cell, and then the table is subjected to relation extraction based on a relation extraction model to extract the key-value pair information of the table.

[0004] However, current table information extraction technology is difficult to apply to all forms of tables. For more complex tables, there is a problem of low detection accuracy. Both its versatility and accuracy need to be improved. Summary of the invention

[0005] The purpose of the present application is to provide a table information extraction method, device, equipment and storage medium to address the deficiencies in the above-mentioned prior art, so as to solve the problem that the prior art is difficult to apply to all forms and has low detection accuracy for more complex forms.

[0006] To achieve the above objectives, the technical solutions adopted in this application are as follows:

[0007] In a first aspect, the present application provides a method for extracting table information, the method comprising:

[0008] Based on the pre-trained entity classification model, the cells of the table to be processed are classified to obtain the entity type of each cell in the table to be processed, wherein the entity classification model obtains the entity type of the cell based on the positional relationship between each word segmentation element in the table to be processed;

[0009] Determining, according to the entity type of each of the cells, relationship information of each of the cells, wherein the relationship information is used to characterize the inclusion relationship between the cell and an associated cell;

[0010] Information is extracted from the table to be processed according to the relationship information of each of the cells to obtain an extraction result of the table to be processed.

[0011] As an optional implementation, the entity classification model obtained by pre-training performs entity classification on the cells of the table to be processed to obtain the entity type of each cell in the table to be processed, including:

[0012] Performing word segmentation processing on the text of the table to be processed to obtain at least one word segmentation sequence and a mapping relationship table between each word segmentation element in the word segmentation sequence and a cell, wherein the word segmentation sequence includes a plurality of the word segmentation elements arranged in sequence;

[0013] Acquire coordinate information of each cell in the table to be processed, wherein the coordinate information of each cell includes: spatial coordinates of the cell and grid coordinates of the cell;

[0014] Determine, according to the word segmentation sequence and the mapping relationship table, a first code and a second code of each word segmentation element in the word segmentation sequence, wherein the first code is used to represent the position of the word segmentation element in the word segmentation sequence, and the second code is used to represent the identifier of the cell to which the word segmentation element belongs;

[0015] The word segmentation sequence, the coordinate information of each cell, the first code and the second code of each word segmentation element are input into the entity classification model, and the entity type of each cell is determined by the entity classification model.

[0016] As an optional implementation, the word segmentation processing is performed on the text of the table to be processed to obtain at least one word segmentation sequence and a mapping relationship table between each word segmentation element in the word segmentation sequence and a cell, including:

[0017] Segmenting the text of the cells of the table to be processed to obtain a cell list, and recording the mapping relationship between the identifier of each segmentation element in the cell list and the identifier of the cell;

[0018] The cell list is segmented based on a sliding window to obtain the at least one word segmentation sequence.

[0019] As an optional implementation, determining the first code and the second code of each word segmentation element in the word segmentation sequence according to the word segmentation sequence and the mapping relationship table includes:

[0020] Determine a first encoding of the word segmentation element according to the position of the word segmentation element in the word segmentation sequence;

[0021] Searching in the mapping relationship table according to the identifier of the word segmentation element to determine the target cell to which the word segmentation element belongs;

[0022] The spatial coordinates of the target cell are used as the second encoding of the word segmentation element.

[0023] As an optional implementation, the step of inputting the word segmentation sequence, the coordinate information of each cell, and the first code and the second code of each word segmentation element into the entity classification model, and determining the entity type of each cell by the entity classification model includes:

[0024] The word segmentation sequence, the coordinate information of each cell, and the first code and the second code of each word segmentation element are input into the entity classification model, and the entity classification model performs the following steps:

[0025] Combining the word segmentation elements in the word segmentation sequence in pairs to obtain multiple word segmentation element groups;

[0026] Determine a score for each segmentation element group according to the first code and the second code of each segmentation element in each segmentation element group, wherein the score is used to represent the position correlation degree between two segmentation elements in the segmentation element group;

[0027] Multi-classification processing is performed according to the scores of each word segmentation element group, the coordinate information of each cell and the word segmentation sequence to obtain the entity type of each cell.

[0028] As an optional implementation, determining the score of each word segmentation element group according to the first code and the second code of each word segmentation element in each word segmentation element group includes:

[0029] Determine a first score of the word segmentation element group according to the first code of each word segmentation element in the word segmentation element group, wherein the first score is used to represent the distance between two word segmentation elements in the word segmentation element group in the word segmentation sequence;

[0030] According to the second codes of each word segmentation element in the word segmentation element group, respectively determine a second score and a third score of the word segmentation element group, wherein the second score is used to represent the distance between two word segmentation elements in the word segmentation element group in the row direction spatial coordinates, and the third score is used to represent the distance between two word segmentation elements in the word segmentation element group in the column direction spatial coordinates;

[0031] The score of the word segmentation element group is determined according to the first score, the second score, and the third score.

[0032] As an optional implementation manner, determining the relationship information of each of the cells according to the entity type of each of the cells includes:

[0033] According to the entity type of each cell, determine the header type cell and the value cell in the table to be processed, wherein the header type cell includes: a keyword cell, a table name cell, a header cell, and a total item cell;

[0034] Partition the table to be processed according to the table name cells and the table header cells to obtain a plurality of table sub-areas;

[0035] Traverse each of the table sub-areas, and for the current table sub-area traversed, determine the inclusion relationship among the keyword cells, table name cells and header cells in the current table sub-area according to the entity type of each cell in the current table sub-area, and determine the inclusion relationship between the header type cells in the current table sub-area and the inclusion relationship between the value cells and the header cells according to the coordinate information of each of the cells in the current table sub-area and the entity type of each of the cells.

[0036] As an optional implementation, determining the inclusion relationship between the header type cells in the current table sub-region and the inclusion relationship between the value cells and the header cells according to the coordinate information of each cell in the current table sub-region and the entity type of each cell includes:

[0037] If the cell is a keyword cell, a table name cell, a table header cell or a total item cell, then an inclusion relationship between the cell and an adjacent table header cell is established;

[0038] If the cell is a value cell, an inclusion relationship between the cell and a header cell corresponding to the cell is established according to the coordinate information of the cell.

[0039] As an optional implementation manner, extracting information from the table to be processed according to the relationship information of each cell to obtain the extraction result of the table to be processed includes:

[0040] Traversing the cells in the table to be processed, and obtaining a target mapping relationship between a header type cell and a value cell according to the relationship information of each of the cells;

[0041] The texts of each cell in the table to be processed are arranged according to the target mapping relationship to obtain an extraction result of the table to be processed.

[0042] In a second aspect, the present application provides a table information extraction device, the device comprising:

[0043] A classification module, used to classify the cells of the table to be processed based on a pre-trained entity classification model to obtain the entity type of each cell in the table to be processed, wherein the entity classification model obtains the entity type of the cell based on the positional relationship between each word segmentation element in the table to be processed;

[0044] A determination module, used to determine the relationship information of each of the cells according to the entity type of each of the cells, wherein the relationship information is used to characterize the inclusion relationship between the cell and the associated cells;

[0045] The extraction module is used to extract information from the table to be processed according to the relationship information of each cell to obtain the extraction result of the table to be processed.

[0046] As a possible implementation manner, the classification module is specifically used for:

[0047] Performing word segmentation processing on the text of the table to be processed to obtain at least one word segmentation sequence and a mapping relationship table between each word segmentation element in the word segmentation sequence and a cell, wherein the word segmentation sequence includes a plurality of the word segmentation elements arranged in sequence;

[0048] Acquire coordinate information of each cell in the table to be processed, wherein the coordinate information of each cell includes: spatial coordinates of the cell and grid coordinates of the cell;

[0049] Determine, according to the word segmentation sequence and the mapping relationship table, a first code and a second code of each word segmentation element in the word segmentation sequence, wherein the first code is used to represent the position of the word segmentation element in the word segmentation sequence, and the second code is used to represent the identifier of the cell to which the word segmentation element belongs;

[0050] The word segmentation sequence, the coordinate information of each cell, the first code and the second code of each word segmentation element are input into the entity classification model, and the entity type of each cell is determined by the entity classification model.

[0051] As a possible implementation manner, the classification module is specifically used for:

[0052] Segmenting the text of the cells of the table to be processed to obtain a cell list, and recording the mapping relationship between the identifier of each segmentation element in the cell list and the identifier of the cell;

[0053] The cell list is segmented based on a sliding window to obtain the at least one word segmentation sequence.

[0054] As a possible implementation manner, the classification module is specifically used for:

[0055] Determine a first encoding of the word segmentation element according to the position of the word segmentation element in the word segmentation sequence;

[0056] Searching in the mapping relationship table according to the identifier of the word segmentation element to determine the target cell to which the word segmentation element belongs;

[0057] The spatial coordinates of the target cell are used as the second encoding of the word segmentation element.

[0058] As a possible implementation manner, the classification module is specifically used for:

[0059] The word segmentation sequence, the coordinate information of each cell, and the first code and the second code of each word segmentation element are input into the entity classification model, and the entity classification model performs the following steps:

[0060] Combining the word segmentation elements in the word segmentation sequence in pairs to obtain multiple word segmentation element groups;

[0061] Determine a score for each segmentation element group according to the first code and the second code of each segmentation element in each segmentation element group, wherein the score is used to represent the position correlation degree between two segmentation elements in the segmentation element group;

[0062] Multi-classification processing is performed according to the scores of each word segmentation element group, the coordinate information of each cell and the word segmentation sequence to obtain the entity type of each cell.

[0063] As a possible implementation manner, the classification module is specifically used for:

[0064] Determine a first score of the word segmentation element group according to the first code of each word segmentation element in the word segmentation element group, wherein the first score is used to represent the distance between two word segmentation elements in the word segmentation element group in the word segmentation sequence;

[0065] According to the coordinate information of each cell and the second code of each word segmentation element in the word segmentation element group, respectively determine the second score and the third score of the word segmentation element group, the second score is used to represent the distance between two word segmentation elements in the word segmentation element group in the row direction spatial coordinates, and the third score is used to represent the distance between two word segmentation elements in the word segmentation element group in the column direction spatial coordinates;

[0066] The score of the word segmentation element group is determined according to the first score, the second score, and the third score.

[0067] As a possible implementation manner, the determining module is specifically used for:

[0068] According to the entity type of each cell, determine the header type cell and the value cell in the table to be processed, wherein the header type cell includes: a keyword cell, a table name cell, a header cell, and a total item cell;

[0069] Partition the table to be processed according to the table name cells and the table header cells to obtain a plurality of table sub-areas;

[0070] Traverse each of the table sub-areas, and for the current table sub-area traversed, determine the inclusion relationship among the keyword cells, table name cells and header cells in the current table sub-area according to the entity type of each cell in the current table sub-area, and determine the inclusion relationship between the header type cells in the current table sub-area and the inclusion relationship between the value cells and the header cells according to the coordinate information of each of the cells in the current table sub-area and the entity type of each of the cells.

[0071] As a possible implementation manner, the determining module is specifically used for:

[0072] If the cell is a keyword cell, a table name cell, a table header cell or a total item cell, then an inclusion relationship between the cell and an adjacent table header cell is established;

[0073] If the cell is a value cell, an inclusion relationship between the cell and a header cell corresponding to the cell is established according to the coordinate information of the cell.

[0074] As a possible implementation manner, the extraction module is specifically used for:

[0075] Traversing the cells in the table to be processed, and obtaining a target mapping relationship between a header type cell and a value cell according to the relationship information of each of the cells;

[0076] The texts of each cell in the table to be processed are arranged according to the target mapping relationship to obtain an extraction result of the table to be processed.

[0077] In a third aspect, an embodiment of the present application further provides an electronic device comprising: a processor, a storage medium and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the storage medium via the bus, and the processor executes the machine-readable instructions to perform the steps of the table information extraction method as described in any one of the first aspects.

[0078] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the table information extraction method as described in any one of the first aspects are executed.

[0079] The beneficial effects of the present application are: by classifying entities of the cells of the table to be processed based on the entity classification model, and then extracting relationships based on the entity type of the cell, compared to the traditional key-value pair extraction scheme, the present application can be applied to complex table styles, and therefore can meet the needs of table extraction in more complex scenarios. And the entity classification model in the present application obtains the entity type of the cell based on the positional relationship between each word segmentation element in the cell, so it can distinguish fine-grained information, and can accurately identify the entity type of the cell for nested tables and multi-level tables, thereby improving the accuracy of subsequent table information extraction. Finally, the present application directly extracts information from the table to be processed based on the extracted relationship information. Compared with the method of using neural networks or large models in the prior art, the method of the present application can extract table information more directly and quickly, and has lower complexity. BRIEF DESCRIPTION OF THE DRAWINGS

[0080] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0081] Figure 1 A schematic diagram showing an application scenario provided by an embodiment of the present application is shown;

[0082] Figure 2 A flowchart of a table information extraction method provided in an embodiment of the present application is shown;

[0083] Figure 3 A flowchart for determining an entity type provided by an embodiment of the present application is shown;

[0084] Figure 4 A flowchart of performing word segmentation processing on a table to be processed provided by an embodiment of the present application is shown;

[0085] Figure 5 A flowchart of determining a first encoding and a second encoding of a word segmentation element provided by an embodiment of the present application is shown;

[0086] Figure 6 A flowchart for determining a cell entity type in an entity classification model provided in an embodiment of the present application is shown;

[0087] Figure 7 A flow chart of attention scoring by an entity classification model provided in an embodiment of the present application is shown;

[0088] Figure 8An overall flow chart of entity classification by an entity classification model provided in an embodiment of the present application is shown;

[0089] Fig. 9 A flowchart of determining the relationship information of cells provided by an embodiment of the present application is shown;

[0090] Fig.10 A flowchart of another method of determining the relationship information of cells provided in an embodiment of the present application is shown;

[0091] Fig.11 A flowchart of extracting table information based on relationship information provided by an embodiment of the present application is shown;

[0092] Fig.12 A schematic diagram of the structure of a table information extraction device provided in an embodiment of the present application is shown;

[0093] Fig.13 A schematic structural diagram of an electronic device 130 provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0094] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0095] It should be noted that the term "comprising" will be used in the embodiments of the present application to indicate the existence of the features declared thereafter, but does not exclude the addition of other features.

[0096] The current table information extraction technology is difficult to apply to all forms of tables. For more complex tables, there is a problem of low detection accuracy. Both its versatility and accuracy need to be improved.

[0097] Specifically, when performing table header detection, the commonly used method in the prior art is table detection at the row and column levels. However, when detecting tables with higher complexity such as multi-level tables and nested tables, the prior art cannot distinguish fine-grained information, and thus the accuracy of the table header detection results is not high.

[0098] On the other hand, when performing relationship extraction, graph neural networks, large model extraction and other solutions are often used in the existing technology. However, this method requires training on different types of table samples before performing relationship extraction. The predictive ability of the model depends on the selection of sample data. Samples need to be selected for training for a variety of table formats, which is relatively complex.

[0099] Based on this, the present application proposes a table information extraction method, which classifies entities in the cells of the table through a lightweight entity classification model, then extracts relationships from the cells based on the entity classification results, and finally extracts information from the table based on the extracted relationship information. Compared with the prior art, the present application can be applied to a variety of complex forms of table information extraction, and has good versatility. In addition, the present application extracts relationships based on entity types. Compared with the prior art that uses a large model for relationship extraction and requires a large amount of sample data for training, the method of the present application has lower complexity.

[0100] Figure 1 This is a schematic diagram of an application scenario of table information extraction given in this application. The method of this application can be applied to electronic devices, refer to Figure 1 After the user inputs the table that needs to be extracted information or the picture containing the table into the electronic device, the electronic device can perform entity classification and relationship extraction on the table, and extract information from the table based on the extracted relationship information to obtain an extraction result. The extraction result can be a data form that is easy to store, such as a key-value pair form or natural language, and the extraction result is stored in a preset location or output to the user.

[0101] Next, combine Figure 2 The table information extraction method of the present application is described. The execution subject of the method can be Figure 1 Electronic devices such as Figure 2 As shown, the method includes:

[0102] S201. Entity classification is performed on cells of a table to be processed based on a pre-trained entity classification model to obtain entity types of each cell in the table to be processed. The entity classification model obtains the entity type of the cell based on the positional relationship between each word segmentation element in the table to be processed.

[0103] As a possible implementation method, the text in each cell of the table to be processed can be segmented, and the text in the cell can be divided into at least one segmentation element. The entity classification model can obtain the entity type of the cell based on the positional relationship between the segmentation elements. Among them, the entity classification model can be a lightweight ser model. It should be noted that when performing the segmentation process to obtain at least one segmentation element, the cells to which each segmentation element belongs and the coordinate information of each cell can also be recorded, so that the entity classification model can determine the positional relationship between the segmentation elements based on the cells to which each segmentation element belongs and the coordinate information of each cell.

[0104] Among them, the positional relationship between the word segmentation elements includes: the word segmentation elements are in the same cell, the word segmentation elements are in the same column or the word segmentation elements are in the same row. The positional relationship between the word segmentation elements can be determined based on the spatial coordinates of the cells to which the word segmentation elements belong. If the spatial coordinates of the cells to which the word segmentation elements belong are the same, for example, the spatial coordinates of the cells to which the word segmentation elements "institution" and "code" in Table 1 are the same, then it can be determined that the word segmentation elements are in the same cell. If the horizontal coordinates of the spatial coordinates of the cells to which the word segmentation elements belong are the same, for example, the horizontal coordinates of the word segmentation elements "balance" and "credit" in Table 1 are the same, then it can be determined that the word segmentation elements are in the same row. If the vertical coordinates of the spatial coordinates of the cells to which the word segmentation elements belong are the same, for example, the vertical coordinates of the word segmentation elements "project" and "flow" in Table 1 are the same, then it can be determined that the word segmentation elements are in the same column.

[0105] Optionally, the basic information of the table to be processed includes: the number of grid rows, the number of grid columns, and the table image. The table to be processed can be a table or an image containing a table. When the table to be processed is in table form, the table to be processed can be converted into an image to obtain the image of the table to be processed. When the table to be processed is in image form, the table can also be extracted from the image to obtain the table to be processed in an editable table form.

[0106] The entity type includes at least one of the following: keyword cell, table name cell, table header cell, total item cell, and value cell. Keyword cell, table name cell, table header cell, and total item cell belong to the broad type of table header cells. Value cells include: cells with actual meaning and cells without actual meaning.

[0107] As shown in Table 1, this is an example of a table provided in the present application. The entity type of a cell is first described below in conjunction with Table 1.

[0108] Table 1 Table Example 1

[0109]

[0110] Keyword cells can be cells used to distinguish different tables during retrieval, such as "current ratio", "quick ratio", "debt ratio", "end of reporting period", "book balance", "year-end balance", "book value", "controlling person", "person in charge", etc. in Table 1. Table name cells can be cells that span a row of tables or have blank cells on the right side of the same row, such as "First Company's quarterly financial report" and "First Company's bills for credit loss provision" in Table 1.

[0111] The header cell can be a cell with header type to the right and below in the first column, such as "Project" and "Category" in Table 1. The total item cell can be "Total" in Table 1. The value cell can be a cell other than the header type cell, among which the cell with actual meaning can be a cell with actual value, such as "1.31", "1.22", "14.10%" in Table 1, etc. The cell without actual meaning can be a cell without actual value, such as "-" in Table 1.

[0112] S202: Determine the relationship information of each cell according to the entity type of each cell, where the relationship information is used to represent the inclusion relationship between the cell and the associated cells.

[0113] Optionally, the associated cell may be a cell that meets preset requirements for the entity type and / or coordinate position of the current cell. The cell relationship information is used to indicate at least one associated cell of the current cell and characterize the containment relationship between the current cell and each associated cell. The containment relationship may be a containment relationship on the entity type and / or a containment relationship on the coordinate position, for example, there is a containment relationship between the table name cell and the adjacent header cell below it, there is a containment relationship between the header cell and the header cells above and below it, there is a containment relationship between the value cell and the header cells to its nearest left and above, there is a containment relationship between the header cell and other header cells adjacent to the header cell and whose coordinate range includes the header cell, etc.

[0114] Taking the current cell as a table header cell as an example, the cell associated with the current cell may be a table name cell adjacent to the current cell. There is an entity type inclusion relationship between the table name cell and the table header cell. For example, the table name cell "First Company Quarterly Financial Report" in Table 1 and the table header cells below it, such as "Project", "End of Reporting Period", "End of Previous Year", "Growth Rate" and "Remarks", have an entity type inclusion relationship.

[0115] Cells can also have a containment relationship in terms of position coordinates. For example, the "Book Balance" in Table 1 has a containment relationship with the cells "Amount" and "Proportion" below it. On the basis of the containment relationship between cells in terms of position coordinates, the containment relationship between cells can be further determined based on entity types. For example, there is a containment relationship between the value cell and the nearest header cell on the left or above, and there is a containment relationship between the header item cell and the nearest header cell at the coordinate or above.

[0116] S203: extract information from the table to be processed according to the relationship information of each cell, and obtain an extraction result of the table to be processed.

[0117] After obtaining the relationship information of the cells, information can be extracted based on the relationship information of the cells, and a mapping relationship can be established from the value to the header and from the header to the previous header. Then, the cells are sorted according to the order, and the table to be processed is stored as a key-value pair or in the form of natural language.

[0118] In an embodiment of the present application, entity classification is performed on the cells of the table to be processed based on a pre-trained entity classification model to obtain the entity type of each cell in the table to be processed. The entity classification model obtains the entity type of the cell based on the positional relationship between each word segmentation element in the cell. According to the entity type of each cell, the relationship information of each cell is determined. The relationship information is used to characterize the inclusion relationship between the cell and the associated cells. According to the relationship information of each cell, information is extracted from the table to be processed to obtain the extraction result of the table to be processed.

[0119] By classifying the entities of the cells of the table to be processed based on the entity classification model, and then extracting relationships based on the entity types of the cells, compared to the traditional key-value pair extraction scheme, the present application can be applied to complex table styles, and therefore can meet the table extraction needs in more complex scenarios. In addition, the entity classification model in the present application obtains the entity type of the cell based on the positional relationship between each word segmentation element in the cell, so it can distinguish fine-grained information, and can accurately identify the entity type of the cell for nested tables and multi-level tables, thereby improving the accuracy of subsequent table information extraction. Finally, the present application directly extracts information from the table to be processed based on the extracted relationship information. Compared with the method of using neural networks or large models in the prior art, the method of the present application can extract table information more directly and quickly, and has lower complexity.

[0120] The following is a further description of how the entity classification model based on the pre-trained entity classification model performs entity classification on the cells of the table to be processed to obtain the entity type of each cell in the table to be processed, such as Figure 3 As shown, the above step S201 includes:

[0121] S301 , performing word segmentation processing on the text of the table to be processed to obtain at least one word segmentation sequence and a mapping relationship table between each word segmentation element in the word segmentation sequence and a cell, wherein the word segmentation sequence includes a plurality of word segmentation elements arranged in sequence.

[0122] The text of the table to be processed may be the text in the cells of the table to be processed. After the text of all the cells of the table to be processed is segmented, all the segmentation results may be concatenated into a sequence, and then the sequence may be segmented to obtain at least one segmentation sequence.

[0123] As an example, the text of all cells can be segmented according to the maximum length of 2N. A segmentation marker can be added in front of the segmentation of the first column, and a segmentation marker can be added after the segmentation of the last column, and all the segmentation results can be concatenated into a sequence.

[0124] When the sequence is segmented to obtain a word segmentation sequence, a sliding window can be used to slide on the sequence according to the cells to obtain multiple word segmentation sequences. A word segmentation sequence includes a word segmentation element of at least one cell, and a cell includes at least one word segmentation element.

[0125] Optionally, the mapping relationship table may record the mapping relationship between the cell identifier and the position of the word segmentation element.

[0126] S302: Obtain coordinate information of each cell in the table to be processed, where the coordinate information of the cell includes: the spatial coordinates of the cell and the grid coordinates of the cell.

[0127] The spatial coordinates of a cell can be the coordinate values ​​of the cell in the coordinate system of the table to be processed, such as (x, y), where x represents the horizontal coordinate and y represents the vertical coordinate. The grid coordinates of a cell can be the row position and column position of the cell in the table to be processed, such as (a, b), where a represents the row number and b represents the column number.

[0128] S303: Determine the first code and the second code of each word segmentation element in the word segmentation sequence according to the word segmentation sequence and the mapping relationship table, wherein the first code is used to represent the position of the word segmentation element in the word segmentation sequence, and the second code is used to represent the identifier of the cell to which the word segmentation element belongs.

[0129] Based on the mapping relationship table, it is possible to determine which word segmentation elements in the word segmentation sequence each cell corresponds to, thereby determining the identifier of the cell to which the word segmentation element belongs, that is, the second code of the word segmentation element.

[0130] The position of the word segmentation element in the word segmentation sequence may be the serial number of the word segmentation element in the word segmentation sequence, and the identifier of the cell to which the word segmentation element belongs may be the serial number or spatial coordinates of the cell to which the word segmentation element belongs.

[0131] S304: Input the word segmentation sequence, the coordinate information of each cell, and the first code and the second code of each word segmentation element into an entity classification model, and determine the entity type of each cell by the entity classification model.

[0132] It is worth noting that while inputting the word segmentation sequence, the coordinate information of each cell, and the first and second codes of each word segmentation element, the image of the table to be processed can also be serialized to obtain a picture sequence, and the picture sequence is also input into the entity classification model. The entity classification model performs classification based on the word segmentation sequence and the picture sequence as well as the input coordinate information of each cell, and the first and second codes of each word segmentation element to obtain the entity type of the cell.

[0133] Among them, the first code can represent the one-dimensional position code of the word segmentation element in the word segmentation sequence, and the entity classification model can determine the spatial coordinates of the word segmentation element based on the coordinate information of the cell and the second code, that is, the two-dimensional position code of the word segmentation element, and determine the positional relationship between the word segmentation elements through the coordinate information of the cell and the second code, that is, whether the word segmentation elements belong to the same cell, the same column or the same row. The entity classification model can classify and output the entity type of the cell based on the one-dimensional position code, the two-dimensional position code and the positional relationship between the word segmentation elements.

[0134] Since the first code and the second code can respectively represent the position of the word segmentation element in the word segmentation sequence and the position of the cell to which the word segmentation element belongs, the entity classification model can realize more fine-grained entity type recognition for the cell based on the first code and the second code of the word segmentation element and the coordinate information of the cell, thereby improving the accuracy of cell entity type recognition.

[0135] The following is a further description of performing word segmentation processing on the text of the table to be processed to obtain at least one word segmentation sequence and a mapping relationship table between each word segmentation element in the word segmentation sequence and a cell, such as Figure 4 As shown, the above step S301 includes:

[0136] S401 , segmenting the text of the cells of the table to be processed to obtain a cell list, and recording the mapping relationship between the identifier of each segmentation element in the cell list and the identifier of the cell.

[0137] The cell list includes multiple word segmentation elements and cell identifiers. When the cell list is obtained by word segmentation, the mapping relationship between the cell identifiers in the cell list and the positions of each word segmentation element in the word segmentation sequence can be recorded to obtain a mapping relationship table.

[0138] S402: Segment the cell list based on the sliding window to obtain at least one word segmentation sequence.

[0139] When segmentation is performed based on a sliding window, the sliding window can be slid according to the cell according to the mapping relationship table. If the length of all the word segmentation elements of the cell does not exceed the maximum length, the sliding window continues to slide to the word segmentation element of the next cell until the total length of the word segmentation elements of m cells is about to exceed the maximum length. The current cell list is segmented to obtain a word segmentation sequence. If the length of all the word segmentation elements of the cell exceeds the maximum length, all the word segmentation elements of the cell are taken as a word segmentation sequence. The maximum length of the word segmentation sequence can be 512, for example.

[0140] The following is a further explanation of determining the first code and the second code of each word segmentation element in the word segmentation sequence according to the word segmentation sequence and the mapping relationship table. Figure 5 As shown, the above step S303 includes:

[0141] S501. Determine a first code of a word segmentation element according to the position of the word segmentation element in a word segmentation sequence.

[0142] S502: Search in the mapping relationship table according to the identifier of the word segmentation element to determine the target cell to which the word segmentation element belongs.

[0143] S503: Use the spatial coordinates of the target cell as the second code of the word segmentation element.

[0144] The first code of the word segmentation element can be obtained according to the sequence number of the word segmentation element in the word segmentation sequence. Exemplarily, assuming that the word segmentation element is the nth in the word segmentation sequence, the first code of the word segmentation element can be n.

[0145] The mapping relationship table can record the correspondence between the identifier of the word segmentation element and the cell identifier. Therefore, by searching in the mapping relationship table based on the identifier of the word segmentation element, the cell identifier can be obtained. The cell represented by the cell identifier is the target cell to which the word segmentation element belongs, and the spatial coordinates (x0, y0, x1, y1) of the two points in the upper left corner and lower right corner of the target cell found are used as the second code of the word segmentation element.

[0146] Next, the steps of inputting the word segmentation sequence and the first code and the second code of each word segmentation element into the entity classification model and determining the entity type of each cell by the entity classification model are described.

[0147] The word segmentation sequence and the first and second codes of each word segmentation element are input into the entity classification model, and the entity classification model performs Figure 6 The following steps are shown:

[0148] S601: Combine word segmentation elements in a word segmentation sequence in pairs to obtain multiple word segmentation element groups.

[0149] S602: Determine a score of each segmentation element group according to the first code and the second code of each segmentation element in each segmentation element group, where the score is used to represent the position correlation between two segmentation elements in the segmentation element group.

[0150] S603 , performing multi-classification processing according to the score of each word segmentation element group, the coordinate information of each cell, and the word segmentation sequence to obtain the entity type of each cell.

[0151] The entity classification model can score the word elements at any two positions in each word segmentation sequence based on the attention mechanism to obtain the position correlation degree of the two word segmentation elements. The position correlation degree is used to characterize the closeness of the relative position information of the word segmentation elements. The higher the score, the closer the relative positions of the word segmentation elements are. Conversely, the lower the score, the farther the relative positions of the word segmentation elements are.

[0152] As a possible implementation method, the entity classification model can combine word segmentation elements in pairs to obtain multiple word segmentation element groups, and score each word segmentation element group based on the attention mechanism to obtain the attention score of the word segmentation element group. Finally, the attention scores of all word segmentation element groups are combined into an attention score matrix. Finally, the grid coordinates and the attention score matrix are integrated into the word segmentation sequence, and then softmax multi-classification is performed on the word segmentation sequence to obtain the entity type of each cell.

[0153] By performing attention scoring based on the first and second codes of the word segmentation elements and integrating the grid coordinate information, the model can fully consider the position information of the row and column levels of the cells, thereby improving the accuracy of the model classification.

[0154] Furthermore, in the above step S602, the score of each word segmentation element group is determined according to the first code and the second code of each word segmentation element in each word segmentation element group, such as Figure 7 As shown, including:

[0155] S701. Determine a first score of the word segmentation element group according to a first code of each word segmentation element in the word segmentation element group, where the first score is used to represent a distance between two word segmentation elements in the word segmentation element group in a word segmentation sequence.

[0156] In a first implementation manner, if two word segmentation elements in a word segmentation element group are word segmentation elements of the same cell, the first score of the word segmentation element group may be calculated according to the first codes of the two word segmentation elements in the word segmentation element group.

[0157] In the second implementation, if two word segmentation elements in the word segmentation element group are not word segmentation elements of the same cell, the first score of the word segmentation element group can be calculated according to the largest first code in the word segmentation sequence where the word segmentation elements are located.

[0158] Among them, the second code of the word segmentation element represents the spatial coordinates of the cell to which the word segmentation element belongs. Therefore, when the second codes of two word segmentation elements are the same, it can be considered that the two word segmentation elements belong to the same cell. Conversely, when the second codes of the two word segmentation elements are different, it can be considered that the two word segmentation elements belong to different cells.

[0159] As shown in the following formula (1), this is a first score given by this application: The calculation method of .

[0160] (1)

[0161] in, Relative Position Bucket function. The scoring result of the function is negatively correlated with the relative distance. That is, the closer the relative distance of the word segmentation elements is, the linearly increasing the scoring result. When the relative distance increases to a certain extent, it increases in log form. If the relative distance continues to increase, the return result is 0. a and b are the positions of two word segmentation elements, pos_ids is the first code of the word segmentation element, and max(pos_ids) represents the largest first code in the word segmentation sequence where the word segmentation element is located, such as the first code of the last word segmentation element in the word segmentation sequence.

[0162] S702. Determine a second score and a third score of the word segmentation element group according to the second code of each word segmentation element in the word segmentation element group, wherein the second score is used to represent the distance between two word segmentation elements in the word segmentation element group in the spatial coordinates in the row direction, and the third score is used to represent the distance between two word segmentation elements in the word segmentation element group in the spatial coordinates in the column direction.

[0163] When determining the second score, as a possible implementation method, the score of the word segmentation element group can be determined according to the horizontal coordinate in the spatial coordinate of the cell to which the word segmentation element belongs, wherein when the cell to which the word segmentation element belongs is in the same row of the table to be processed, the word segmentation element can have a higher attention score.

[0164] As shown in the following formula (2), this application provides a calculation method for determining the second score.

[0165] (2)

[0166] in, Represents the horizontal coordinate of the upper left corner of the cell to which the word segmentation element belongs.

[0167] For the third score, the score of the word segmentation element group can be determined according to the ordinate in the spatial coordinates of the cell to which the word segmentation element belongs. When the cells to which the word segmentation elements in the word segmentation element group belong are in the same column of the table to be processed, the word segmentation elements can have a higher score.

[0168] As shown in the following formula (3), this is a calculation method for determining the third score given in the present application.

[0169] (3)

[0170] in, Represents the ordinate of the lower right corner of the spatial coordinates of the cell to which the word segmentation element belongs.

[0171] S703: Determine the score of the word segmentation element group according to the first score, the second score, and the third score.

[0172] As a possible implementation method, the first score, the second score, and the third score may be summed or weighted to obtain the score of the word segmentation element group. The specific calculation method may be shown in the following formula (4).

[0173] (4)

[0174] Among them, d1 represents the first score, d2 represents the second score, and d3 represents the third score. Indicates the score of the word segmentation element group.

[0175] After completing the score calculation of all word segmentation element groups, the attention score matrix of the word segmentation sequence can be obtained. The entity classification model can perform softmax multi-classification based on the attention score matrix and the word segmentation sequence to obtain the entity type of the cell.

[0176] Next, combine Figure 8 The entity classification process of the entity classification model in this application is generally described. Figure 8 As shown, the word segmentation sequence, the picture sequence, and the first and second codes of the word segmentation elements are input into the entity classification model. The entity classification model can perform attention scoring on the word segmentation elements in the word segmentation sequence to obtain the attention score matrix of the word segmentation sequence, and fuse the grid coordinates of the cell of the word segmentation sequence, perform multi-classification on the word segmentation sequence that fuses the grid coordinates and contains the attention score to obtain the entity type of the cell.

[0177] Before using the entity classification model, you can train the entity classification model based on sample data. The following is a further explanation of the training process.

[0178] First, the sample data can be enhanced by randomly cropping the sample table in the sample data, cropping the last row or the last column, and correspondingly, the cells belonging to the cropped row and column are also removed from the input; or the coordinates of the cell (x1, y1, x2, y2) are randomly added or subtracted by a small value.

[0179] Secondly, the sample table is segmented. Specifically, the text of all cells is segmented according to the maximum length of 2N (the first N and the last N characters are taken. If the cell belongs to the first column, a segmentation mark is added in front of the segmentation result. If the cell belongs to the last column, a segmentation mark is added after the segmentation result). The cell list is obtained, and the cell list is spliced ​​into a long sequence, recording the mapping relationship between the cell id and the position of each segmentation element in the sequence. Then the sliding window is used to slide according to the cells to cut the sequence into multiple segmentation sequences with a maximum length of 512.

[0180] The word segmentation sequence includes annotation information, and the annotation information includes: a first code, a second code, grid coordinates, and an entity type of each labeled cell.

[0181] Next, the sample table can be input into the initial entity classification model, which will perform attention scoring and entity classification based on the obtained attention score matrix and word segmentation sequence to obtain the entity type of each cell predicted by the initial entity classification model. Finally, the loss value is calculated based on the predicted entity type and the pre-labeled entity type, and the initial entity classification model is iteratively optimized based on the loss value. After multiple rounds of training, if the preset iteration requirements are met, the obtained entity training model can be used as the final entity training model.

[0182] After determining the entity type of the cell, you can extract relationship information for the cell based on the entity type of the cell, such as Fig. 9 As shown, the above step S202 includes:

[0183] S901. Determine the header type cells and value cells in the table to be processed according to the entity type of each cell, wherein the header type cells include: keyword cells, table name cells, header cells, and total item cells.

[0184] S902. Partition the table to be processed according to the table name cells and the table header cells to obtain multiple table sub-areas.

[0185] As a possible implementation method, the table to be processed can be partitioned according to the table name cells and the header cells. For example, the table to be processed shown in Table 1 can be partitioned according to the header cells "First Company's Quarterly Financial Report" and "First Company's Bills for Credit Loss Provision" to obtain the following three table sub-areas of Table 2, Table 3 and Table 4.

[0186] Table 2 Table sub-area 1

[0187]

[0188] Table 3 Table sub-area 2

[0189]

[0190] Table 4 Table sub-area 3

[0191]

[0192] S903, traverse each table sub-area, and for the current table sub-area traversed, determine the inclusion relationship among the keyword cells, table name cells, and header cells in the current table sub-area according to the entity type of each cell in the current table sub-area, and determine the inclusion relationship between the header type cells in the current table sub-area and the inclusion relationship between the value cells and the header cells according to the coordinate information of each cell in the current table sub-area and the entity type of each cell.

[0193] For each table sub-area divided in Table 1 above, the keyword cells of each table sub-area can be traversed to establish a mapping relationship between the keyword cells and the table header cells and the table name cells, and to establish an inclusion relationship between the table name cells and the table header cells.

[0194] In each table sub-area, all cells are traversed. If the cell is a header type, that is, a keyword cell, a table name cell, a header cell, or a total item cell, an inclusion relationship between the cell and the adjacent header cell can be established. If the cell type is a value cell, an inclusion relationship between the cell and the header cell with the closest horizontal and vertical distance can be established based on the cell's coordinate information. Among them, the adjacent header cell can be the header cell closest to the left or top of the current cell.

[0195] For example, when traversing the table sub-areas of Table 2, the inclusion relationship between the table name cell "First Company Quarterly Financial Report" and the table header cells "Project", "End of Reporting Period", "End of Previous Year", "Growth Rate", "Remarks", "Current Ratio", "Quick Ratio" and "Debt Ratio" can be established. For the value cell "1.31", its relationship with "End of Reporting Period" and "Current Ratio" can be established.

[0196] In another example, for the table header type cell "Total" in Table 3, a relationship may be established between it and the adjacent table header cell "Combined Notes Receivable".

[0197] Next, the process of determining the inclusion relationship between the header type cells in the current table sub-region and the inclusion relationship between the value cells and the header cells according to the coordinate information of each cell in the current table sub-region and the entity type of each cell is further described. Fig.10 As shown, the above step S903 includes:

[0198] S1001. If the cell is a keyword cell, a table name cell, a table header cell or a total item cell, then an inclusion relationship is established between the cell and an adjacent table header cell.

[0199] The adjacent header cells may be cells adjacent to the current cell and having a strict inclusion relationship between the horizontal coordinates or the vertical coordinates. The strict inclusion relationship may be that the horizontal coordinates are the same or the vertical coordinates are the same.

[0200] Continuing to refer to Table 4, for the table header cell "Combined Notes Receivable", an inclusion relationship can be established between it and the adjacent table header cell "Single Item Notes Receivable". For the total item cell "Total", an inclusion relationship can be established between it and the adjacent table header cell "Combined Notes Receivable". For the table header cell "Expected Credit Loss Rate", an inclusion relationship can be established between it and the table header cell "Amount".

[0201] S1002: If the cell is a value cell, an inclusion relationship between the cell and the header cell corresponding to the cell is established according to the coordinate information of the cell.

[0202] The header cell corresponding to a cell may be the header cell that is closest to the cell horizontally or vertically. Continuing to refer to Table 4, the header cells corresponding to the value cell "0.1" are "Expected Credit Loss Rate" and "Combined Notes Receivable". The header cells corresponding to the value item "0.04" in Table 3 are "End of Previous Year" and "Debt Ratio".

[0203] In the embodiment of the present application, by establishing relationships between cells in the table to be processed based on entity types, it can be applied to more complex and diverse table formats, covering more complex table information extraction usage scenarios, and improving the versatility of table information extraction.

[0204] After extracting the relationship information of the cells in the table to be processed, the information of the table to be processed can be extracted according to the relationship information of each cell to obtain the extraction result of the table to be processed, such as Fig.11 As shown, the above step S203 includes:

[0205] S1101, traverse the cells in the table to be processed, and obtain the target mapping relationship between the header type cell and the value cell according to the relationship information of each cell.

[0206] S1102 , arranging the texts of each cell in the table to be processed according to the target mapping relationship to obtain an extraction result of the table to be processed.

[0207] Traverse the cells in the table to be processed, and establish a mapping relationship from the value cell to the header type cell, and a mapping relationship from the header type cell to the previous header type cell based on the cell relationship information. Traverse each value cell, search based on the established mapping relationship, and obtain the target mapping relationship between the header type cell and the value cell.

[0208] After determining the target mapping relationship, the cells can be arranged in order based on the target mapping relationship, and the cell identifiers can be converted into corresponding texts to obtain the final key-value pair results, and the key-value pair results can be used as the extraction results of the table to be processed.

[0209] Based on the same inventive concept, a table information extraction device corresponding to the table information extraction method is also provided in the embodiment of the present application. Since the principle of solving the problem by the device in the embodiment of the present application is similar to the above-mentioned table information extraction method in the embodiment of the present application, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0210] Fig.12 A schematic diagram of the structure of a table information extraction device provided in an embodiment of the present application is shown, and the device includes: a classification module 1201, a determination module 1202 and an extraction module 1203.

[0211] A classification module 1201 is used to classify the cells of the table to be processed based on a pre-trained entity classification model to obtain the entity type of each cell in the table to be processed. The entity classification model obtains the entity type of the cell based on the positional relationship between each word segmentation element in the table to be processed.

[0212] A determination module 1202 is used to determine the relationship information of each cell according to the entity type of each cell, where the relationship information is used to represent the inclusion relationship between the cell and the associated cells;

[0213] The extraction module 1203 is used to extract information from the table to be processed according to the relationship information of each cell to obtain the extraction result of the table to be processed.

[0214] As a possible implementation, the classification module 1201 is specifically used for:

[0215] Perform word segmentation processing on the text of the table to be processed to obtain at least one word segmentation sequence and a mapping relationship table between each word segmentation element in the word segmentation sequence and a cell, wherein the word segmentation sequence includes a plurality of word segmentation elements arranged in sequence;

[0216] Obtaining the coordinate information of each cell in the table to be processed, the coordinate information of the cell includes: the spatial coordinates of the cell and the grid coordinates of the cell;

[0217] According to the word segmentation sequence and the mapping relationship table, determine the first code and the second code of each word segmentation element in the word segmentation sequence, the first code is used to represent the position of the word segmentation element in the word segmentation sequence, and the second code is used to represent the identifier of the cell to which the word segmentation element belongs;

[0218] The word segmentation sequence, the coordinate information of each cell, and the first code and the second code of each word segmentation element are input into the entity classification model, and the entity classification model determines the entity type of each cell.

[0219] As a possible implementation, the classification module 1201 is specifically used for:

[0220] Segment the text of the cells of the table to be processed to obtain a cell list, and record the mapping relationship between the identifier of each segmentation element in the cell list and the identifier of the cell;

[0221] The cell list is segmented based on the sliding window to obtain at least one word segmentation sequence.

[0222] As a possible implementation, the classification module 1201 is specifically used for:

[0223] Determine the first encoding of the word segmentation element according to the position of the word segmentation element in the word segmentation sequence;

[0224] Search the mapping relationship table according to the identifier of the word segmentation element to determine the target cell to which the word segmentation element belongs;

[0225] The spatial coordinates of the target cell are used as the second encoding of the word segmentation element.

[0226] As a possible implementation, the classification module 1201 is specifically used for:

[0227] The word segmentation sequence, the coordinate information of each cell, and the first and second codes of each word segmentation element are input into the entity classification model, and the entity classification model performs the following steps:

[0228] Combine the word segmentation elements in the word segmentation sequence in pairs to obtain multiple word segmentation element groups;

[0229] Determine the score of each segmentation element group according to the first code and the second code of each segmentation element in each segmentation element group, where the score is used to represent the position correlation degree between two segmentation elements in the segmentation element group;

[0230] Multi-classification processing is performed according to the score of each word segmentation element group, the coordinate information of each cell, and the word segmentation sequence to obtain the entity type of each cell.

[0231] As a possible implementation, the classification module 1201 is specifically used for:

[0232] Determine a first score of the word segmentation element group according to the first code of each word segmentation element in the word segmentation element group, where the first score is used to represent the distance between two word segmentation elements in the word segmentation element group in the word segmentation sequence;

[0233] According to the second codes of each word segmentation element in the word segmentation element group, respectively determine a second score and a third score of the word segmentation element group, the second score is used to represent the distance between two word segmentation elements in the word segmentation element group in the row direction, and the third score is used to represent the distance between two word segmentation elements in the word segmentation element group in the column direction;

[0234] The score of the word segmentation element group is determined according to the first score, the second score, and the third score.

[0235] As a possible implementation manner, the determination module 1202 is specifically configured to:

[0236] According to the entity type of each cell, determine the header type cell and the value cell in the table to be processed, wherein the header type cell includes: keyword cell, table name cell, header cell, and total item cell;

[0237] Partition the table to be processed according to the table name cell and the table header cell to obtain multiple table sub-areas;

[0238] Traverse each table sub-area, and for the current table sub-area traversed, determine the inclusion relationship among the keyword cells, table name cells, and header cells in the current table sub-area according to the entity type of each cell in the current table sub-area, and determine the inclusion relationship between the header type cells in the current table sub-area and the inclusion relationship between the value cells and the header cells according to the coordinate information of each cell in the current table sub-area and the entity type of each cell.

[0239] As a possible implementation manner, the determination module 1202 is specifically configured to:

[0240] If the cell is a keyword cell, table name cell, table header cell or total item cell, then an inclusion relationship between the cell and the adjacent table header cell is established;

[0241] If the cell is a value cell, an inclusion relationship between the cell and the header cell corresponding to the cell is established according to the coordinate information of the cell.

[0242] As a possible implementation, the extraction module 1203 is specifically used for:

[0243] Traverse the cells in the table to be processed, and obtain the target mapping relationship between the header type cell and the value cell according to the relationship information of each cell;

[0244] The texts of each cell in the table to be processed are arranged according to the target mapping relationship to obtain the extraction result of the table to be processed.

[0245] The embodiment of the present application classifies entities of the cells of the table to be processed based on the entity classification model, and then extracts relationships based on the entity type of the cell. Compared with the traditional key-value pair extraction scheme, the present application can be applied to complex table styles, and therefore can meet the table extraction requirements in more complex scenarios. In addition, the entity classification model in the present application obtains the entity type of the cell based on the positional relationship between each word segmentation element in the cell, so it can distinguish fine-grained information, and can also accurately identify the entity type of the cell for nested tables and multi-level tables, thereby improving the accuracy of subsequent table information extraction. Finally, the present application directly extracts information from the table to be processed based on the extracted relationship information. Compared with the method of using neural networks or large models in the prior art, the method of the present application can extract table information more directly and quickly, and therefore has lower complexity.

[0246] Fig.13 A structural schematic diagram of an electronic device 130 provided in an embodiment of the present application is shown, including: a processor 1301, a storage medium 1302 and a bus 1303, wherein the storage medium 1302 stores machine-readable instructions executable by the processor 1301, and when the electronic device runs the table information extraction method as in the embodiment, the processor 1301 communicates with the storage medium 1302 via the bus 1303, and the processor 1301 executes the machine-readable instructions, and the processor 1301 executes the preamble of the method item to execute the steps in the above-mentioned table information extraction method.

[0247] An embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. The computer program is executed when a processor is running, and the processor executes the steps in the above-mentioned table information extraction method.

[0248] In the embodiment of the present application, the computer program can also execute other machine-readable instructions when run by the processor to execute other methods described in the embodiment. For the specific execution method steps and principles, please refer to the description of the embodiment, which will not be repeated here.

[0249] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0250] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0251] In addition, each functional unit in the embodiments provided in the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0252] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, and other media that can store program codes.

[0253] It should be noted that similar numbers and letters represent similar items in the following figures. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. In addition, the terms "first", "second", "third", etc. are only used to distinguish the description and are not to be understood as indicating or implying relative importance.

[0254] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present application, which are used to illustrate the technical solutions of the present application, rather than to limit them. The protection scope of the present application is not limited thereto. Although the present application is described in detail with reference to the above-mentioned embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-mentioned embodiments within the technical scope disclosed in the present application, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application. They should all be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A table information extraction method, characterized in that: include: Based on the pre-trained entity classification model, the cells of the table to be processed are classified to obtain the entity type of each cell in the table to be processed, wherein the entity classification model obtains the entity type of the cell based on the positional relationship between each word segmentation element in the table to be processed; Determining, according to the entity type of each of the cells, relationship information of each of the cells, wherein the relationship information is used to characterize the inclusion relationship between the cell and an associated cell; Extracting information from the table to be processed according to the relationship information of each of the cells to obtain an extraction result of the table to be processed; The determining, according to the entity type of each cell, the relationship information of each cell includes: According to the entity type of each cell, determine the header type cell and the value cell in the table to be processed, wherein the header type cell includes: a keyword cell, a table name cell, a header cell, and a total item cell; Partition the table to be processed according to the table name cells and the table header cells to obtain a plurality of table sub-areas; Traverse each of the table sub-areas, and for the current table sub-area traversed, determine the inclusion relationship among the keyword cells, table name cells and header cells in the current table sub-area according to the entity type of each cell in the current table sub-area, and determine the inclusion relationship between the header type cells in the current table sub-area and the inclusion relationship between the value cells and the header cells according to the coordinate information of each of the cells in the current table sub-area and the entity type of each of the cells.

2. The method according to claim 1, characterized in that: The entity classification model obtained by pre-training performs entity classification on the cells of the table to be processed to obtain the entity type of each cell in the table to be processed, including: Performing word segmentation processing on the text of the table to be processed to obtain at least one word segmentation sequence and a mapping relationship table between each word segmentation element in the word segmentation sequence and a cell, wherein the word segmentation sequence includes a plurality of the word segmentation elements arranged in sequence; Acquire coordinate information of each cell in the table to be processed, wherein the coordinate information of each cell includes: spatial coordinates of the cell and grid coordinates of the cell; Determine, according to the word segmentation sequence and the mapping relationship table, a first code and a second code of each word segmentation element in the word segmentation sequence, wherein the first code is used to represent the position of the word segmentation element in the word segmentation sequence, and the second code is used to represent the identifier of the cell to which the word segmentation element belongs; The word segmentation sequence, the coordinate information of each cell, the first code and the second code of each word segmentation element are input into the entity classification model, and the entity type of each cell is determined by the entity classification model.

3. The method according to claim 2, characterized in that The word segmentation processing is performed on the text of the table to be processed to obtain at least one word segmentation sequence and a mapping relationship table between each word segmentation element in the word segmentation sequence and a cell, including: Segmenting the text of the cells of the table to be processed to obtain a cell list, and recording the mapping relationship between the identifier of each segmentation element in the cell list and the identifier of the cell; The cell list is segmented based on a sliding window to obtain the at least one word segmentation sequence.

4. The method according to claim 2, characterized in that: The determining, according to the word segmentation sequence and the mapping relationship table, the first code and the second code of each word segmentation element in the word segmentation sequence comprises: Determine a first encoding of the word segmentation element according to the position of the word segmentation element in the word segmentation sequence; Searching in the mapping relationship table according to the identifier of the word segmentation element to determine the target cell to which the word segmentation element belongs; The spatial coordinates of the target cell are used as the second encoding of the word segmentation element.

5. The method according to claim 2, characterized in that: The step of inputting the word segmentation sequence, the coordinate information of each cell, and the first code and the second code of each word segmentation element into the entity classification model, and determining the entity type of each cell by the entity classification model includes: The word segmentation sequence, the coordinate information of each cell, and the first code and the second code of each word segmentation element are input into the entity classification model, and the entity classification model performs the following steps: Combining the word segmentation elements in the word segmentation sequence in pairs to obtain multiple word segmentation element groups; Determine a score for each segmentation element group according to the first code and the second code of each segmentation element in each segmentation element group, wherein the score is used to represent the position correlation degree between two segmentation elements in the segmentation element group; Multi-classification processing is performed according to the scores of each word segmentation element group, the coordinate information of each cell and the word segmentation sequence to obtain the entity type of each cell.

6. The method according to claim 5, characterized in that The step of determining the score of each word segmentation element group according to the first code and the second code of each word segmentation element in each word segmentation element group comprises: Determine a first score of the word segmentation element group according to the first code of each word segmentation element in the word segmentation element group, wherein the first score is used to represent the distance between two word segmentation elements in the word segmentation element group in the word segmentation sequence; According to the second codes of each word segmentation element in the word segmentation element group, respectively determine a second score and a third score of the word segmentation element group, wherein the second score is used to represent the distance between two word segmentation elements in the word segmentation element group in the row direction spatial coordinates, and the third score is used to represent the distance between two word segmentation elements in the word segmentation element group in the column direction spatial coordinates; The score of the word segmentation element group is determined according to the first score, the second score, and the third score.

7. The method according to claim 1, characterized in that The determining, according to the coordinate information of each cell in the current table sub-region and the entity type of each cell, the inclusion relationship between the header type cells in the current table sub-region and the inclusion relationship between the value cells and the header cells, comprises: If the cell is a keyword cell, a table name cell, a table header cell or a total item cell, then an inclusion relationship between the cell and an adjacent table header cell is established; If the cell is a value cell, an inclusion relationship between the cell and a header cell corresponding to the cell is established according to the coordinate information of the cell.

8. The method according to claim 1, characterized in that The extracting information from the table to be processed according to the relationship information of each cell to obtain the extraction result of the table to be processed includes: Traversing the cells in the table to be processed, and obtaining a target mapping relationship between a header type cell and a value cell according to the relationship information of each of the cells; The texts of each cell in the table to be processed are arranged according to the target mapping relationship to obtain an extraction result of the table to be processed.

9. A table information extraction device, characterized in that: include: A classification module, used to classify the cells of the table to be processed based on a pre-trained entity classification model to obtain the entity type of each cell in the table to be processed, wherein the entity classification model obtains the entity type of the cell based on the positional relationship between each word segmentation element in the table to be processed; A determination module, used to determine the relationship information of each of the cells according to the entity type of each of the cells, wherein the relationship information is used to characterize the inclusion relationship between the cell and the associated cells; An extraction module, used for extracting information from the table to be processed according to the relationship information of each cell, to obtain an extraction result of the table to be processed; The determining module is also used for: According to the entity type of each cell, determine the header type cell and the value cell in the table to be processed, wherein the header type cell includes: a keyword cell, a table name cell, a header cell, and a total item cell; Partition the table to be processed according to the table name cells and the table header cells to obtain a plurality of table sub-areas; Traverse each of the table sub-areas, and for the current table sub-area traversed, determine the inclusion relationship among the keyword cells, table name cells and header cells in the current table sub-area according to the entity type of each cell in the current table sub-area, and determine the inclusion relationship between the header type cells in the current table sub-area and the inclusion relationship between the value cells and the header cells according to the coordinate information of each of the cells in the current table sub-area and the entity type of each of the cells.

10. An electronic device, characterized in that: include: A processor, a storage medium and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the storage medium communicate via the bus, and the processor executes the machine-readable instructions to perform the steps of the table information extraction method as described in any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the table information extraction method according to any one of claims 1 to 8 are executed.

Images