Table restoration method, device and related equipment

Abstract

The present application discloses a table restoration method, device and related equipment, the method comprising: obtaining the position information and text content of each text block from the table image to be restored; obtaining each text block combination based on the position information of each text block; performing named entity recognition on the text content of each text block in each text block combination, and merging each text block in the target text block combination into one text block according to the recognition result; performing cluster analysis on each text block based on the position information and text content of each text block in the table image to obtain cluster information, the cluster information including each row baseline and each column baseline; restoring each text block to a row-column matrix composed of each row baseline and each column baseline to obtain each cell information of the restored table. The present application can distinguish text blocks that span rows or columns, and restore the cells where the text blocks are located according to the row baseline and the column baseline, and finally can adaptively process tables that span rows and columns.

Description

Technical Field

[0001] The present application relates to the field of table recognition technology, and more specifically, to a table restoration method, device and related equipment. Background Art

[0002] In the operation involving the reading of table information, it is necessary to first perform structural restoration of the table, that is, restore the row and column relationship and each cell of the table. Usually, the table is input in the form of an image, and structural restoration of the table inevitably involves processing the image.

[0003] Some existing table restoration methods use pixel-by-pixel prediction of table and column areas to solve the problem of detecting gridless tables. For cross-row and cross-column situations, the usual approach is that if an entry spans multiple rows, the row with the largest number of non-empty items in the table is marked as the starting point of the new row. For example, see Figure 1 In a multi-column table, some columns (such as the "Symbol" column) can contain entries in only one row, while other columns (such as the "Chemical Name" column) can have entries that span multiple rows. Therefore, after all entities in each column are filled, a new row begins. This method can only effectively handle the case where the entire data spans multiple rows, and cannot adapt to irregular table cell contents.

[0004] Some other table restoration methods use GRU sequence-based models to extract table structures. Compared with heuristic algorithms and CNN-based models, sequence-based models can effectively capture repeated row / column structures in tables. However, this method relies on the repeatability of the layout style (layout) of the table in the row or column structure to obtain correct results, and it is difficult to adapt to a large number of irregular cross-row and cross-column table samples in business environments. Summary of the invention

[0005] In view of this, the present application provides a table restoration method, apparatus and related equipment to achieve restoration of irregular tables.

[0006] To achieve the above-mentioned purpose, the first aspect of the present application provides a table restoration method, comprising:

[0007] Obtaining the position information and text content of each text block from the table image to be restored;

[0008] Based on the position information of each text block, each text block combination is obtained, where each text block combination includes at least two text blocks that are adjacent to each other;

[0009] Performing named entity recognition on the text content of each text block in each text block combination, and screening out a target text block combination according to the recognition result, merging each text block in the target text block combination into one text block, and updating the position information and text content of the merged text block, wherein each text block in the target text block combination can be merged into a single semantic entity;

[0010] Performing cluster analysis on each text block based on position information and text content of each text block in the table image to obtain cluster information, wherein the cluster information includes each row baseline and each column baseline;

[0011] Each text block is restored to a row-column matrix consisting of each row baseline and each column baseline to obtain information of each cell of the restored table.

[0012] Preferably, the process of performing cluster analysis on each text block based on the position information and text content of each text block in the table image to obtain cluster information includes:

[0013] For each block of text:

[0014] Based on the text content of the text block, embedding the text block into a word vector to obtain a word vector of the text block;

[0015] Based on the position information of the text block, obtaining a rectangular bounding box of the text block;

[0016] Fusion the coordinate information of the rectangular bounding box to the word vector to obtain a fused vector of the text block;

[0017] The fusion vectors of each text block are clustered and analyzed by K-means to obtain clustering information.

[0018] Preferably, the process of performing cluster analysis on the fusion vectors of each text block by K-means to obtain cluster information includes:

[0019] K-means is used to perform cluster analysis on the fusion vectors of each text block in the horizontal and vertical directions to obtain the baselines of each row and column.

[0020] Preferably, the clustering information also includes text blocks that are clustered to a row baseline and a column baseline at the same time, and free text blocks that cannot be clustered to a row baseline and a column baseline at the same time; the process of restoring each text block to a row-column matrix composed of each row baseline and each column baseline includes:

[0021] Restoring a text block that is clustered to both a row baseline and a column baseline to a cell formed by the row baseline and the column baseline;

[0022] For each cell formed by the row baselines and the column baselines:

[0023] Determine whether there is a free text block inside the cell;

[0024] If so, restore the free text block to the cell.

[0025] Preferably, the process of obtaining the combination of each text block based on the position information of each text block includes:

[0026] Based on the position information of each text block, two or more adjacent text blocks that are adjacent to each other vertically or horizontally are determined to obtain each text block combination.

[0027] Preferably, the clustering information also includes semantic clustering of each text block, and the method further includes:

[0028] The header information of the target table is determined according to the semantic clustering of each text block and the row baseline or column baseline to which each text block is clustered.

[0029] Preferably, the process of obtaining the position information and text content of each text block from the table image to be restored includes:

[0030] The position information and text content of each text block are obtained from the table image to be restored through optical character recognition technology.

[0031] A second aspect of the present application provides a table restoration device, comprising:

[0032] A text block acquisition unit, used to acquire the position information and text content of each text block from the table image to be restored;

[0033] An adjacent block acquisition unit, used to acquire each text block combination based on the position information of each text block, each text block combination includes at least two text blocks that are adjacent to each other;

[0034] A text block merging unit, used to perform named entity recognition on the text content of each text block in each text block combination, and screen out a target text block combination according to the recognition result, merge each text block in the target text block combination into one text block, and update the position information and text content of the merged text block, wherein each text block in the target text block combination can be merged into a single semantic entity;

[0035] A text block clustering unit, used for performing cluster analysis on each text block based on the position information and text content of each text block in the table image to obtain cluster information, wherein the cluster information includes each row baseline and each column baseline;

[0036] The cell restoration unit is used to restore each text block to a row-column matrix composed of each row baseline and each column baseline to obtain each cell information of the restored table.

[0037] A third aspect of the present application provides a table restoration device, including: a memory and a processor;

[0038] The memory is used to store programs;

[0039] The processor is used to execute the program to implement each step of the above table restoration method.

[0040] A fourth aspect of the present application provides a storage medium on which a computer program is stored. When the computer program is executed by a processor, the various steps of the table restoration method as described above are implemented.

[0041] It can be known from the above technical scheme that the present application first obtains the position information and text content of each text block from the table image to be restored. Then, based on the position information of each text block, adjacent text block combinations are obtained, wherein the text block combination may be a text block separated due to automatic line separation, or may be a text block that is originally independent of each other. Then, named entity recognition is performed on the text content of each text block in each text block combination, and the target text block combination is screened out according to the recognition result, and the text blocks in the target text block combination are merged into one text block, and the position information and text content of the merged text block are updated. Among them, through named entity recognition, it can be determined whether the text block combination belongs to a valid entity, thereby restoring the separated text blocks, and merging the text blocks in the target text block combination into a valid entity. Then, cluster analysis is performed on each text block based on the position information and text content of each text block in the table image to obtain cluster information, and the cluster information includes each row baseline and each column baseline, so as to effectively identify the row and column relationship of the table. Finally, each text block is restored to a row-column matrix consisting of row baselines and column baselines to obtain the information of each cell of the restored table. The present application can distinguish text blocks that span rows or columns, and restore the cells where the text blocks are located according to the row baselines and column baselines. Finally, it can adaptively process tables that span rows and columns, improving the ability to parse table structures and understand content. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0043] Figure 1 Schematic diagrams of irregular tables are illustrated;

[0044] Figure 2 A schematic diagram of a table restoration method disclosed in an embodiment of the present application;

[0045] Figure 3 A schematic diagram illustrating a text block combination disclosed in an embodiment of the present application;

[0046] Figure 4 A schematic diagram illustrating the merging of text blocks across lines disclosed in an embodiment of the present application is illustrated;

[0047] Figure 5 A schematic diagram illustrating the baselines of each row and each column disclosed in the embodiment of the present application is illustrated;

[0048] Figure 6 Another schematic diagram of the table restoration method disclosed in the embodiment of the present application;

[0049] Figure 7 The schematic diagram of the table header determined based on clustering disclosed in the embodiment of the present application is illustrated;

[0050] Figure 8 A schematic diagram of a table restoration device disclosed in an embodiment of the present application;

[0051] Fig. 9 A schematic diagram of a table restoration device disclosed in an embodiment of the present application. DETAILED DESCRIPTION

[0052] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0053] The following describes the table restoration method provided by the embodiment of the present application. Figure 2 The table restoration method provided in the embodiment of the present application may include the following steps:

[0054] Step S101, obtaining the position information and text content of each text block from the table image to be restored.

[0055] There are many methods for obtaining the position information and text content information of a text block from an image, such as OCR (Optical Character Recognition) technology, document format parsing technology, and the like.

[0056] Step S102, obtaining each text block combination based on the position information of each text block.

[0057] Each text block combination includes at least two adjacent text blocks. Figure 3 , Figure 3 The dashed box in the figure illustrates some text block combinations. The text block combination can be a combination of two or more text blocks in adjacent rows, or a combination of two or more text blocks in adjacent columns. Figure 3 Only some text block combinations are illustrated, and these text block combinations still need to be processed in step S103 to further determine whether they can be merged.

[0058] Step S103, performing named entity recognition on the text content of each text block in each text block combination, and merging and updating the text blocks according to the recognition results.

[0059] Specifically, by performing NER (Name Entity Recognition) analysis on the text content composed of each text block in each text block combination, that is, performing named entity recognition, it is possible to determine whether these text blocks in a text block combination constitute a valid entity noun, and then filter out the target text block combination based on the recognition result, wherein the text blocks in the target text block combination can be merged to form a valid entity. Specifically, the combination of the text blocks in the target text block combination can form an entity with a single semantic meaning. Finally, the text blocks in the target text block combination are merged into one text block, and the position information and text content of the merged text block are updated. Exemplarily, for Figure 3 The text block combination composed of "dodecylbenzene sulfonate" and "calcium sulfonate" in the target text block combination is identified as the target text block combination, and the text blocks in the target text block combination are merged into a text block "calcium dodecylbenzene sulfonate", and the position information and text content of the merged text block are updated at the same time. Figure 3 The target text blocks in the table in the table are merged into each text block as follows Figure 4 shown.

[0060] Step S104, performing cluster analysis on each text block based on the position information and text content of each text block in the table image to obtain cluster information.

[0061] It can be understood that the text blocks here are the text blocks processed by the above step S103, that is, they include the text blocks that have not been merged and the new text blocks after merging. Figure 1 The table in the table, the object of cluster analysis here is Figure 4 Each block of text in .

[0062] The clustering information includes the baselines of each row and each column. Figure 5 By clustering the position information of each text block, text blocks that are close in position can be clustered into a row cluster or a column cluster; by clustering the text information of each text block, text blocks that are semantically close can be clustered into a semantic cluster.

[0063] Step S105, restoring each text block to a row-column matrix composed of each row baseline and each column baseline, and obtaining information of each cell of the restored table.

[0064] Specifically, since the baselines of each row and each column have been known, it is equivalent to having determined the structure of the entire table, and then filling each text block into the corresponding cell to restore the entire table.

[0065] The embodiment of the present application first obtains the position information and text content of each text block from the table image to be restored. Then, based on the position information of each text block, adjacent text block combinations are obtained, wherein the text block combination may be a text block separated due to automatic line separation, or may be a text block that is originally independent of each other. Then, named entity recognition is performed on the text content of each text block in each text block combination, and the target text block combination is screened out according to the recognition result, and the text blocks in the target text block combination are merged into one text block, and the position information and text content of the merged text block are updated. Among them, through named entity recognition, it can be determined whether the text block combination belongs to a valid entity, thereby restoring the separated text blocks, and merging the text blocks in the target text block combination into a valid entity. Then, based on the position information and text content of each text block in the table image, cluster analysis is performed on each text block to obtain cluster information, which includes each row baseline and each column baseline, thereby effectively identifying the row and column relationship of the table. Finally, each text block is restored to a row-column matrix consisting of row baselines and column baselines to obtain the information of each cell of the restored table. The present application can distinguish text blocks that span rows or columns, and restore the cells where the text blocks are located according to the row baselines and column baselines. Finally, it can adaptively process tables that span rows and columns, improving the ability to parse table structures and understand content.

[0066] In some embodiments of the present application, the process of obtaining the position information and text content of each text block from the table image to be restored in step S101 may include:

[0067] The position information and text content of each text block are obtained from the table image to be restored through optical character recognition technology.

[0068] Among them, optical character recognition technology (OCR) as an existing technology is a process of translating shapes into computer text using character recognition methods, specifically a process of obtaining text and layout information by analyzing and processing image files.

[0069] In some embodiments of the present application, the process of obtaining the combination of each text block based on the position information of each text block in step S102 may include:

[0070] Based on the position information of each text block, two or more adjacent text blocks that are adjacent to each other vertically or horizontally are determined to obtain each text block combination.

[0071] For example, you can try to combine two or more adjacent text blocks in a loop from left to right and from top to bottom, and then use NER technology to process the combined text blocks. If a valid entity noun is found, it is considered that the two or more text blocks can be combined in the corresponding order to form a content item. This process is iterated repeatedly on the entire page to complete the redivision of the page text blocks according to the meaning of the entity words.

[0072] In some embodiments of the present application, the process of performing cluster analysis on each text block based on the position information and text content of each text block in the table image and obtaining cluster information in step S104 may include:

[0073] For each block of text:

[0074] S1, based on the text content of the text block, embed the text block into a word vector to obtain the word vector of the text block.

[0075] Among them, word embedding, also known as word embedding, is a general term for a set of language modeling and feature learning techniques in natural language processing (NLP), which maps words or phrases from the vocabulary to vectors composed of real numbers. Conceptually, it involves mathematical embedding from a one-dimensional space for each word to a continuous vector space with a lower dimension. Methods for generating this mapping include neural networks, dimensionality reduction of word co-occurrence matrices, and probabilistic models.

[0076] S2: Based on the position information of the text block, obtain a rectangular bounding box of the text block.

[0077] The rectangular boundary box is also called a bounding box, which can be described by four coordinate points of the rectangular boundary of the text block on the page plane coordinates.

[0078] S3, merging the coordinate information of the rectangular bounding box into the word vector to obtain a fused vector of the text block.

[0079] Specifically, the four coordinate points of the rectangular bounding box can form a vector, which can be combined with the word vector in S1 to form a higher-dimensional vector, so as to map the information of multiple modalities together into a unified multimodal vector space. Assuming that the word vector of the text content in the text block is a 100-dimensional vector and the rectangular bounding box of the text block is an 8-dimensional vector, the two vectors are concatenated into a 108-dimensional vector, and the concatenation is repeated for each text block on the entire page to obtain the fusion vector of each text block.

[0080] S4, cluster analysis is performed on the fusion vectors of each text block through K-means to obtain cluster information.

[0081] It is understandable that in addition to K-means, other clustering algorithms can also be used to perform clustering analysis on the fusion vectors of each text block, such as mean shift clustering algorithm, density-based clustering algorithm (DBSCAN), maximum expectation (EM) clustering using Gaussian mixture model (GMM), and graph community detection.

[0082] In some embodiments of the present application, the process of performing cluster analysis on the fusion vectors of each text block by K-means in S4 to obtain cluster information may include:

[0083] K-means is used to perform cluster analysis on the fusion vectors of each text block in the horizontal and vertical directions to obtain the baselines of each row and column.

[0084] Exemplarily, the x coordinates of the left and right sides of the text block, or the y coordinates of the top and bottom, can be used to find clustered text blocks that are relatively close in the x or y direction using the K-means clustering algorithm to obtain the baselines of each row and column.

[0085] In some embodiments of the present application, the clustering information mentioned above also includes text blocks that are clustered to both a row baseline and a column baseline, and free text blocks that cannot be clustered to both a row baseline and a column baseline. The process of restoring each text block to a row-column matrix composed of each row baseline and each column baseline in step S105 may include:

[0086] S1, restore the text block that is clustered to both a row baseline and a column baseline to the cell formed by the row baseline and the column baseline.

[0087] S2, for each cell consisting of the row baselines and the column baselines:

[0088] Determine whether there is a free text block inside the cell. If so, restore the free text block to the cell.

[0089] For example, see Figure 5 , for the column where "risk phrase" is located, all text blocks are clustered into the column baseline of the column, while "R38", "R22", and "R65" are clustered into the row baseline at the same time. Therefore, these text blocks can be directly restored to the corresponding cells according to the above S1. According to the above S3, for the first cell of the column, there are free text blocks "R43" and "R50 / 53", so these two free text blocks are restored to the first cell of the column. Similarly, other cells of the column can be processed.

[0090] In some embodiments of this application, please refer to Figure 6 The clustering information mentioned above also includes the semantic clustering of each text block. The table restoration method also includes:

[0091] Step S106 , determining the header information of the target table according to the semantic clustering of each text block and the row baseline or column baseline to which each text block is clustered.

[0092] For example, see Figure 7 ,According to the semantic clustering of each text block, the text blocks in the table can be divided into 7 clusters.,Combined with the row baseline or column baseline to which each text block is clustered, it can be found that the text blocks in the first row of the table, namely, “Chemical Name”, “CAS-No”, “EC-No”, “Symbol”, “Risk Phrase” and “Concentration”, are the header of the table.

[0093] In addition, since the elements of the table header can be independent of each other, that is, the similarity between these elements is not strong, if there is a top row or left column in the table that cannot be semantically clustered into a cluster, combined with the tabulation habits, the top row or left column can also be confirmed as the table header of the table. Among them, the top row refers to the topmost row in the table, and the left column refers to the leftmost column in the table.

[0094] The table restoration device provided in an embodiment of the present application is described below. The table restoration device described below and the table restoration method described above can be referenced to each other.

[0095] See also Figure 8 The table restoration device provided in the embodiment of the present application may include:

[0096] The text block acquisition unit 21 is used to acquire the position information and text content of each text block from the table image to be restored;

[0097] The adjacent block acquisition unit 22 is used to acquire each text block combination based on the position information of each text block, each text block combination includes at least two text blocks that are adjacent to each other;

[0098] The text block merging unit 23 is used to perform named entity recognition on the text content of each text block in each text block combination, and select a target text block combination according to the recognition result, merge the text blocks in the target text block combination into one text block, and update the position information and text content of the merged text block, wherein the text blocks in the target text block combination can be merged into a single semantic entity;

[0099] The text block clustering unit 24 is used to perform cluster analysis on each text block based on the position information and text content of each text block in the table image to obtain cluster information, wherein the cluster information includes each row baseline and each column baseline;

[0100] The cell restoration unit 25 is used to restore each text block to a row-column matrix composed of each row baseline and each column baseline to obtain each cell information of the restored table.

[0101] In some embodiments of the present application, the process of the text block acquisition unit 21 acquiring the combination of each text block based on the position information of each text block may include:

[0102] Based on the position information of each text block, two or more adjacent text blocks that are adjacent to each other vertically or horizontally are determined to obtain each text block combination.

[0103] In some embodiments of the present application, the text block clustering unit 24 performs cluster analysis on each text block based on the position information and text content of each text block in the table image to obtain cluster information, which may include:

[0104] For each block of text:

[0105] Based on the text content of the text block, embedding the text block into a word vector to obtain a word vector of the text block;

[0106] Based on the position information of the text block, obtaining a rectangular bounding box of the text block;

[0107] Fusion the coordinate information of the rectangular bounding box to the word vector to obtain a fused vector of the text block;

[0108] The fusion vectors of each text block are clustered and analyzed by K-means to obtain clustering information.

[0109] In some embodiments of the present application, the text block clustering unit 24 performs clustering analysis on the fusion vectors of each text block through K-means to obtain clustering information, which may include:

[0110] K-means is used to perform cluster analysis on the fusion vectors of each text block in the horizontal and vertical directions to obtain the baselines of each row and column.

[0111] In some embodiments of the present application, the clustering information also includes text blocks that are clustered to both a row baseline and a column baseline, and free text blocks that cannot be clustered to both a row baseline and a column baseline. The process in which the text block clustering unit 24 restores each text block to a row-column matrix composed of each row baseline and each column baseline may include:

[0112] Restoring a text block that is clustered to both a row baseline and a column baseline to a cell formed by the row baseline and the column baseline;

[0113] For each cell formed by the row baselines and the column baselines:

[0114] Determine whether there is a free text block inside the cell;

[0115] If so, restore the free text block to the cell.

[0116] In some embodiments of the present application, the process of the cell restoration unit 25 acquiring the position information and text content of each text block from the table image to be restored may include:

[0117] The position information and text content of each text block are obtained from the table image to be restored through optical character recognition technology.

[0118] In some embodiments of the present application, the clustering information also includes semantic clustering of each text block, and the table restoration device may further include:

[0119] The header determination unit is used to determine the header information of the target table according to the semantic clustering of each text block and the row baseline or column baseline to which each text block is clustered.

[0120] The table restoration device provided in the embodiment of the present application can be applied to table restoration devices, such as computers, etc. Optionally, Fig. 9 The hardware structure diagram of the table restoration device is shown in FIG. Fig. 9 The hardware structure of the table restoration device may include: at least one processor 31 , at least one communication interface 32 , at least one memory 33 and at least one communication bus 34 .

[0121] In the embodiment of the present application, the number of the processor 31, the communication interface 32, the memory 33, and the communication bus 34 is at least one, and the processor 31, the communication interface 32, and the memory 33 communicate with each other through the communication bus 34;

[0122] The processor 31 may be a central processing unit CPU, or an application-specific integrated circuit ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of the present application, etc.;

[0123] The memory 32 may include a high-speed RAM memory, and may also include a non-volatile memory, such as at least one disk memory;

[0124] The memory 33 stores a program, and the processor 31 can call the program stored in the memory 33, and the program is used to:

[0125] Obtaining the position information and text content of each text block from the table image to be restored;

[0126] Based on the position information of each text block, each text block combination is obtained, where each text block combination includes at least two text blocks that are adjacent to each other;

[0127] Performing named entity recognition on the text content of each text block in each text block combination, and screening out a target text block combination according to the recognition result, merging each text block in the target text block combination into one text block, and updating the position information and text content of the merged text block, wherein each text block in the target text block combination can be merged into a single semantic entity;

[0128] Performing cluster analysis on each text block based on position information and text content of each text block in the table image to obtain cluster information, wherein the cluster information includes each row baseline and each column baseline;

[0129] Each text block is restored to a row-column matrix consisting of each row baseline and each column baseline to obtain information of each cell of the restored table.

[0130] Optionally, the detailed functions and extended functions of the program may refer to the above description.

[0131] The embodiment of the present application further provides a storage medium, which may store a program suitable for execution by a processor, wherein the program is used to:

[0132] Obtaining the position information and text content of each text block from the table image to be restored;

[0133] Based on the position information of each text block, each text block combination is obtained, where each text block combination includes at least two text blocks that are adjacent to each other;

[0134] Performing named entity recognition on the text content of each text block in each text block combination, and screening out a target text block combination according to the recognition result, merging each text block in the target text block combination into one text block, and updating the position information and text content of the merged text block, wherein each text block in the target text block combination can be merged into a single semantic entity;

[0135] Performing cluster analysis on each text block based on position information and text content of each text block in the table image to obtain cluster information, wherein the cluster information includes each row baseline and each column baseline;

[0136] Each text block is restored to a row-column matrix consisting of each row baseline and each column baseline to obtain information of each cell of the restored table.

[0137] Optionally, the detailed functions and extended functions of the program may refer to the above description.

[0138] In summary:

[0139] The present application first obtains the position information and text content of each text block from the table image to be restored. Then, based on the position information of each text block, adjacent text block combinations are obtained, wherein the text block combination may be a text block separated due to automatic line separation, or may be a text block that is originally independent of each other. Then, named entity recognition is performed on the text content of each text block in each text block combination, and the target text block combination is screened out according to the recognition result, the text blocks in the target text block combination are merged into one text block, and the position information and text content of the merged text block are updated. Among them, through named entity recognition, it can be determined whether the text block combination belongs to a valid entity, thereby restoring the separated text block, and merging the text blocks in the target text block combination into a valid entity. Then, based on the position information and text content of each text block in the table image, cluster analysis is performed on each text block to obtain cluster information, which includes each row baseline and each column baseline, so as to effectively identify the row and column relationship of the table. Finally, each text block is restored to a row and column matrix composed of each row baseline and each column baseline, and the information of each cell of the restored table is obtained. The present application can distinguish text blocks that span rows or columns, and restore the cells where the text blocks are located according to the row baseline and column baseline. Ultimately, it can adaptively handle tables that span rows and columns, thereby improving the ability to parse table structures and understand content.

[0140] This application identifies the similarity of content in multiple cells by combining the semantic information of the content, and uses this as a feature to integrate into a network model based on text block layout feature capture. By comparing the similarity in text content between all cell text blocks in the table, the pattern of repeated occurrence of similar data is confirmed, and the boundaries of rows and columns are effectively segmented. At the same time, independent cross-row and cross-column cells can be distinguished, and adjacency operations are performed according to the baseline position of the main data row. Ultimately, some independent cross-row and cross-column cell problems in business tables can be accurately handled, and the ability to parse table structure and understand content through machine learning models in complex scenarios is improved, creating a more complete basic condition for downstream key tasks such as KV matching and table QA.

[0141] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0142] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The various embodiments can be combined as needed, and the same or similar parts can refer to each other.

[0143] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A table restoration method, characterized in that: include: Obtaining the position information and text content of each text block from the table image to be restored; Based on the position information of each text block, each text block combination is obtained, where each text block combination includes at least two text blocks that are adjacent to each other; Performing named entity recognition on the text content of each text block in each text block combination, and screening out a target text block combination according to the recognition result, merging each text block in the target text block combination into one text block, and updating the position information and text content of the merged text block, wherein each text block in the target text block combination can be merged into a single semantic entity; Performing cluster analysis on each text block based on position information and text content of each text block in the table image to obtain cluster information, wherein the cluster information includes each row baseline and each column baseline; Restore each text block to a row-column matrix consisting of each row baseline and each column baseline to obtain each cell information of the restored table; The process of performing cluster analysis on each text block based on the position information and text content of each text block in the table image to obtain cluster information includes: For each block of text: Based on the text content of the text block, embedding the text block into a word vector to obtain a word vector of the text block; Based on the position information of the text block, obtaining a rectangular bounding box of the text block; Fusion the coordinate information of the rectangular bounding box to the word vector to obtain a fused vector of the text block; The fusion vectors of each text block are clustered and analyzed by K-means to obtain clustering information.

2. The method according to claim 1, characterized in that The process of performing cluster analysis on the fusion vectors of each text block by K-means to obtain cluster information includes: K-means is used to perform cluster analysis on the fusion vectors of each text block in the horizontal and vertical directions to obtain the baselines of each row and column.

3. The method according to claim 1, characterized in that The clustering information also includes text blocks that are clustered to both a row of baselines and a column of baselines, and free text blocks that cannot be clustered to both a row of baselines and a column of baselines; The process of restoring each text block to a row-column matrix composed of row baselines and column baselines includes: Restoring a text block that is clustered to both a row baseline and a column baseline to a cell formed by the row baseline and the column baseline; For each cell formed by the row baselines and the column baselines: Determine whether there is a free text block inside the cell; If so, restore the free text block to the cell.

4. The method according to claim 1, characterized in that: The process of obtaining the combination of each text block based on the position information of each text block includes: Based on the position information of each text block, two or more adjacent text blocks that are adjacent to each other vertically or horizontally are determined to obtain each text block combination.

5. The method according to claim 1, characterized in that The clustering information also includes the semantic clustering of each text block, and the method further includes: The header information of the target table is determined according to the semantic clustering of each text block and the row baseline or column baseline to which each text block is clustered.

6. The method according to any one of claims 1 to 5, characterized in that: The process of obtaining the position information and text content of each text block from the table image to be restored includes: The position information and text content of each text block are obtained from the table image to be restored through optical character recognition technology.

7. A table restoration device, characterized in that: include: A text block acquisition unit, used to acquire the position information and text content of each text block from the table image to be restored; An adjacent block acquisition unit, used to acquire each text block combination based on the position information of each text block, each text block combination includes at least two text blocks that are adjacent to each other; A text block merging unit, used to perform named entity recognition on the text content of each text block in each text block combination, and screen out a target text block combination according to the recognition result, merge each text block in the target text block combination into one text block, and update the position information and text content of the merged text block, wherein each text block in the target text block combination can be merged into a single semantic entity; A text block clustering unit, used for performing cluster analysis on each text block based on the position information and text content of each text block in the table image to obtain cluster information, wherein the cluster information includes each row baseline and each column baseline; A cell restoration unit is used to restore each text block to a row-column matrix composed of each row baseline and each column baseline to obtain information of each cell of the restored table; The process of performing cluster analysis on each text block based on the position information and text content of each text block in the table image to obtain cluster information includes: For each block of text: Based on the text content of the text block, embedding the text block into a word vector to obtain a word vector of the text block; Based on the position information of the text block, obtaining a rectangular bounding box of the text block; Fusion the coordinate information of the rectangular bounding box to the word vector to obtain a fused vector of the text block; The fusion vectors of each text block are clustered and analyzed by K-means to obtain clustering information.

8. A table restoration device, characterized in that: include: Memory and processor; The memory is used to store programs; The processor is used to execute the program to implement each step of the table restoration method according to any one of claims 1 to 6.

9. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, each step of the table restoration method according to any one of claims 1 to 6 is implemented.

Images