A table extraction method and electronic device

Abstract

This application provides a table extraction method and an electronic device. The method includes acquiring a document containing a table to be extracted; processing the document to obtain multiple text blocks and text block information corresponding to each text block, wherein the text block information includes semantic vectors, positional information, and type vectors; obtaining a table start pointer sequence, a table content pointer sequence, and a table end pointer sequence based on the text block information; and obtaining the table based on the text blocks, the table start pointer sequence, the table content pointer sequence, and the table end pointer sequence. When extracting tables from a document, extraction is no longer based on lines present in the document, but rather on information from text blocks within the document. This effectively avoids the drawback of tables being unrecognizable due to missing borders, resulting in stronger generalization performance and improved extraction efficiency for tables with few or no lines.

Description

Technical Field

[0001] This application relates to the field of table extraction technology, and in particular to a table extraction method and electronic device. Background Technology

[0002] Tables are a means of organizing data and are widely used in communication, scientific research, and data analysis. Spreadsheet extraction refers to the process of using computers to automatically extract tables from rich document files (such as PDFs, images, etc.), identify their content, and ultimately output a structured format. Table extraction and detection are commonly used in daily work. Since rich document text files currently cannot directly perform secondary processing on tables (such as summarizing, calculating, and transforming), tables need to be extracted and standardized before being processed in spreadsheet software (such as Excel, WPS). Therefore, a tool is needed to quickly extract tables from rich document files.

[0003] Currently, existing table extraction tools, such as pdfplumber and camelot, are all based on the method of extracting wireframes. This method divides table content extraction into table detection and table content recognition steps. During detection, it is necessary to detect horizontal and vertical straight lines that may be used to form a table from the page. Then, the intersection points of the detected lines are determined. Next, based on the intersection points, the smallest cell enclosed by them is found. Finally, the connected cells are integrated to generate a detected table object. Finally, by detecting characters in each divided cell, the content recognition is completed, and the spreadsheet is output.

[0004] This approach works well when table borders are clearly defined in the document, such as... Figure 1 As shown, the extraction results are relatively good. However, in practical applications, to maintain aesthetics, vertical lines are often made invisible when editing tables in documents, such as... Figure 2 As shown, even all borders are set to invisible, such as Figure 3 As shown, at this point, the method of extracting tables based on extracting wireframes is less effective. Summary of the Invention

[0005] The purpose of this application is to provide a table extraction method that, when extracting tables from a document, no longer relies on lines present in the document, but extracts them by utilizing information from text blocks in the document, thereby improving the extraction effect for tables with few or no lines.

[0006] In a first aspect, one technical solution adopted in the embodiments of this application is: providing a table extraction method, comprising: obtaining a document containing a table to be extracted; processing the document to obtain multiple text blocks and text block information corresponding to each text block, wherein the text block information includes semantic vectors, position information, and type vectors; obtaining a table start pointer sequence, a table content pointer sequence, and a table end pointer sequence based on each text block information; and obtaining the table based on each text block, the table start pointer sequence, the table content pointer sequence, and the table end pointer sequence.

[0007] In some embodiments, before processing the document, the method further includes converting the document's format to an image format.

[0008] In some embodiments, processing the document to obtain multiple text blocks and text block information corresponding to each text block includes: performing OCR recognition on the document to obtain the text content of each text block and the position information corresponding to each text block; encoding each text content to obtain the semantic vector and the type vector corresponding to each text block.

[0009] In some embodiments, obtaining the table start pointer sequence, table content pointer sequence, and table end pointer sequence based on the information of each text block includes: obtaining the input vector corresponding to each text block based on each semantic vector and each type vector; obtaining the output vector corresponding to each text block using a self-attention mechanism based on each input vector and each position information; and obtaining the table start pointer sequence, the table content pointer sequence, and the table end pointer sequence based on each output vector.

[0010] In some embodiments, obtaining the output vector corresponding to each text block using a self-attention mechanism based on each input vector and each position information includes: obtaining the relative position encoding between each text block based on each position information; and obtaining each output vector using a self-attention mechanism based on each input vector and each relative position encoding.

[0011] In some embodiments, the calculation formula for the self-attention mechanism is:

[0012]

[0013]

[0014]

[0015] Where, d headLet t be the dimension length of the input vector corresponding to the i-th text block. i Let t be the input vector corresponding to the i-th text block. j W is the input vector corresponding to the j-th text block. Q W is the query parameter for the self-attention mechanism. K W is the keyword parameter for the self-attention mechanism. V The value parameter of the self-attention mechanism is... The position information corresponding to the j-th text block and the position information corresponding to the i-th text block are encoded on the first coordinate axis as a first relative position. The position information corresponding to the j-th text block and the position information corresponding to the i-th text block are encoded on the second coordinate axis in a second relative position, h. i Let x be the output vector corresponding to the i-th text block. j The position information of the j-th text block on the first coordinate axis is given by i and j, where i and j are both integers greater than 0 and less than (N+1), and N is the total number of text blocks.

[0016] In some embodiments, obtaining the table start pointer sequence, the table content pointer sequence, and the table end pointer sequence based on each of the output vectors includes: inputting each of the output vectors into a table start pointer network to obtain the table start pointer sequence; inputting each of the output vectors into a table content pointer network to obtain the table content pointer sequence; and inputting each of the output vectors into a table end pointer network to obtain the table end pointer sequence.

[0017] In some embodiments, obtaining the table based on each of the text blocks, the table start pointer sequence, the table content pointer sequence, and the table end pointer sequence includes: obtaining the table title based on each of the text blocks, the table start pointer sequence, and the table content pointer sequence; obtaining the table body based on each of the text blocks, the table content pointer sequence, and the table end pointer sequence; and obtaining the table based on the title and the body.

[0018] In a second aspect, embodiments of this application provide an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method as described in any one of the first aspects.

[0019] Thirdly, embodiments of this application also provide a computer-readable storage medium storing computer-executable instructions for causing a computer to perform the method described in the first aspect above.

[0020] Fourthly, embodiments of this application also provide a computer program product, the computer program product including a computer program stored on a computer-readable storage medium, the computer program including program instructions, which, when executed by a computer, cause the computer to perform the method described in the first aspect above.

[0021] Compared with existing technologies, the beneficial effects of this application are as follows: Unlike existing technologies, embodiments of this application provide a table extraction method and electronic device. The method includes acquiring a document containing a table to be extracted; processing the document to obtain multiple text blocks and text block information corresponding to each text block, wherein the text block information includes semantic vectors, positional information, and type vectors; obtaining a table start pointer sequence, a table content pointer sequence, and a table end pointer sequence based on each text block information; and obtaining a table based on each text block, the table start pointer sequence, the table content pointer sequence, and the table end pointer sequence. When extracting tables from a document, extraction is no longer based on lines present in the document, but rather on information from text blocks within the document. This effectively avoids the drawback of tables being unrecognizable due to missing borders, resulting in stronger generalization performance and improved extraction efficiency for tables with few or no lines. Attached Figure Description

[0022] One or more embodiments are illustrated by way of example with reference to the accompanying drawings. These illustrations do not constitute a limitation on the embodiments. Elements / modules and steps with the same reference numerals in the drawings are represented as similar elements / modules and steps. Unless otherwise stated, the figures in the drawings do not constitute a limitation on scale.

[0023] Figure 1 This is a schematic diagram of a table structure provided in an embodiment of this application;

[0024] Figure 2 This is another table structure diagram provided in the embodiments of this application;

[0025] Figure 3 This is another schematic diagram of a table structure provided in the embodiments of this application;

[0026] Figure 4 This is a flowchart illustrating a table extraction method provided in an embodiment of this application;

[0027] Figure 5 This is a schematic diagram illustrating a table extraction method provided in an embodiment of this application;

[0028] Figure 6 This is a schematic diagram of the extraction process of a table extraction method provided in an embodiment of this application;

[0029] Figure 7 This is a partial flowchart illustrating a table extraction method provided in an embodiment of this application;

[0030] Figure 8 This is one of the embodiments provided in this application. Figure 4 A flowchart illustrating step S200;

[0031] Figure 9 This is one of the embodiments provided in this application. Figure 4 A flowchart illustrating step S300;

[0032] Figure 10 This is one of the embodiments provided in this application. Figure 9 A flowchart illustrating step S320;

[0033] Figure 11 This is one of the embodiments provided in this application. Figure 9 A flowchart illustrating step S330;

[0034] Figure 12 This is one of the embodiments provided in this application. Figure 4 A flowchart illustrating step S400;

[0035] Figure 13 This is a structural block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0036] The present application will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present application, but do not limit the present application in any way. It should be noted that those skilled in the art can make several modifications and improvements without departing from the concept of the present application. These all fall within the protection scope of the present application.

[0037] To facilitate understanding of this application, a more detailed description is provided below with reference to the accompanying drawings and specific embodiments. Unless otherwise defined, all technical and scientific terms used in this specification have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the application. The term "and / or" as used in this specification includes any and all combinations of one or more of the associated listed items.

[0038] It should be noted that, unless there is a conflict, the various features in the embodiments of this application can be combined with each other, all of which are within the protection scope of this application. Furthermore, although functional modules are divided in the device schematic diagram, in some cases, they can be divided differently from those in the device. In addition, the terms "first" and "second" used herein do not limit the data or execution order, but only distinguish between identical or similar items with essentially the same function and effect.

[0039] Currently available table extraction tools, such as pdfplumber and camelot, are third-party open-source tools, and Alibaba Cloud and Tencent Cloud have also launched corresponding table recognition APIs. However, these methods are all based on lattice extraction of wireframes. They determine the presence of tables by identifying the existence of horizontal and vertical lines in the document, and then detect the characters within the cells to extract the spreadsheet. This method is extremely ineffective when the border lines are not visible. Furthermore, current table algorithms rely heavily on the relative position of tables while neglecting the understanding of the table content itself. In reality, tables contain multiple types of information, including their content (semantic information), content type (text type), and relative position. Each type is important, but relying on only one piece of information to determine the existence of a table and extract its content is insufficient; a multi-faceted approach is necessary. Therefore, a new table extraction method is needed that can extract various types of tables and, in addition to relative position, utilize semantic information and other aspects to identify table information from multiple perspectives, thus understanding the table's content.

[0040] Based on this, this application proposes a table extraction method. This method utilizes text block information to obtain table pointers, and then outputs the table based on these pointers. When extracting tables from a document, it no longer relies on lines present in the document, but instead extracts them using information from text blocks. This allows for the extraction of various table types and improves the extraction efficiency for tables with few or no lines. Furthermore, the extraction process considers the semantic information, text type, and positional information of the text blocks, enabling table extraction from multiple angles and modalities, thus avoiding the shortcomings of table extraction from a single angle.

[0041] Firstly, this application provides a table extraction method, please refer to [link to relevant documentation]. Figure 4 The table extraction methods include:

[0042] Step S100: Obtain the document containing the table to be extracted.

[0043] The document is a rich document file, and its format can be pdf, doc, docx, png, jpg, etc.

[0044] Step S200: Process the document to obtain multiple text blocks and text block information corresponding to each text block. The text block information includes semantic vectors, position information, and type vectors.

[0045] Please see Figure 5 , Figure 5 (a) in the document is the document from which the table is to be extracted. Figure 5 The dashed boxes in (b) represent the bounding boxes of the identified text blocks, which include semantic and positional information within the bounding boxes.

[0046] Semantic information refers to the text information within the bounding box. The semantic vector of a text block is a vector of the semantic information corresponding to the text block, which can be obtained by encoding the semantic information.

[0047] Location information can be obtained by averaging the two-dimensional coordinates of the four corners of the bounding box, which is also the two-dimensional coordinates of the center within the bounding box. Understandably, before obtaining the location information, a two-dimensional Cartesian coordinate system needs to be established on the document. This system includes a first and second coordinate axis that are perpendicular to each other, such as... Figure 5 In this context, the positive direction of the first coordinate axis can be defined as the horizontal direction to the right, and the positive direction of the second coordinate axis can be defined as the vertical direction downwards. This allows us to obtain the two-dimensional coordinates (x, y) corresponding to the i-th text block. i ,y i ), which is the position information corresponding to the i-th text block, where i is an integer greater than 1 and less than (N+1), and N is the total number of text blocks.

[0048] A text type refers to the text type of text information, which includes at least one of Chinese text type, English text type, and numeric text type. A type vector is a vector of the text type corresponding to a text block, which can be obtained by encoding the text type.

[0049] Step S300: Based on the information of each text block, obtain the table start pointer sequence, table content pointer sequence, and table end pointer sequence.

[0050] Specifically, the text blocks can be pre-sorted, following the document editing order. Typically, document editing proceeds from left to right and top to bottom; therefore, when sorting the text blocks, they should also be sorted according to their corresponding position information, following a left-to-right and top-to-bottom order. Figure 5In the document shown, when sorting, the text block containing text 1 should be sorted in the following order: text block containing text 2, text block containing heading 1, text block containing heading 2, ..., text block containing content 3, text block containing content 4. In this case, the second text block is the text block containing text 2.

[0051] Next, please refer to Figure 6 The information of each text block is sequentially input into the encoding layer and the pointer network according to their positional order. Finally, the pointer network outputs the table start pointer sequence, table content pointer sequence, and table end pointer sequence. The order of the table start pointer sequence, table content pointer sequence, and table end pointer sequence obtained in the above manner is consistent with the order of the text blocks, which facilitates the subsequent extraction of text blocks to obtain the table based on each pointer sequence.

[0052] If a table exists in the document, there will be at least one table start pointer in the table start pointer sequence, at least one table content pointer in the table content pointer sequence, and at least one table end pointer in the table end pointer sequence. Since their order matches the text block position order, and the text block position order matches the document editing order, the text blocks from the table start pointer to the table end pointer all contain table information and not other irrelevant content. The table content pointers distinguish between the actual table content and the table title; that is, the text blocks from the table start pointer to the text block corresponding to the previous pointer of the table content pointer contain the table title, and the text blocks from the table content pointer to the text block corresponding to the table end pointer contain the table body. If the current document does not contain a table, both the start and end pointers will be null.

[0053] Step S400: Obtain the table based on each text block, the table start pointer sequence, the table content pointer sequence, and the table end pointer sequence.

[0054] Specifically, after obtaining the aforementioned pointer sequence, we can determine whether a table start pointer exists in the table start pointer sequence and whether a table end pointer exists in the table end pointer sequence. If both table start and end pointers exist, it indicates that a table exists in the current document, thus completing the table detection task. Next, in sequence, the text block corresponding to the table start pointer up to the text block corresponding to the previous pointer of the table content pointer is the table title, and the text block between the text block corresponding to the table content pointer and the text block corresponding to the table end pointer is the table body. After extracting the title and body, the structured output of the table can be completed. The extracted table can then be displayed in other format types, such as a web interface.

[0055] In this embodiment, when extracting spreadsheets from a document, extraction is no longer based on lines present in the document, but rather on information from text blocks within the document. That is, the method of extracting tables based on text blocks is adopted instead of extracting tables based on lines. This effectively avoids the drawback of tables not being recognized due to missing borders, resulting in stronger generalization performance and improving the extraction effect on tables with few or no lines.

[0056] Moreover, compared to the method of first performing table detection and then decomposing the table structure after the detection is passed, this embodiment uses a pointer network structure to jointly output the table detection result and the table, which can simultaneously output the table recognition result and the table content, reduce the propagation of errors, and improve the accuracy of table extraction.

[0057] Furthermore, text types in the same column of a table are typically the same. In this embodiment, however, text type is considered during table extraction, improving the accuracy of table recognition and extraction. Moreover, this embodiment considers table text features, text position, and text type during extraction, enabling multi-faceted and multi-modal table extraction and avoiding the limitations of single-perspective extraction.

[0058] In some of these embodiments, please refer to Figure 7 Before step S200, the method further includes:

[0059] Step S110: Convert the document format to image format.

[0060] Specifically, after obtaining the rich document file, it can be uniformly converted into JPG image format. For example, after obtaining the rich document file, you can first determine whether the document format is JPG. If it is, no conversion is needed; if not, you can use an open-source third-party conversion toolkit available online to convert the document format to JPG.

[0061] In this embodiment, by uniformly converting the document format to JPG image format, unified management is facilitated. Only the document processing method under JPG image format needs to be studied, eliminating the need to study processing methods under other formats, thus reducing research costs and difficulty. Furthermore, converting the document format to image format facilitates subsequent document processing to obtain multiple text blocks and their corresponding text block information. In practical applications, other image formats such as PNG can also be used.

[0062] In some of these embodiments, please refer to Figure 8 Step S200 includes:

[0063] Step S210: Perform OCR recognition on the document to obtain the text content of each text block and the corresponding position information of each text block.

[0064] OCR (Optical Character Recognition) is a process that converts image shapes into text characters. Specifically, after obtaining a document in image format, open-source OCR tools available online can be used to perform OCR recognition on the document. These open-source OCR tools can include Baidu's PadLeoCr tool or other open-source tools.

[0065] Step S220: Encode each text content to obtain the semantic vector and type vector corresponding to each text block.

[0066] First, the text content is encoded to obtain the semantic information and text type corresponding to each text block. Then, the semantic information is encoded to obtain the semantic vector, and the text type is encoded to obtain the type vector.

[0067] Since some text blocks contain little semantic information, or even just a single number, overly complex models should not be used when encoding them. Specifically, word segmentation can be performed on each semantic information segment to obtain a corresponding word segmentation sequence; a word vector dictionary can be obtained; based on the word segmentation sequence and the word vector dictionary, the word vector corresponding to each semantic information can be obtained; and the weighted average of the word vectors corresponding to each semantic information can be calculated to obtain the semantic vector. The word vector dictionary can be obtained through pre-training. In practical applications, word segmentation can refer to existing techniques and is not limited here.

[0068] When encoding different text types, one-hot encoding can be used to encode each text type, resulting in a vector for each type. One-hot encoding, also known as one-bit encoding, can be referenced from existing technologies and is not limited here.

[0069] In this embodiment, after OCR recognition of the document and subsequent encoding, each text block, its corresponding position information, its corresponding semantic vector, and its corresponding type vector can be obtained.

[0070] In some of these embodiments, please refer to Figure 9 Step S300 includes:

[0071] Step S310: Based on each semantic vector and each type vector, obtain the input vector corresponding to each text block.

[0072] Specifically, after obtaining the information of each text block, the information of each text block can be processed in the order of the text blocks to obtain the input vector corresponding to each text block. The processing formula at this time can be:

[0073] t i =SenEmb(i) + TypeEmb(i);

[0074] Among them, t i SenEmb(i) is the input vector corresponding to the i-th text block, and TypeEmb(i) is the semantic vector of the i-th text block.

[0075] Step S320: Based on each input vector and each position information, use a self-attention mechanism to obtain the output vector corresponding to each text block.

[0076] Please see Figure 6 After obtaining each input vector, the text blocks are input into the self-attention layer according to their positional order, i.e., from left to right and from top to bottom. The self-attention mechanism is used to encode the input vectors to obtain the output vectors corresponding to each text block.

[0077] Step S330: Based on each output vector, obtain the table start pointer sequence, table content pointer sequence, and table end pointer sequence.

[0078] Finally, based on each output vector, the results are output using a pointer network method.

[0079] In this embodiment, a novel network structure for table extraction is provided by processing the input vector using a self-attention mechanism and a pointer network. Employing an end-to-end deep learning algorithm, it can simultaneously perform table detection and table structure extraction. Compared to rule-based methods, this network structure can learn and train the relative weights and relationships of specific features, effectively avoiding model deviations caused by human subjective factors, making it more consistent with actual data, and improving the effectiveness of table extraction.

[0080] In some of these embodiments, please refer to Figure 10 Step S320 includes:

[0081] Step S321: Obtain the relative position encoding between each text block based on the position information.

[0082] Specifically, based on the location information, the first relative distance x between the j-th text block and the i-th text block on the first coordinate axis can be obtained. j -x i And obtain the second relative distance y between the j-th text block and the i-th text block on the second coordinate axis. j -y i j is an integer greater than 1 and less than (N+1); then, for the first relative distance x j -x i Second relative distance y j -y iEncode the position information corresponding to the j-th text block and the position information corresponding to the i-th text block to obtain the first relative position encoding on the first coordinate axis. And obtain the position information corresponding to the j-th text block and the position information corresponding to the i-th text block on the second coordinate axis, and the second relative position encoding.

[0083] Among them, for the first relative distance x j -x i Second relative distance y j -y i Encoding can be performed using a sine function to represent the first relative distance x. j -x i The output value mapped to the sine function is used to obtain the first relative position code through a first preset mapping relationship. And using the sine function to calculate the second relative distance y j -y i The output value of the sine function is used to obtain the second relative position code through a second preset mapping relationship. The first preset mapping relationship is the mapping relationship between the sinusoidal output value of the first relative distance and the encoded value of the first relative position, and the second preset relationship is the mapping relationship between the sinusoidal output value of the second relative distance and the encoded value of the second relative position. In practical applications, the specific encoding methods for the first and second relative distances may differ, and the first and second preset mapping relationships should be inconsistent to distinguish between the first and second coordinate axes.

[0084] Step S322: Based on each input vector and each relative position encoding, a self-attention mechanism is used to obtain each output vector.

[0085] Specifically, the calculation formula for the self-attention mechanism is as follows:

[0086]

[0087]

[0088]

[0089] Where, d head Let t be the dimension length of the input vector corresponding to the i-th text block. i Let t be the input vector corresponding to the i-th text block. j Let W be the input vector corresponding to the j-th text block. Q For the query (Q) parameter of the self-attention mechanism, W K For the key (K) parameter of the self-attention mechanism, W V The value (Vue, V) parameter represents the value of the self-attention mechanism. Encode the first relative position of the position information corresponding to the j-th text block and the position information corresponding to the i-th text block on the first coordinate axis. The position information corresponding to the j-th text block and the position information corresponding to the i-th text block are encoded on the second coordinate axis in terms of their second relative positions, h. i Let x be the output vector corresponding to the i-th text block. j Let i represent the position information of the j-th text block on the first coordinate axis, where i and j are both integers greater than 0 and less than (N+1), and N is the total number of text blocks. The Q, K, and V parameters in the self-attention mechanism can be obtained through pre-learning and training.

[0090] As can be seen, the self-attention mechanism in this embodiment not only considers the semantic information and text type of each text block, but also the two-dimensional relative position between each text block. It can consider table extraction from multiple angles and multiple modalities, avoid the defects of table extraction from a single angle, and improve the table extraction effect.

[0091] In some of these embodiments, please refer to Figure 11 Step S330 includes:

[0092] Step S331: Input each of the output vectors into the table start pointer network to obtain the table start pointer sequence;

[0093] Step S332: Input each of the output vectors into the table content pointer network to obtain the table content pointer sequence;

[0094] Step S333: Input each of the output vectors into the table end pointer network to obtain the table end pointer sequence.

[0095] Specifically, the formulas for the table start pointer network, table content pointer network, and table end pointer network can all be obtained through pre-training. The formulas before training are as follows:

[0096] u j =v T tanh(W1h j );

[0097] P(C j |C1, ..., C N ) = softmax(u j );

[0098] Among them, V T Let W1 be a randomly initialized vector, and h be the trainable weights. j Let C be the output vector corresponding to the j-th text block. jThis represents the table pointer (start pointer, content pointer, or end pointer) corresponding to the j-th text block. By training the three pointer networks with a large number of documents, randomly initialized vectors and weights W1 can be obtained for the start pointer network, content pointer network, and end pointer network. Furthermore, the output value of each pointer network is either 0 or 1; 0 indicates no pointer at that position, and 1 indicates a pointer exists at that position. For example... Figure 6 As shown, if there is a 1 in the table start pointer sequence, it indicates that a table start pointer exists at that position.

[0099] In this embodiment, by inputting the output vector into the pointer network, the table start pointer sequence, table content pointer sequence, and table end pointer sequence can be obtained. Subsequently, the existence of the table can be determined based on the above pointer sequences, and the table can be output.

[0100] In some of these embodiments, please refer to Figure 12 Step S400 includes:

[0101] Step S410: Obtain the table title based on the text blocks, the table start pointer sequence, and the table content pointer sequence;

[0102] Step S420: Obtain the table text based on the text blocks, the table content pointer sequence, and the table end pointer sequence;

[0103] Step S430: Obtain a table based on the title and body text.

[0104] Specifically, such as Figure 6 As shown, the presence of a table start pointer in the table start pointer sequence and a table end pointer in the table end pointer sequence indicates the existence of a table in the document. The text block corresponding to the table start pointer and the text block corresponding to the previous pointer of the table content pointer constitutes the table title, while the text block corresponding to the table content pointer and the text block corresponding to the table end pointer constitutes the table body. Furthermore, if the table title has n text blocks, then starting from the text body, every n text blocks represents the next row of the current table. For example... Figure 6 As shown, the text block corresponding to the start pointer of the table is the text block where Heading 1 is located, the text block corresponding to the content pointer of the table is the text block where Content 1 is located, and the text block corresponding to the end pointer of the table is the text block where Content 4 is located. That is, the title of the table is the text block where Heading 1 is located to the text block where Heading 4 is located, and the body of the table is the text block where Content 1 is located to the text block where Content 4 is located.

[0105] As can be seen, in this embodiment, after obtaining the result of the pointer network, the table title and table content can be displayed in a row-by-row format, and the entire table can be presented in a structured form for input to downstream tasks.

[0106] Secondly, this application also provides an electronic device, please refer to... Figure 13 It demonstrates the ability to execute Figure 4 , Figures 7 to 12 The hardware structure of the electronic device for the table extraction method described above.

[0107] The electronic device includes: at least one processor 110; and a memory 120 communicatively connected to the at least one processor 110. Figure 13 Taking a processor 110 as an example, the memory 120 stores instructions that can be executed by at least one processor 110. These instructions are executed by at least one processor 110 to enable the at least one processor 110 to perform the aforementioned... Figure 4 , Figures 7 to 12 The table extraction method described above. The processor 110 and memory 120 can be connected via a bus or other means. Figure 13 Taking the example of a connection between China and Israel via a bus.

[0108] The memory 120, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules, such as the program instructions / modules corresponding to the methods in the embodiments of this application. The processor 110 executes various functional applications and data processing of the server by running the non-volatile software programs, instructions, and modules stored in the memory 120, that is, it implements the table extraction method in the above method embodiments.

[0109] The memory 120 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the table retrieval device, etc. Furthermore, the memory 120 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 120 may optionally include memory remotely located relative to the processor 110, and these remote memories can be connected to the electronic device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0110] One or more modules are stored in memory 120. When executed by one or more processors 110, they perform the table extraction method in any of the above method embodiments, for example, the method described above. Figure 4 , Figures 7 to 12 The steps of the table extraction method are described above.

[0111] The above-described product can execute the table extraction method provided in the embodiments of this application, and has the corresponding functional modules and beneficial effects of the method. Technical details not described in detail in this embodiment can be found in the table extraction method provided in the embodiments of this application.

[0112] This application also provides a non-volatile computer-readable storage medium storing computer-executable instructions that are executed by one or more processors, for example, to execute the instructions described above. Figure 4 , Figures 7 to 12 The steps of the table extraction method are described above.

[0113] This application also provides a computer program product, including a computing program stored on a non-volatile computer-readable storage medium. The computer program includes program instructions, which, when executed by a computer, cause the computer to perform the table extraction method in any of the above method embodiments, for example, to perform the above-described... Figure 4 , Figures 7 to 12 The steps of the table extraction method described above.

[0114] It should be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0115] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a general-purpose hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the related technology, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions for at least one computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments.

[0116] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and not to limit them; under the concept of this application, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations of different aspects of this application as described above, which are not provided in detail for the sake of brevity; although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A method for extracting data from a table, characterized in that, include: Retrieve the document containing the table to be extracted; The document is processed to obtain multiple text blocks and text block information corresponding to each text block, wherein the text block information includes semantic vectors, position information and type vectors; Based on the semantic vectors and type vectors, the input vectors corresponding to each text block are obtained; Based on the location information, the relative position encoding between the text blocks is obtained; Based on the input vectors and relative position codes, a self-attention mechanism is used to obtain the output vector corresponding to each text block. The calculation formula for the self-attention mechanism is as follows: ； ； ； Where, d head Let t be the dimension length of the input vector corresponding to the i-th text block. i Let t be the input vector corresponding to the i-th text block. j W is the input vector corresponding to the j-th text block. Q W is the query parameter for the self-attention mechanism. K W is the keyword parameter for the self-attention mechanism. V The value parameter of the self-attention mechanism is... The position information corresponding to the j-th text block and the position information corresponding to the i-th text block are encoded on the first coordinate axis as a first relative position. The position information corresponding to the j-th text block and the position information corresponding to the i-th text block are encoded on the second coordinate axis in a second relative position, h. i Let x be the output vector corresponding to the i-th text block. j The position information of the j-th text block on the first coordinate axis, where i and j are both integers greater than 0 and less than (N+1), and N is the total number of text blocks; Based on the output vectors described above, we obtain the table start pointer sequence, table content pointer sequence, and table end pointer sequence; The table is obtained based on each of the text blocks, the table start pointer sequence, the table content pointer sequence, and the table end pointer sequence.

2. The table extraction method according to claim 1, characterized in that, Before processing the document, the method further includes: Convert the document to image format.

3. The table extraction method according to claim 2, characterized in that, The process of processing the document to obtain multiple text blocks and text block information corresponding to each text block includes: The document is subjected to OCR recognition to obtain the text content of each text block and the corresponding position information of each text block; The text content is encoded to obtain the semantic vector and the type vector corresponding to each text block.

4. The table extraction method according to claim 1, characterized in that, The step of obtaining the table start pointer sequence, the table content pointer sequence, and the table end pointer sequence based on each of the output vectors includes: Each of the output vectors is input into the table start pointer network to obtain the table start pointer sequence; Each of the output vectors is input into the table content pointer network to obtain the table content pointer sequence; Each of the output vectors is input into the table end pointer network to obtain the table end pointer sequence.

5. The table extraction method according to any one of claims 1-3, characterized in that, The step of obtaining the table based on each of the text blocks, the table start pointer sequence, the table content pointer sequence, and the table end pointer sequence includes: The title of the table is obtained based on each of the text blocks, the table start pointer sequence, and the table content pointer sequence; The main text of the table is obtained based on each of the text blocks, the table content pointer sequence, and the table end pointer sequence; The table is obtained based on the title and the body text.

6. An electronic device, characterized in that, include: At least one processor; as well as, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method as described in any one of claims 1-5.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions for causing a computer to perform the method as described in any one of claims 1-5.

Images