Information extraction method, device, equipment and storage medium

Abstract

The embodiments of the present application provide an information extraction method, apparatus, device and storage medium, which relate to the field of computer technology, including: determining matching text that matches triple identification information from the text content of a table to be extracted, and the position information of the matching text in the table to be extracted; determining the table style of the table to be extracted based on the position information of the matching text in the table to be extracted; extracting combined text from the table to be extracted based on the table style of the table to be extracted and the position information of the matching text in the table to be extracted; and determining at least one target triple content based on the triple identification information and the combined text through an information extraction model. The embodiments of the present application realize automated and more refined information extraction, greatly reducing labor costs and effectively improving the extraction efficiency of information extraction.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of computer technology, and in particular to an information extraction method, apparatus, device, and storage medium. Background Art

[0002] Knowledge graphs represent knowledge as triples and are a type of knowledge base presented in the form of a graph. Knowledge graph technology has been widely applied in various fields. Knowledge graph data sources primarily include databases, text, and tables. Tabular data, typically structured or semi-structured, is a key source of knowledge graph data.

[0003] Under related technologies, the main methods for extracting table information include the following: First, manually configured rule extraction, that is, business experts observe the tables. If the number of tables is small, the business experts directly mark and extract the table information. If the number of tables is large, the business experts need to first configure a series of rules before extracting the table information. Therefore, this method requires high labor costs and is inefficient in extracting table information. Second, specific format table information extraction, that is, using manual preprocessing to convert the table into a table in a specific format, and then extracting data from the table. Since this method relies on manual intervention, it is not suitable for situations where there is a lot of table information or the formats are diverse, and the efficiency of table information extraction is low.

[0004] Therefore, how to reduce manual intervention in the table information extraction process, reduce labor costs, and effectively improve the extraction efficiency of table information extraction is a technical problem that needs to be solved urgently under related technologies. Summary of the Invention

[0005] The embodiment of the present application provides an information extraction method, which reduces labor costs and effectively improves the extraction efficiency of information extraction.

[0006] In a first aspect, an embodiment of the present application provides an information extraction method, comprising:

[0007] Determine, from the text content of the table to be extracted, a matching text that matches the triple identification information, and position information of the matching text in the table to be extracted;

[0008] Determining a table style of the table to be extracted based on position information of the matching text in the table to be extracted;

[0009] Extracting the combined text from the table to be extracted based on the table style of the table to be extracted and the position information of the matching text in the table to be extracted;

[0010] At least one target triple content is determined based on the triple identification information and the combined text through an information extraction model.

[0011] In an embodiment of the present application, by determining the matching text that matches the triple identification information and the position information of the matching text in the table to be extracted from the text content of the table to be extracted, and then determining the table style of the table to be extracted, and then extracting the combined text from the table to be extracted based on the table style of the table to be extracted and the position information of the matching text in the table to be extracted, and finally inputting the triple identification information and the combined text into the information extraction model to determine at least one target triple content, thereby realizing automated and more refined information extraction, greatly reducing labor costs, and effectively improving the extraction efficiency of information extraction.

[0012] In an optional implementation, format conversion and table parsing are performed on the document to be processed to obtain the table to be extracted; and the table to be extracted is stored in a two-dimensional matrix form.

[0013] In an optional implementation manner, the triplet identification information includes: a first entity identification and a second entity identification;

[0014] For the text content in each cell in the table to be extracted, perform the following operations respectively:

[0015] Inputting the first entity identifier and the text content of a cell into a text similarity model to obtain a first similarity between the first entity identifier and the text content of the cell;

[0016] Inputting the second entity identifier and the text content of a cell into a text similarity model to obtain a second similarity between the second entity identifier and the text content of the cell;

[0017] Based on the obtained multiple first similarities and the obtained multiple second similarities, matching texts that respectively match the first entity identifier and the second entity identifier and corresponding position information are determined from the text content of the table to be extracted.

[0018] Under the above implementation, based on the multiple first similarities obtained and the multiple second similarities obtained, the matching texts and corresponding location information that match the first entity identifier and the second entity identifier respectively are determined from the text content of the table to be extracted. This process does not require manual intervention, effectively reduces labor costs, and thus improves the extraction efficiency of subsequent information extraction.

[0019] In an optional embodiment, the position information of the matching text that matches the triple identification information in the table to be extracted includes: the first cell coordinates corresponding to the matching text of the first entity identification, and the second cell coordinates corresponding to the matching text of the second entity identification;

[0020] The multiple first similarities, the multiple second similarities, the triple identification information and the similarity threshold are input into a matching function to obtain a target dictionary corresponding to the triple identification information, wherein the target dictionary includes: the first entity identifier, the first cell coordinates, the similarity between the first entity identifier and the corresponding matching text, the second entity identifier, the second cell coordinates, and the similarity between the second entity identifier and the corresponding matching text.

[0021] In the above implementation, the process of obtaining the target dictionary corresponding to the triple identification information through the matching function does not require manual intervention, which effectively reduces labor costs.

[0022] In an optional implementation, candidate text is extracted from the table to be extracted based on the table style of the table to be extracted and the position information of the matching text in the table to be extracted;

[0023] Locating table title text from the table to be extracted;

[0024] The table title text and the candidate text are combined to obtain the combined text.

[0025] Under the above implementation, not only the text content of the table to be extracted is included in the information extraction scope, but also the importance of the table title text is fully measured, the table title text and the candidate text are combined to obtain the combined text, which effectively improves the accuracy of information extraction.

[0026] In an optional implementation, the triple identification information and the combined text are input into an information extraction model to obtain a plurality of preliminary triple contents; the plurality of preliminary triple contents are deduplicated and screened to determine at least one target triple content.

[0027] Under the above implementation method, a method combining document information extraction and paragraph information extraction is adopted. By inputting triple identification information and combined text into the information extraction model, and then deduplicating and screening the multiple preliminary triple contents obtained, at least one target triple content is determined, thereby effectively improving the information extraction efficiency.

[0028] In an optional implementation manner, the table styles of the table to be extracted include: a horizontal table and a vertical table.

[0029] In a second aspect, an embodiment of the present application provides an information extraction device, comprising:

[0030] An information matching module, configured to determine, from the text content of the table to be extracted, a matching text that matches the triple identification information, and position information of the matching text in the table to be extracted;

[0031] a table style determination module, configured to determine the table style of the table to be extracted based on the position information of the matching text in the table to be extracted;

[0032] A combined text extraction module, configured to extract the combined text from the table to be extracted based on the table style of the table to be extracted and the position information of the matching text in the table to be extracted;

[0033] The information extraction module is used to determine at least one target triple content based on the triple identification information and the combined text through an information extraction model.

[0034] The beneficial effects of the embodiments of the present application can be referred to the beneficial effects of the above-mentioned first aspect and each embodiment of the first aspect, and will not be repeated here.

[0035] In an optional embodiment, a document pre-processing module is further included;

[0036] The document preprocessing module is specifically used for:

[0037] Performing format conversion and table parsing on the document to be processed to obtain the table to be extracted; and storing the table to be extracted in a two-dimensional matrix form.

[0038] In an optional implementation manner, the information matching module is specifically configured to:

[0039] For the text content in each cell in the table to be extracted, perform the following operations respectively:

[0040] Inputting the first entity identifier and the text content of a cell into a text similarity model to obtain a first similarity between the first entity identifier and the text content of the cell;

[0041] Inputting the second entity identifier and the text content of a cell into a text similarity model to obtain a second similarity between the second entity identifier and the text content of the cell;

[0042] Based on the obtained multiple first similarities and the obtained multiple second similarities, matching texts that respectively match the first entity identifier and the second entity identifier and corresponding position information are determined from the text content of the table to be extracted.

[0043] In an optional implementation manner, the information matching module is specifically configured to:

[0044] The multiple first similarities, the multiple second similarities, the triple identification information and the similarity threshold are input into a matching function to obtain a target dictionary corresponding to the triple identification information, wherein the target dictionary includes: the first entity identifier, the first cell coordinates, the similarity between the first entity identifier and the corresponding matching text, the second entity identifier, the second cell coordinates, and the similarity between the second entity identifier and the corresponding matching text.

[0045] In an optional implementation, the combined text extraction module is specifically configured to:

[0046] Extracting candidate text from the table to be extracted based on the table style of the table to be extracted and the position information of the matching text in the table to be extracted;

[0047] Locating table title text from the table to be extracted;

[0048] The table title text and the candidate text are combined to obtain the combined text.

[0049] In an optional implementation manner, the information extraction module is specifically configured to:

[0050] The triple identification information and the combined text are input into an information extraction model to obtain a plurality of preliminary triple contents; the plurality of preliminary triple contents are deduplicated and screened to determine at least one target triple content.

[0051] In a third aspect, an embodiment of the present application provides a computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned information extraction method when executing the program.

[0052] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium storing a computer program executable by a computer device. When the program runs on the computer device, the computer device executes the steps of the above-mentioned information extraction method.

[0053] In a fifth aspect, an embodiment of the present application provides a computer program product, which includes a computer program stored on a computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer device, the computer device executes the steps of the above-mentioned information extraction method. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0055] Figure 1 A schematic diagram of the structure of a system architecture provided in an embodiment of the present application;

[0056] Figure 2 A flowchart of an information extraction method provided in an embodiment of the present application;

[0057] Figure 3 A schematic diagram of a table structure provided in an embodiment of the present application;

[0058] Figure 4 A schematic diagram of a table storage format provided in an embodiment of the present application;

[0059] Figure 5 A schematic diagram of a process for extracting table information provided in an embodiment of the present application;

[0060] Figure 6 A schematic diagram of the structure of an information extraction device provided in an embodiment of the present application;

[0061] Figure 7 A schematic diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0062] In order to make the purpose, technical solutions and beneficial effects of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0063] For ease of understanding, the terms involved in the embodiments of the present invention are explained below.

[0064] Knowledge graph: A series of various graphics that show the development process and structural relationship of knowledge, using visualization technology to describe knowledge resources and their carriers, and to mine, analyze, construct, draw and display knowledge and their interrelationships.

[0065] Triples: The core of the knowledge graph is actually triples. Triples are composed of entities, attributes, and relations, and are in the form of [entity, relationship, entity] or [entity, attribute, attribute value].

[0066] Structured data: Structured data is logically expressed and implemented by a two-dimensional table structure, strictly follows data format and length specifications, and is mainly stored and managed through relational databases.

[0067] Semi-structured data: refers to data with irregular or incomplete structure, no predefined data model, and data that is not convenient to be represented by a two-dimensional logical table in a database.

[0068] Bert model: The full English name is Bidirectional Encoder Representation from Transformers. The Bert model is a pre-training model.

[0069] UIE model: Universal Information Extraction model, the full English name is Universal Information Extraction. The UIE model is a Chinese universal information extraction model built based on the ERNIE 3.0 knowledge enhancement pre-training model. This model can achieve unified modeling of tasks such as entity extraction, relationship extraction, event extraction, and sentiment analysis, and has good migration and generalization capabilities.

[0070] See also Figure 1 , which is a system architecture diagram applicable to an embodiment of the present application. The system architecture includes at least a terminal device 101 and an information extraction system 102. The number of terminal devices 101 can be one or more, and the number of information extraction systems 102 can also be one or more. This application does not specifically limit the number of terminal devices 101 and information extraction systems 102.

[0071] Applications are pre-installed in the terminal device 101, where the applications are client applications, web applications, mini-program applications, etc. The terminal device 101 can be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc., but is not limited thereto.

[0072] The information extraction system 102 is the backend server of the application. The information extraction system 102 can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. The terminal device 101 and the information extraction system 102 can be directly or indirectly connected via wired or wireless communication, which is not limited in this application.

[0073] The information extraction method in the embodiment of the present application can be executed by the terminal device 101, or by the information extraction system 102, or can be executed interactively by the terminal device 101 and the information extraction system 102.

[0074] It should be noted that the application scenarios of the information extraction method provided in this application include but are not limited to: information extraction from log files, XML documents, HTML documents, etc.

[0075] based on Figure 1 The system architecture diagram shown in FIG, the embodiment of the present application provides a flow chart of an information extraction method, such as Figure 2 As shown, the process of the method is executed by a computer device, which may be Figure 1 The terminal device 101 and / or information extraction system 102 shown includes the following steps:

[0076] Step 201 : determining, from the text content of the table to be extracted, the matching text that matches the triple identification information, and the position information of the matching text in the table to be extracted.

[0077] Specifically, triple identification information can be: entity pair, entity attribute pair, entity and attribute, etc.

[0078] In an optional implementation, format conversion and table parsing are performed on the document to be processed to obtain the table to be extracted; and the table to be extracted is stored in the form of a two-dimensional matrix.

[0079] Specifically, data preprocessing is performed on the document to be processed, and data preprocessing includes format conversion and table parsing. Format conversion refers to converting the PDF file in the document to be processed into a word document. For example, the format conversion module can convert the PDF format into the word format through the pdf2docx module in python; table parsing refers to parsing the table to be extracted from the word document. For example, the table parsing module parses the table to be extracted through the python-docx module in python.

[0080] A document may contain more than one table. Therefore, all tables to be extracted are stored sequentially in a table data list obtained through data preprocessing. The shape is (k,m,n), where k is the number of tables, and m and n are the maximum horizontal and vertical lengths of the table. Each table to be extracted is stored in the form of a two-dimensional matrix, and each element in the matrix corresponds to a cell in the table to be extracted.

[0081] For example, see Figure 3 , assuming that there is only one table to be extracted in the document to be extracted, the storage format of the table to be extracted is as follows Figure 4It should be noted that when a table spans rows or columns, the corresponding cells will record the same information; by observing Figure 3 It can also be found from the table information that semi-structured table information cannot be used directly. For example, the attribute value of "platform account" is a note description, which requires further information extraction. For example, "application platform function" has three attributes and may also need further differentiation. At the same time, there are also cases where there is no corresponding attribute value, so further information extraction is also required.

[0082] In an optional embodiment, the triple identification information includes: a first entity identifier and a second entity identifier; for the text content in each cell in the table to be extracted, the following operations are performed: the first entity identifier and the text content of the cell are input into a text similarity model to obtain a first similarity between the first entity identifier and the text content of the cell; the second entity identifier and the text content of the cell are input into the text similarity model to obtain a second similarity between the second entity identifier and the text content of the cell;

[0083] Based on the obtained multiple first similarities and the obtained multiple second similarities, matching texts that respectively match the first entity identifier and the second entity identifier and corresponding position information are determined from the text content of the table to be extracted.

[0084] Specifically, text similarity models include but are not limited to: Bert model, DRCN model (Deeply-Recursive Convolutional Network, super-resolution network model), DL model (Deep Learning, neuron model), etc.

[0085] The obtained multiple first similarities form a first similarity matrix; at the same time, the obtained multiple second similarities form a second similarity matrix; the value range of each element in the first similarity matrix and the second similarity matrix is ​​(0,1].

[0086] Based on the obtained first similarity matrix and second similarity matrix, a mapping between the first entity identifier and the second entity identifier and the cell is established, and matching texts that match the first entity identifier and the second entity identifier respectively and corresponding position information are determined from the text content of the table to be extracted.

[0087] In an optional implementation, the position information of the matching text matching the triple identification information in the table to be extracted includes: the first cell coordinates corresponding to the matching text of the first entity identification, and the second cell coordinates corresponding to the matching text of the second entity identification.

[0088] Multiple first similarities, multiple second similarities, triple identification information and a similarity threshold are input into a matching function to obtain a target dictionary corresponding to the triple identification information, where the target dictionary includes: a first entity identifier, a first cell coordinate, the similarity between the first entity identifier and the corresponding matching text, a second entity identifier, a second cell coordinate, and the similarity between the second entity identifier and the corresponding matching text.

[0089] Specifically, in order to ensure that the first entity identifier and the second entity identifier are not mapped to the same cell, an embodiment of the present application provides another information matching method.

[0090] Specifically, head_entity represents the first entity identifier, and tail_entity represents the second entity identifier. max_model is a matching function that returns the cell position and similarity with the highest similarity within a range; the input includes the first similarity matrix, the second similarity matrix, the triple identifier information, the minimum threshold (min_sim), and the maximum threshold (max_sim). The output is map_dict, which is a dictionary whose key is the triple identifier information and whose value is a list of length 2. The first element is the cell coordinate, and the second element is the similarity between the triple identifier information and the corresponding matching text.

[0091] For example, suppose Figure 3 The triple identification information to be extracted is "attribute" and "attribute value", then the first entity is identified as "attribute", and the second entity is identified as "attribute value". Multiple first similarities in the first similarity matrix, multiple second similarities in the second similarity matrix, "attribute", "attribute value", the minimum threshold (min_sim), and the maximum threshold (max_sim) are input into the matching function, and the target dictionary map_dict is output.

[0092] Target dictionary map_dict = {'attribute': [[0,1],1.0], 'attribute value': [[0,4],1.0]}, where "'attribute': [[0,1],1.0]" means: the location information of the matching text of the first entity identifier 'attribute' is row 0 and column 1, that is, the first cell coordinate is [0,1], and the similarity between the first entity identifier 'attribute' and the matching text of the first cell coordinate is 1.0.

[0093] "'Attribute value':[[0,4],1.0]" means: the location information of the matching text of the second entity identifier 'Attribute value' is row 0 and column 4, that is, the second cell coordinate is [0,4], and the similarity between the second entity identifier 'Attribute value' and the matching text of the second cell coordinate is 1.0.

[0094] It should be noted that, different triple identification information is stored as different target dictionaries according to the position information and similarity of the matching texts matched by the triple identification information in the table to be extracted.

[0095] Under the above implementation, based on the multiple first similarities obtained and the multiple second similarities obtained, the matching texts and corresponding location information that match the first entity identifier and the second entity identifier respectively are determined from the text content of the table to be extracted. This process does not require manual intervention, effectively reduces labor costs, and thus improves the extraction efficiency of subsequent information extraction.

[0096] Step 202: Determine the table style of the table to be extracted based on the position information of the matching text in the table to be extracted.

[0097] In an optional implementation manner, the table styles of the table to be extracted include: a horizontal table and a vertical table.

[0098] Specifically, Figure 3 For example, based on the target dictionary map_dict obtained and the cross-table cell situation in the table to be extracted, the position of the table header and the corresponding matching text are determined. First, the length of the target dictionary map_dict is determined. If the length of the target dictionary map_dict is less than 2, it means that the triple identifier information only matches the first entity identifier or the second entity identifier in the table to be extracted, or neither matches.

[0099] When the target dictionary map_dict has a length of 2, for example in Figure 4 In the target dictionary map_dict = {'attribute': [[0,1],1.0], 'attribute value': [[0,4],1.0]}, based on the first cell coordinates corresponding to the first entity identifier and the second cell coordinates corresponding to the second entity identifier, it can be preliminarily determined that the table to be extracted is a horizontal table. Then, by analyzing the table data list, it can be found that the header does not span rows or columns. Therefore, the table style can be determined, that is, the table to be extracted is a horizontal table, where the header is row 0 and the content is rows 1-4. Furthermore, the table style is stored as a matrix, where 1 represents the header, 0 represents the corresponding content, and 2 represents other content.

[0100] In an embodiment of the present application, the table style of the table to be extracted is determined based on the position information of the matching text in the table to be extracted. This process does not require manual intervention, effectively reduces labor costs, and thus effectively improves the extraction efficiency of subsequent information extraction.

[0101] Step 203 : extracting the combined text from the table to be extracted based on the table style of the table to be extracted and the position information of the matching text in the table to be extracted.

[0102] In an optional implementation, based on the table style of the table to be extracted and the position information of the matching text in the table to be extracted, candidate text is extracted from the table to be extracted; the table title text is located from the table to be extracted; the table title text and the candidate text are combined to obtain a combined text.

[0103] Specifically, the first cell coordinates corresponding to the matching text of the first entity identifier, and the second cell coordinates corresponding to the matching text of the second entity identifier.

[0104] When the table to be extracted is a horizontal table, the column containing the first cell coordinate and the column containing the second cell coordinate are located in the table to be extracted. Then, text extraction is performed on each row of the non-header portion of the table to be extracted. Specifically, for each row of text in the non-header portion, the subtext content of the column containing the first cell coordinate and the subtext content of the column containing the second cell coordinate are extracted. The multiple extracted subtext contents are concatenated to obtain row extraction content. Based on the obtained multiple row extraction contents, candidate text is determined.

[0105] The process of extracting candidate text when the table style is a vertical table is the same as the process of extracting candidate text when the table style is a horizontal table, and will not be repeated here.

[0106] In addition to the cell contents, the table title sometimes also contains important information. Therefore, the table title text is located from the table to be extracted through table positioning; for example Figure 4 The title indicates that this table contains information about a product.

[0107] Combine the table title text and the candidate text into a combined text in the following manner: "table title text" + "\n" + "candidate text 1" + "\n" + ...; regularize the line breaks in the cells, for example, replace them with spaces or commas.

[0108] For example, from Figure 3 The combined text extracted from the table shown is: "**The product includes the following attributes"\n"Platform account: that is, the group service number generated by ordering***"\n"Application platform function: **Application service / **Value-added service / **Integrated service"\n"Whether to dispatch IoT installation and maintenance: Yes / No"\n"Device terminal model code:".

[0109] In the embodiment of the present application, not only the text content of the table to be extracted is included in the information extraction scope, but also the importance of the table title text is fully measured, and the table title text and the candidate text are combined to obtain the combined text, thereby effectively improving the accuracy of information extraction.

[0110] Step 204: Determine at least one target triple content based on the triple identification information and the combined text through the information extraction model.

[0111] In an embodiment of the present application, the matching text that matches the triple identification information and the position information of the matching text in the table to be extracted are determined from the text content of the table to be extracted, and then the table style of the table to be extracted is determined. Then, based on the table style of the table to be extracted and the position information of the matching text in the table to be extracted, the combined text is extracted from the table to be extracted. Finally, the triple identification information and the combined text are input into the information extraction model to determine at least one target triple content, thereby realizing more refined information extraction for semi-structured tables, greatly reducing labor costs, and effectively improving the extraction efficiency of table information extraction.

[0112] In an optional implementation, the triple identification information and the combined text are input into an information extraction model to obtain a plurality of preliminary triple contents; the plurality of preliminary triple contents are deduplicated and screened to determine at least one target triple content.

[0113] Specifically, the triple identification information and the combined text are input into an information extraction model, such as a Bert-based model, a UIE model, etc., to obtain multiple preliminary triple contents, and then the multiple preliminary triple contents are input into a judgment module to remove preliminary triples with a possibility probability lower than a threshold. Finally, the retained preliminary triple contents with a possibility probability higher than or equal to the threshold are deduplicated to determine at least one target triple content.

[0114] The embodiment of the present application adopts a method that combines document information extraction and paragraph information extraction. By inputting triple identification information and combined text into the information extraction model, and then deduplicating and screening the multiple preliminary triple contents obtained, at least one target triple content is determined, thereby effectively improving the information extraction efficiency.

[0115] In order to better explain the embodiment of the present application, the following describes a flow chart of a table information extraction method provided by the embodiment of the present application in combination with a specific implementation scenario. Figure 5 As shown, the process of the method is executed by the information extraction system 102, including a data preprocessing stage 501, a table style determination stage 502, a data matching stage 503 and a triple extraction stage 504.

[0116] Specifically, the data preprocessing stage 501 includes the following steps:

[0117] Step 5011: Format conversion.

[0118] Specifically, the PDF file in the document to be extracted is converted into a Word file through format conversion.

[0119] Step 5012: Table parsing.

[0120] Specifically, the table to be extracted obtained after parsing the Word file through the table is stored as a table data list.

[0121] The table style determination stage 502 specifically includes the following steps:

[0122] Step 5021: Text similarity analysis.

[0123] Specifically, the entity pair includes a first entity identifier and a second entity identifier. For the text content in each cell of the table to be extracted in the table data list, the following operations are performed respectively: the first entity identifier and the text content of a cell are input into the text similarity model to obtain a first similarity between the first entity identifier and the text content of a cell; the second entity identifier and the text content of a cell are input into the text similarity model to obtain a second similarity between the second entity identifier and the text content of a cell.

[0124] Step 5022: Cell matching.

[0125] Specifically, based on the obtained multiple first similarities and the obtained multiple second similarities, matching texts that match the first entity identifier and the second entity identifier respectively and corresponding position information are determined from the text content of the table to be extracted in the table data list.

[0126] Step 5023: Determine the table style.

[0127] Specifically, based on the matching text of the first entity identifier and the second entity identifier and the corresponding position information in the table to be extracted, the table style of the table to be extracted is determined.

[0128] The data matching stage 503 specifically includes the following steps:

[0129] Step 5031: Traverse the matching.

[0130] Specifically, candidate texts are extracted from the table to be extracted according to the table style of the table to be extracted and the position information of the matching text in the table to be extracted.

[0131] Step 5032: Table positioning.

[0132] Specifically, the table title text is located from the table to be extracted.

[0133] Step 5033: Combine text.

[0134] Specifically, the table title text and the candidate text are combined into a combined text.

[0135] The triple extraction stage 504 specifically includes the following steps:

[0136] Step 5041: Triplet extraction pre-training model.

[0137] The triples to be extracted are equivalent to entity pairs. The triples to be extracted and the combined text are input into an information extraction model, such as a BERT-based model, a UIE model, etc., to obtain triples 1 and triples 2.

[0138] Step 5042: Determine module.

[0139] Specifically, triple 1 and triple 2 are input into the judgment module, triple 1 and triple 2 are deduplicated, and triples with probability lower than the threshold are removed, and finally triple 1 is output as the target triple.

[0140] Based on the same inventive concept, see Figure 6 , an embodiment of the present application provides an information extraction device, comprising:

[0141] An information matching module 601 is used to determine, from the text content of the table to be extracted, a matching text that matches the triple identification information, and position information of the matching text in the table to be extracted;

[0142] A table style determination module 602 is configured to determine the table style of the table to be extracted based on the position information of the matching text in the table to be extracted;

[0143] The combined text extraction module 603 is used to extract the combined text from the table to be extracted based on the table style of the table to be extracted and the position information of the matching text in the table to be extracted;

[0144] The information extraction module 604 is configured to determine at least one target triple content based on the triple identification information and the combined text through an information extraction model.

[0145] In an optional embodiment, a document pre-processing module 605 is further included;

[0146] The document preprocessing module is specifically used for:

[0147] Perform format conversion and table parsing on the document to be processed to obtain the table to be extracted; store the table to be extracted in the form of a two-dimensional matrix.

[0148] In an optional implementation manner, the information matching module 601 is further configured to:

[0149] For the text content in each cell in the table to be extracted, perform the following operations:

[0150] Inputting the first entity identifier and the text content of a cell into a text similarity model to obtain a first similarity between the first entity identifier and the text content of the cell;

[0151] Inputting the second entity identifier and the text content of a cell into a text similarity model to obtain a second similarity between the second entity identifier and the text content of the cell;

[0152] Based on the obtained multiple first similarities and the obtained multiple second similarities, matching texts that respectively match the first entity identifier and the second entity identifier and corresponding position information are determined from the text content of the table to be extracted.

[0153] In an optional implementation manner, the information matching module 601 is further configured to:

[0154] Multiple first similarities, multiple second similarities, triple identification information and a similarity threshold are input into a matching function to obtain a target dictionary corresponding to the triple identification information, where the target dictionary includes: a first entity identifier, a first cell coordinate, the similarity between the first entity identifier and the corresponding matching text, a second entity identifier, a second cell coordinate, and the similarity between the second entity identifier and the corresponding matching text.

[0155] In an optional implementation, the combined text extraction module 603 is further configured to:

[0156] Extracting candidate texts from the table to be extracted based on the table style of the table to be extracted and the position information of the matching text in the table to be extracted;

[0157] Locate the table title text from the table to be extracted;

[0158] Combine the table title text and the candidate text to obtain the combined text.

[0159] In an optional implementation, the information extraction module 604 is further configured to:

[0160] The triple identification information and the combined text are input into the information extraction model to obtain a plurality of preliminary triple contents; the plurality of preliminary triple contents are deduplicated and screened to determine at least one target triple content.

[0161] In an embodiment of the present application, the matching text that matches the triple identification information and the position information of the matching text in the table to be extracted are determined from the text content of the table to be extracted, and then the table style of the table to be extracted is determined. Then, based on the table style of the table to be extracted and the position information of the matching text in the table to be extracted, the combined text is extracted from the table to be extracted. Finally, the triple identification information and the combined text are input into the information extraction model to determine at least one target triple content, thereby realizing more refined information extraction for semi-structured tables, greatly reducing labor costs, and effectively improving the extraction efficiency of table information extraction.

[0162] Based on the same technical concept, the embodiment of the present application provides a computer device, which can be Figure 1 The terminal device and / or information extraction system shown, such as Figure 7 As shown, it includes at least one processor 701 and a memory 702 connected to the at least one processor. The specific connection medium between the processor 701 and the memory 702 is not limited in the embodiment of the present application. Figure 7 For example, the processor 701 and the memory 702 are connected via a bus. The bus can be divided into an address bus, a data bus, a control bus, and the like.

[0163] In an embodiment of the present application, the memory 702 stores instructions that can be executed by at least one processor 701. The at least one processor 701 can perform the steps of the above-mentioned information extraction method by executing the instructions stored in the memory 702.

[0164] The processor 701 is the control center of the computer device. It can connect various parts of the computer device using various interfaces and lines, and extract information by running or executing instructions stored in the memory 702 and calling data stored in the memory 702. Optionally, the processor 701 may include one or more processing units. The processor 701 may integrate an application processor and a modem processor. The application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understood that the modem processor may not be integrated into the processor 701. In some embodiments, the processor 701 and the memory 702 may be implemented on the same chip. In some embodiments, they may also be implemented on separate chips.

[0165] The processor 701 can be a general-purpose processor, such as a central processing unit (CPU), a digital signal processor, an application-specific integrated circuit (ASIC), a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component, and can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present application. A general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of the present application can be directly embodied as being executed by a hardware processor, or can be executed by a combination of hardware and software modules in the processor.

[0166] The memory 702 is a non-volatile computer-readable storage medium that can be used to store non-volatile software programs, non-volatile computer executable programs and modules. The memory 702 may include at least one type of storage medium, such as a flash memory, a hard disk, a multimedia card, a card-type memory, a random access memory (Random Access Memory, RAM), a static random access memory (Static Random Access Memory, SRAM), a programmable read-only memory (Programmable Read Only Memory, PROM), a read-only memory (Read Only Memory, ROM), an electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, EEPROM), a magnetic memory, a disk, an optical disk, etc. The memory 702 is any other medium that can be used to carry or store a desired program code in the form of an instruction or data structure and can be accessed by a computer device, but is not limited thereto. The memory 702 in the embodiment of the present application can also be a circuit or any other device that can realize a storage function, for storing program instructions and / or data.

[0167] Based on the same inventive concept, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program that can be executed by a computer device. When the program runs on the computer device, the computer device executes the steps of the above-mentioned information extraction method.

[0168] Based on the same inventive concept, an embodiment of the present application provides a computer program product, which includes a computer program stored on a computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer device, the computer device executes the steps of the above-mentioned information extraction method.

[0169] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0170] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as a combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer device or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0171] These computer program instructions may also be stored in a computer readable memory that can direct a computer device or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0172] These computer program instructions can also be loaded onto a computer device or other programmable data processing device so that a series of operating steps are executed on the computer device or other programmable device to produce a process implemented by the computer device, thereby providing instructions for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0173] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0174] Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if such changes and modifications fall within the scope of the claims and their equivalents, the present invention is intended to include such changes and modifications.

Claims

1. An information extraction method, characterized in that: include: Determine, from the text content of the table to be extracted, a matching text that matches the triple identification information, and position information of the matching text in the table to be extracted; The triplet identification information is an entity pair, including: a first entity identification and a second entity identification; The location information includes: first cell coordinates corresponding to the matching text of the first entity identifier, and second cell coordinates corresponding to the matching text of the second entity identifier; Determining the table style of the table to be extracted based on the length of a target dictionary, the first cell coordinates, the second cell coordinates, and whether the table header spans rows or columns; the target dictionary includes: the first entity identifier, the first cell coordinates, the similarity between the first entity identifier and the corresponding matching text, the second entity identifier, the second cell coordinates, and the similarity between the second entity identifier and the corresponding matching text; When the table style of the table to be extracted is a horizontal table, for each row of text content in the non-header portion, extract the subtext content of the column where the first cell coordinates are located, and the subtext content of the column where the second cell coordinates are located, and concatenate the multiple extracted subtext contents to obtain row extracted content; and determine candidate text based on the obtained multiple row extracted contents; Locating a table title text from the table to be extracted; combining the table title text with the candidate text to obtain a combined text; At least one target triple content is determined based on the triple identification information and the combined text through an information extraction model.

2. The method according to claim 1, wherein The method of determining a matching text that matches the triple identification information from the text content of the table to be extracted, and the matching text being before the position information in the table to be extracted, further includes: Perform format conversion and table parsing on the document to be processed to obtain the table to be extracted; The table to be extracted is stored in a two-dimensional matrix form.

3. The method according to claim 1, wherein The step of determining, from the text content of the table to be extracted, the matching text that matches the triple identification information and the position information of the matching text in the table to be extracted includes: For the text content in each cell in the table to be extracted, perform the following operations respectively: Inputting the first entity identifier and the text content of a cell into a text similarity model to obtain a first similarity between the first entity identifier and the text content of the cell; Inputting the second entity identifier and the text content of a cell into a text similarity model to obtain a second similarity between the second entity identifier and the text content of the cell; Based on the obtained multiple first similarities and the obtained multiple second similarities, matching texts that respectively match the first entity identifier and the second entity identifier and corresponding position information are determined from the text content of the table to be extracted.

4. The method according to claim 3, wherein The step of determining, based on the obtained multiple first similarities and the obtained multiple second similarities, matching texts that respectively match the first entity identifier and the second entity identifier and corresponding position information from the text content of the table to be extracted includes: The multiple first similarities, the multiple second similarities, the triple identification information, and a similarity threshold are input into a matching function to obtain a target dictionary corresponding to the triple identification information.

5. The method according to claim 1, wherein The determining, by using an information extraction model, at least one target triple content based on the triple identification information and the combined text includes: Inputting the triple identification information and the combined text into an information extraction model to obtain a plurality of preliminary triple contents; The plurality of preliminary triple content groups are deduplicated and screened to determine at least one target triple content group.

6. The method according to any one of claims 1 to 5, characterized in that: The table styles of the table to be extracted include: horizontal table and vertical table.

7. An information extraction device, characterized in that: include: An information matching module, configured to determine, from the text content of the table to be extracted, a matching text that matches the triple identification information, and position information of the matching text in the table to be extracted; The triplet identification information is an entity pair, including: a first entity identification and a second entity identification; The location information includes: first cell coordinates corresponding to the matching text of the first entity identifier, and second cell coordinates corresponding to the matching text of the second entity identifier; a table style determination module, configured to determine the table style of the table to be extracted based on the length of a target dictionary, the first cell coordinates, the second cell coordinates, and whether the table header spans rows or columns; the target dictionary comprising: the first entity identifier, the first cell coordinates, the similarity between the first entity identifier and the corresponding matching text, the second entity identifier, the second cell coordinates, and the similarity between the second entity identifier and the corresponding matching text; a combined text extraction module configured to, when the table to be extracted is a horizontal table, extract, for each row of text content in the non-header portion, subtext content of the column where the first cell coordinates are located, and subtext content of the column where the second cell coordinates are located, and concatenate the multiple extracted subtext contents to obtain row extracted content; and determine candidate text based on the obtained multiple row extracted contents; Locating a table title text from the table to be extracted; combining the table title text with the candidate text to obtain a combined text; The information extraction module is used to determine at least one target triple content based on the triple identification information and the combined text through an information extraction model.

8. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium, characterized in that It stores a computer program that can be executed by a computer device. When the program is run on the computer device, the computer device executes the steps of any one of the methods according to claims 1 to 6.

10. A computer program product, characterized in that The computer program product comprises a computer program stored on a computer-readable storage medium, wherein the computer program comprises program instructions. When the program instructions are executed by a computer device, the computer device is caused to perform the steps of the method according to any one of claims 1 to 6.

Images