A method and system for entity relation extraction from open-domain text
By expanding the training data and using a word frequency filtering strategy, the problem of candidate tail entity noise in open-domain text was solved, achieving efficient entity relation extraction and knowledge graph construction, and improving the performance of the model in practical applications.
Patent Information
- Application Number
- CN202111268377.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-10-29
- Publication Date
- 2025-11-14
- Estimated Expiration
- 2041-10-29
AI Technical Summary
Existing methods for entity relation extraction in open-domain text suffer from candidate tail entity noise, leading to poor performance in practical applications and increasing model complexity or manual annotation costs.
By expanding the training data, positive and negative examples are generated using regular expressions. Combined with word frequency filtering strategies, a neural network model is trained to remove high-frequency noise words and construct a knowledge graph.
Without increasing model complexity or manual annotation costs, it significantly improves the accuracy and efficiency of entity relationship identification and constructs a high-quality knowledge graph.
Smart Images
Figure CN114021566B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of natural language processing, and particularly relates to a method and system for entity relation extraction from open-domain text. Background Art
[0002] A knowledge graph is a large-scale semantic network that stores human knowledge in the form of a graph (Graph). Nodes (Nodes) in the knowledge graph represent entities (Entities), and edges (Edges) represent relationships (Relations). In a knowledge graph, facts (Facts) are generally represented in the form of triples (Triples). A triple is generally denoted as <head entity, relation, tail entity>. For example, <XXX, place of birth, Hawaii> is a triple, where the head entity is "XXX", the tail entity is "Hawaii", and the relation type is "place of birth".
[0003] Currently, the methods for entity relation extraction are mainly divided into the pipeline method and the joint learning method. The steps of the pipeline method are as follows: First, entity extraction is performed on the input sentence to obtain candidate tail entities, then the identified tail entities are combined with the head entity, and finally entity relation classification is performed. The joint learning method is to use a single model to simultaneously implement the two tasks of entity extraction and relation classification. The joint learning method requires the design of specific marking strategies to simultaneously solve the two sub-tasks, has high requirements for labeled data, and requires a large amount of expensive human costs.
[0004] The process of entity relation extraction from open-domain text mainly consists of two steps. The input is the text and a pre-defined head entity. In the first step, all entities and noun phrases are extracted from the text as candidate tail entities; in the second step, given the text, the head entity, and the candidate tail entities extracted in the first step, a classification model such as a neural network is used to determine the relationship between the head entity and each candidate tail entity, and the relationship categories are pre-defined.
[0005] When the training set is relatively simple, the above entity relationship classification method can achieve an F1 score of 0.95 on the training set. However, in real-world data, a significant amount of noise will be present. This is essentially due to the difference between the construction of tail entities in the real-world data and the construction of the training set. In the training data, tail entities are pre-labeled, and each tail entity has its corresponding relationship type (i.e., head entities and tail entities must have a relationship, and a large portion of the given candidate tail entities do not have a relationship). However, in real-world data, to ensure the coverage of candidate tail entities, named entities and noun phrases are extracted, which introduces a lot of noise into the candidate tail entities. For example, "XXX was born in country A, grew up in place B, but spent one year in place C and four years in country D during his childhood." After entity extraction and part-of-speech tagging, the resulting set of tail entities is {country A, place B, childhood, place C, country D, one year, four years}. Only a small portion of the candidate tail entities can form relationships with the given head entities. Because the training set is relatively simple, most training data only have one tail entity. However, in actual data, there may be multiple tail entities. For example, in the above example, multiple locations appear in a text, but the model may identify the relationship between the head entity and the tail entity representing the date as the birthplace when classifying.
[0006] The above problems can be solved at the model level by adding contextual information to create new models, but this will increase costs significantly, and the complexity of the model will also present certain problems in practical applications.
[0007] To address the problem of multiple entities representing locations appearing in a single text, more complex entity relationship types can be defined, allowing the entity relationship classification model to learn more information and improve classification performance. However, the real world contains a vast number of relationship types. Increasing the number of entity types places higher demands on annotators and increases annotation costs. Summary of the Invention
[0008] The purpose of this invention is to overcome the noise problem of candidate tail entities in the prior art. Without increasing the complexity of data annotation and model, it can effectively solve the problem of poor performance of entity relationship recognition in practical applications caused by noise from candidate tail entities without increasing the complexity of data annotation and model.
[0009] To address the shortcomings of existing technologies, this invention proposes a method for extracting entity relations from open-domain text, including:
[0010] Step 1: Obtain the text with labeled head entities, relations and tail entities as the original training data. Use entity recognition to extract all named entities and noun phrases in the original training data as a candidate set. Remove the named entities and noun phrases in the candidate tail entity set that are the same as the labeled tail entities to obtain a selected set. Label the text with all named entities and noun phrases in the selected set as tail entities and label their relations with the labeled head entities as unknown to obtain the expanded training data.
[0011] Step 2: Using the text, head entity, and tail entity in the expanded training data as input, and the relationship between the head entity and tail entity in the expanded training data as the training objective, train the neural network model to obtain the entity relationship classification model;
[0012] Step 3: Calculate the word frequency of each named entity and noun phrase in the expanded training data, and mark the named entities and noun phrases with a word frequency greater than the preset value as filtered words;
[0013] Step 4: Obtain the open-domain text to be classified and its corresponding head entity. Extract all named entities and noun phrases from the open-domain text except for the filtered words, and use them as the tail entities of the open-domain text. Input the open-domain text and its corresponding head and tail entities into the entity classification model to obtain the entity relationships of the open-domain text.
[0014] The open-domain text entity relation extraction method, wherein step 1 includes: replacing the labeled tail entities in the original training data with regular expressions to form multiple new entities, relations and tail entities as positive examples, and labeling the positive examples to the expanded training data.
[0015] The open-domain text entity relation extraction method, wherein step 1 includes: using regular expressions to replace the labeled relations in the original training data with new relations, forming multiple new entities, relations and tail entities as negative examples, and the new relations do not belong to any relation type in the current original training data, and labeling the negative examples to the expanded training data.
[0016] The method for extracting entity relations from open-domain text, wherein step 4 includes: constructing a knowledge base based on the head entity, entity relations and tail entity of the open-domain text, and establishing a knowledge graph of the open-domain text based on the knowledge base.
[0017] This invention also proposes an entity relation extraction system for open-domain text, including:
[0018] Module 1 is used to obtain the text with labeled head entities, relations and tail entities as the original training data. Entity recognition is used to extract all named entities and noun phrases in the original training data as a candidate set. Named entities and noun phrases that are the same as the labeled tail entities in the candidate tail entity set are removed to obtain a selected set. All named entities and noun phrases in the selected set are labeled as tail entities in the text, and their relations with the labeled head entities are labeled as unknown to obtain the expanded training data.
[0019] Module 2 is used to train a neural network model by taking the text, head entity, and tail entity in the expanded training data as input, and the relationship between the head entity and tail entity in the expanded training data as the training objective, and to obtain an entity relationship classification model.
[0020] Module 3 is used to count the word frequency of each named entity and noun phrase in the expanded training data, and to mark named entities and noun phrases with a word frequency greater than a preset value as filtered words;
[0021] Module 4 is used to obtain the open-domain text to be classified as an entity relationship and its corresponding head entity, extract all named entities and noun phrases in the open-domain text except for the filtered words, as the tail entity of the open-domain text, and input the open-domain text and its corresponding head and tail entities into the entity relationship classification model to obtain the entity relationship of the open-domain text.
[0022] The open-domain text entity relation extraction system, wherein module 1 is used to replace the labeled tail entities in the original training data with regular expressions to form multiple new entities, relations and tail entities as positive examples, and to label the positive examples to the expanded training data.
[0023] The open-domain text entity relation extraction system, wherein module 1 is used to replace the labeled relations in the original training data with new relations using regular expressions, forming multiple new entities, relations and tail entities as negative examples, and the new relations do not belong to any relation type in the current original training data, and the negative examples are labeled to the expanded training data.
[0024] The entity relation extraction system for open domain text, wherein module 4 includes: constructing a knowledge base based on the head entity, entity relation and tail entity of the open domain text, and establishing a knowledge graph of the open domain text based on the knowledge base.
[0025] The present invention also proposes a storage medium for a program of any of the above-described open-domain text entity relation extraction methods.
[0026] The present invention also proposes a client for entity relation extraction systems of any of the above-mentioned open-domain texts.
[0027] As can be seen from the above solutions, the advantages of the present invention are:
[0028] This invention can significantly improve the performance of large-scale open-domain entity relation extraction tasks with relatively simple training data, without increasing model complexity, and without incurring expensive manual annotation costs.
[0029] This invention can construct a structured knowledge base from unstructured open-domain text, and then build a knowledge graph. Specifically, a knowledge graph is graph-structured text, which also includes triples stored in a database. The narrow definition of a knowledge graph is the formal representation of triple relationships using a graph. The structured knowledge extracted by this invention can assist in applications across multiple scenarios such as search recommendation, advertising systems, question-answering systems, and financial risk control. For example, in recommendation systems, knowledge graph technology can assist in personalized recommendations based on the relationships between entities; in intelligent customer service scenarios, the knowledge base can act as the "brain," making the semantic information of questions and answers more complete and easier to understand; in the field of internet finance, knowledge graph technology can perform information inconsistency checks to determine the potential risk of borrower fraud. Attached Figure Description
[0030] Figure 1 Here is a diagram of the BERT-Attention model structure;
[0031] Figure 2 This is a diagram showing the word frequency distribution. Detailed Implementation
[0032] While conducting research on entity relationship classification, the inventors discovered that the shortcomings of existing technologies are caused by noise from candidate tail entities. Analysis of real data revealed that this deficiency can be addressed through effective data augmentation strategies. This approach can solve the problem of poor performance in practical applications due to noise from candidate tail entities in a low-cost and effective manner, without altering the model itself or incurring additional costs.
[0033] To make the above features and effects of the present invention clearer and easier to understand, specific embodiments are described below, and detailed descriptions are provided in conjunction with the accompanying drawings.
[0034] The first step is to expand the training data. As mentioned above, the entity relationship classification method performs poorly on real data mainly because the training data is too simplistic. Therefore, correcting the training data is key to solving this problem. This invention proposes two methods: expanding the training data by reducing the differences in data distribution and increasing the number of positive and negative examples.
[0035] (1) Expanding training data by reducing the differences in data distribution:
[0036] To ensure the coverage of candidate tail entities, named entities and noun phrases were extracted as candidate tail entities during the entity extraction stage using entity recognition and part-of-speech tagging. This resulted in a significant gap between the data distribution of the extracted candidate tail entities and that of the candidate tail entities in the training set.
[0037] The training set instance “XXX, born August 4, 1961 in Hawaii, USA, works in the AA, eight-year term.” has the candidate tail entity {August 4, 1961, Hawaii, USA}. If this data is real data, the extracted candidate tail entity should be {August 4, 1961, Hawaii, USA, AA, eight years}.
[0038] To narrow the gap between the training data and the real data, the training data was modified. First, all named entities and noun phrases were extracted from the training data to expand its candidate tail entities. Second, a new entity relation type "unknown" was added, and the relations between other candidate tail entities and head entities, except for the gold label, were marked as the new relation type "unknown".
[0039] (2) Expand the training data by adding positive and negative examples.
[0040] The example "XXX, born on August 4, 1961, graduated from Harvard Law School with honors in 1991, and has been teaching in Chicago for 12 years since 1992" contains many dates. We want the entity relationship classification model to learn not these specific dates, but rather the relationship type "born" as a whole. To enhance the "born" relationship type information, we can make simple modifications to the training set. First, we enhance the positive examples: for "XXX, born on August 4, 1961," we replace the dates after "born" with a regular expression, keeping the relationship type unchanged. Second, without affecting other relationship types representing time (such as death date), we add some negative examples. The dates remain the same, but "born" is changed to other relationships, such as "XXX graduated on August 4, 1961." These negative examples are then categorized as the new relationship type "unknown" (ensuring that "graduated from" does not belong to any of the given relationship types). By modifying the dataset using these two methods, we can effectively control the model's learning focus during training to prioritize the "born" relationship type, rather than learning specific date information.
[0041] The second step is to train the entity relationship classification model. After obtaining the corrected training data, we use the BERT-Attention model, an entity relationship classification model based on BERT (an internationally advanced text representation method that represents the input text as a low-dimensional dense vector). The BERT-Attention structure is as follows: Figure 1 As shown.
[0042] The model's input consists of text, head entities, and tail entities. The input is first encoded by a BERT layer to obtain hidden feature representations of the input text, head entities, and tail entities. The head and tail entities are concatenated in the hidden layer, and then passed through an attention layer to calculate the attention mechanism weights of the concatenated vector relative to the input text. Next, the hidden feature representations of the input text are weighted; this step also emphasizes the guiding role of the head and tail entities in the classification model. Finally, the weighted text representation is concatenated with the head and tail entity representations in the hidden layer, and then passed through a fully connected layer before being input into the classifier to obtain the relation classification result.
[0043] The third step is a tail-tail entity filtering method based on word frequency statistics. When extracting named entities and noun phrases, many words are not actually needed, introducing noise data, especially those words that appear frequently in the entire corpus. Therefore, we extract named entities and noun phrases from all documents in the training set, count their word frequencies, and obtain a word frequency distribution table for the entire dataset. The word frequency follows a long-tail distribution (e.g., ...). Figure 2 As shown in the figure, the more frequently a word appears, the higher the probability that it is a meaningless noise word. Figure 2 The horizontal axis represents each word, and the vertical axis represents the frequency of each word. Words at the top of the horizontal axis are high-frequency, meaningless stop words, such as: is, are, to, from, of… These words are few in number but appear frequently. Words further down the horizontal axis are low-frequency words, such as initiative, philosophy… These words have small vertical axes but a very large number of low-frequency words. Therefore, the statistical distribution of words in the text exhibits a long-tail distribution trend. Thus, we can set a word frequency threshold to filter out words with frequencies higher than this threshold. Because the word frequency distribution is long-tailed, the vocabulary of high-frequency words will not be too large, ensuring feasibility. In practice, given a newly extracted candidate tail entity, if it is a high-frequency word, it is removed from the candidate tail entity list; otherwise, it is retained.
[0044] The fourth step is entity relation classification. Following the three steps above, entity relations can be extracted from open-domain text data. For a given text and head entity, i) first, all named entities and noun phrases in the text are extracted; ii) the filtering strategy from the third step is used to obtain filtered candidate tail entities; iii) the entity relation classification model trained in the second step is used for classification.
[0045] The following are system embodiments corresponding to the above method embodiments. This embodiment can be implemented in conjunction with the above embodiments. The relevant technical details mentioned in the above embodiments are still valid in this embodiment, and will not be repeated here to reduce repetition. Accordingly, the relevant technical details mentioned in this embodiment can also be applied to the above embodiments.
[0046] This invention also proposes an entity relation extraction system for open-domain text, including:
[0047] Module 1 is used to obtain the text with labeled head entities, relations and tail entities as the original training data. Entity recognition is used to extract all named entities and noun phrases in the original training data as a candidate set. Named entities and noun phrases that are the same as the labeled tail entities in the candidate tail entity set are removed to obtain a selected set. All named entities and noun phrases in the selected set are labeled as tail entities in the text, and their relations with the labeled head entities are labeled as unknown to obtain the expanded training data.
[0048] Module 2 is used to train a neural network model by taking the text, head entity, and tail entity in the expanded training data as input, and the relationship between the head entity and tail entity in the expanded training data as the training objective, and to obtain an entity relationship classification model.
[0049] Module 3 is used to count the word frequency of each named entity and noun phrase in the expanded training data, and to mark named entities and noun phrases with a word frequency greater than a preset value as filtered words;
[0050] Module 4 is used to obtain the open-domain text to be classified as an entity relationship and its corresponding head entity, extract all named entities and noun phrases in the open-domain text except for the filtered words, as the tail entity of the open-domain text, and input the open-domain text and its corresponding head and tail entities into the entity relationship classification model to obtain the entity relationship of the open-domain text.
[0051] The open-domain text entity relation extraction system, wherein module 1 is used to replace the labeled tail entities in the original training data with regular expressions to form multiple new entities, relations and tail entities as positive examples, and to label the positive examples to the expanded training data.
[0052] The open-domain text entity relation extraction system, wherein module 1 is used to replace the labeled relations in the original training data with new relations using regular expressions, forming multiple new entities, relations and tail entities as negative examples, and the new relations do not belong to any relation type in the current original training data, and the negative examples are labeled to the expanded training data.
[0053] The entity relation extraction system for open domain text, wherein module 4 is used to construct a knowledge base based on the head entity, entity relation and tail entity of the open domain text, and to establish a knowledge graph of the open domain text based on the knowledge base.
[0054] The present invention also proposes a storage medium for a program of any of the above-described open-domain text entity relation extraction methods.
[0055] The present invention also proposes a client for entity relation extraction systems of any of the above-mentioned open-domain texts.
Claims
1. A method for extracting entity relations from open-domain text, characterized in that, include: Step 1: Obtain the text with labeled head entities, relations and tail entities as the original training data. Use entity recognition to extract all named entities and noun phrases in the original training data as a candidate set. Remove the named entities and noun phrases in the candidate tail entity set that are the same as the labeled tail entities to obtain a selected set. Use all named entities and noun phrases in the selected set as tail entities to expand the annotation of the text, and label the relationship between the text and the labeled head entities as unknown to obtain the expanded training data. Step 2: Using the text, head entity, and tail entity in the expanded training data as input, and the relationship between the head entity and tail entity in the expanded training data as the training objective, train the neural network model to obtain the entity relationship classification model; Step 3: Calculate the word frequency of each named entity and noun phrase in the expanded training data, and mark the named entities and noun phrases with a word frequency greater than the preset value as filtered words; Step 4: Obtain the open-domain text to be classified and its corresponding head entity. Extract all named entities and noun phrases from the open-domain text except for the filtered words, and use them as the tail entities of the open-domain text. Input the open-domain text and its corresponding head and tail entities into the entity classification model to obtain the entity relationships of the open-domain text.
2. The entity relation extraction method for open-domain text as described in claim 1, characterized in that, Step 1 includes: replacing the labeled tail entities in the original training data with regular expressions to form multiple new entities, relations and tail entities as positive examples, and labeling the positive examples in the augmented training data.
3. The entity relation extraction method for open-domain text as described in claim 1 or 2, characterized in that, Step 1 includes: using regular expressions to replace the labeled relations in the original training data with new relations, forming multiple new entities, relations and tail entities as negative examples, and the new relations do not belong to any relation type in the current original training data, and labeling the negative examples to the expanded training data.
4. The entity relation extraction method for open-domain text as described in claim 1, characterized in that, Step 4 includes: constructing a knowledge base based on the head entity, entity relationship and tail entity of the open domain text, and building a knowledge graph of the open domain text based on the knowledge base.
5. A system for extracting entity relations from open-domain text, characterized in that, include: Module 1 is used to obtain the text with labeled head entities, relations and tail entities as the original training data. Entity recognition is used to extract all named entities and noun phrases in the original training data as a candidate set. Named entities and noun phrases that are the same as the labeled tail entities in the candidate tail entity set are removed to obtain a selected set. All named entities and noun phrases in the selected set are labeled as tail entities in the text, and their relations with the labeled head entities are labeled as unknown to obtain the expanded training data. Module 2 is used to train a neural network model by taking the text, head entity, and tail entity in the expanded training data as input, and the relationship between the head entity and tail entity in the expanded training data as the training objective, and to obtain an entity relationship classification model. Module 3 is used to count the word frequency of each named entity and noun phrase in the expanded training data, and to mark named entities and noun phrases with a word frequency greater than a preset value as filtered words; Module 4: Obtain the open-domain text to be classified as an entity relation and its corresponding head entity. Extract all named entities and noun phrases from the open-domain text except for the filtered words, and use them as the tail entities of the open-domain text. Input the open-domain text and its corresponding head and tail entities into the entity relation classification model to obtain the entity relations of the open-domain text.
6. The entity relation extraction system for open-domain text as described in claim 5, characterized in that, Module 1 is used to replace the labeled tail entities in the original training data with regular expressions to form multiple new entities, relations and tail entities as positive examples, and to label the positive examples in the augmented training data.
7. The entity relation extraction system for open-domain text as described in claim 5 or 6, characterized in that, Module 1 is used to replace the labeled relations in the original training data with new relations using regular expressions, forming multiple new entities, relations and tail entities as negative examples. The new relations do not belong to any relation type in the current original training data, and the negative examples are labeled in the expanded training data.
8. The entity relation extraction system for open-domain text as described in claim 5, characterized in that, Module 4 is used to construct a knowledge base based on the head entities, entity relationships, and tail entities of the open domain text, and to build a knowledge graph of the open domain text based on the knowledge base.
9. A storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the entity relation extraction method for open-domain text as described in any one of claims 1-4.
10. A client for an entity relation extraction system for open-domain text as described in any one of claims 5 to 8.
Citation Information
Patent Citations
Open-type entity and type identification method thereof
CN103034693A
Syntax mode and machine learning-based open relationship extraction method
CN111027324A