Open domain relation extraction method and device based on hierarchical entity ranking and generation of text comparison
By using a method based on hierarchical entity sorting and generated text comparison, spurious correlation clues in open domain relation extraction are eliminated. Causal intervention and clustering techniques are used to solve the problem of model instability, and efficient open domain relation extraction is achieved under unlabeled data.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- INST OF SOFTWARE - CHINESE ACAD OF SCI
- Filing Date
- 2021-06-04
- Publication Date
- 2026-05-26
AI Technical Summary
Existing open-domain relation extraction methods are susceptible to spurious cues in the absence of clear supervision signals, leading to model instability and an inability to effectively transfer to new relations and domains.
We employ a relation representation learning method based on hierarchical entity ranking and generated text comparison. By constructing a hierarchical entity tree and knowledge base, we remove spurious relevance cues using causal intervention, train the encoder by combining ranking loss and contrastive loss functions, and perform clustering to obtain stable relation representations.
It enables open-domain relation extraction on unlabeled data, improving the model's stability and generalization ability, and enhancing the accuracy of relation extraction.
Smart Images
Figure CN115438177B_ABST
Abstract
Description
Technical Field
[0001] This invention designs an open-domain relation representation method, and in particular relates to an open-domain relation extraction method and apparatus based on hierarchical entity sorting and generated text comparison, belonging to the field of natural language processing technology. Background Technology
[0002] Relation extraction is a key task in information extraction and natural language processing. The goal of this task is to identify the relation category expressed by a pair of entities within a given context. For example, given the context "A was elected president of country B," and the entity pair "A" and "country B," a relation classification system should be able to correctly identify the relation between the entities that expresses "is the president of...".
[0003] In recent years, with the development of deep learning methods, the accuracy of relation classification systems has improved significantly. However, these methods require a large amount of labeled data, resulting in excessive labeling costs for training the model. Furthermore, models obtained using these methods have poor generalization ability and cannot be transferred to new relations and domains.
[0004] Therefore, open-domain relation extraction has received widespread attention in recent years. This task does not predefine the relation categories to be extracted, but only provides a large number of unlabeled sentences, each containing entity pairs to be classified. The model needs to cluster these unlabeled sentences based on the features they exhibit, with sentences in the same cluster considered to have the same relation. Ultimately, this achieves the goal of open-domain relation extraction.
[0005] Traditional open-domain relation extraction methods are mainly divided into two categories: generative methods and discriminative methods. Generative methods treat open-domain relation extraction as a topic modeling task and utilize manually extracted features to generate potential relation categories. Discriminative methods utilize heuristic supervision signals for model training and updates.
[0006] However, since open-domain relation extraction tasks lack clear supervision signals, the above methods are easily affected by spurious correlation cues in the dataset, resulting in a lack of model stability. Summary of the Invention
[0007] To overcome the instability of existing open-domain relation extraction methods, this invention proposes an open-domain relation extraction method and apparatus based on hierarchical entity ranking and generated text comparison. By utilizing a causal intervention approach, a relation representation learning method based on hierarchical entity ranking and generated text comparison is proposed, thereby avoiding the problem of model instability.
[0008] To achieve the above objectives, the specific technical solution adopted by the present invention is as follows:
[0009] An open-domain relation extraction method based on hierarchical entity ranking and generated text comparison includes the following steps:
[0010] 1) Input each instance in the unlabeled dataset into the trained encoder to obtain the relational representation of that instance;
[0011] 2) Clustering is performed based on the aforementioned relation representation to obtain the relation extraction results;
[0012] The trained encoder is obtained through the following steps:
[0013] a) Extract an entity from any sample instance in the sample dataset, and select several knowledge entities in the knowledge base to build a hierarchical entity tree. In the hierarchical entity tree, higher-level entities have a more similar metatype to the entity, while lower-level entities have a greater difference in metatype from the entity.
[0014] b) Select a knowledge entity at each level of the hierarchical entity tree, replace the selected knowledge entity with the sample instance, generate a semantic relationship order based on the hierarchical level of the replaced entity in the hierarchical tree, and perform sorting learning based on the semantic relationship order and the relationship instances in the sample instance, and calculate the sorting loss function.
[0015] c) Based on the relation instance in the sample instance, obtain the entity pair, and combine it with the alias of the relation instance extracted from the knowledge base, the knowledge relation instances that are different from the relation instance and the entity pair, and the knowledge entity to form the original triplet, alias triplet, extended triplet and other triplets.
[0016] d) Classify the alias triples and triples with the same relation instances in the extended triples as positive examples, and the other triples and triples with different relation instances in the extended triples as negative examples. Compare the positive examples and negative examples with the original triples and calculate the contrast loss function.
[0017] e) Use the ranking loss function and the contrastive loss function to train the pre-trained encoder until a well-trained encoder is obtained.
[0018] Furthermore, the knowledge base includes: the WikiData knowledge base.
[0019] Furthermore, the knowledge entities in the hierarchical entity tree include: entities with the same meta-type as the entity as sibling entities, entities with the same second-level meta-type as the entity as neighbor entities, and other entities.
[0020] Furthermore, the original triple consists of: entity pairs and relation instances.
[0021] Furthermore, alias triples include: entity pairs and relation instance aliases.
[0022] Further, the extended triples are obtained through the following steps:
[0023] 1) Obtain the head entity, relation instance, and tail entity of the original triple;
[0024] 2) Extract knowledge entities that are the same as the head entity, knowledge relation instances that are different from the relation instance, and knowledge entities that are different from the tail entity from the knowledge base, or knowledge entities that are different from the head entity, knowledge relation instances that are different from the relation instance, and knowledge entities that are the same as the tail entity, to form triples to be expanded.
[0025] 3) Concatenate the triplet to be expanded with the original triplet to obtain the expanded triplet.
[0026] Furthermore, other triples include: entity pairs and knowledge relation instances that are distinct from relation instances.
[0027] Furthermore, clustering methods include the K-means algorithm.
[0028] A storage medium storing a computer program, wherein the computer program is configured to execute the method described above when run.
[0029] An electronic device includes a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer to perform the methods described above.
[0030] Compared with the prior art, the positive effects of the present invention are as follows:
[0031] 1) Based on the structural causal model, it was found that the dataset is a confusion factor. Therefore, a relational representation learning method based on hierarchical entity ranking and generated text comparison was proposed by using causal intervention.
[0032] 2) Based on the learned relation representations, clustering is performed on unlabeled relation instances to complete open-domain relation extraction, thereby avoiding the problem of model instability. Attached Figure Description
[0033] Figure 1 This is a structural causal model diagram for open-domain relation extraction.
[0034] Figure 2 A framework diagram for learning methods of hierarchical entity sorting and generating text contrastive representations. Detailed Implementation
[0035] To enable those skilled in the art to better understand the technical solutions in the embodiments of the present invention, and to make the objectives, features, and advantages of the present invention more apparent and understandable, the core technologies of the present invention will be further described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
[0036] The open-domain relation extraction method of the present invention includes the following key steps:
[0037] Key technologies
[0038] 1) The open-domain relation extraction process is modeled based on the structural causal model. The structural causal model is used to model the causal relationship between the dataset, entity pairs, context, relation representation and clustering results in the open-domain relation extraction process, and the confusion caused by the dataset is found.
[0039] 2) Based on the findings of the above structural causal model, we propose a relation representation learning method based on hierarchical entity ranking and generated text comparison to perform relation representation learning in open domain scenarios.
[0040] 3) Based on the relation representations obtained above, we cluster these relation representations to obtain the relation extraction results under the open domain.
[0041] This invention relates to the following key elements:
[0042] 1. Modeling the Open-Domain Relation Extraction Process Based on Structural Causal Model
[0043] In open-domain relation extraction tasks, most methods rely on large-scale unlabeled relation instances, using manual or heuristic assumptions to extract features or model supervised signals from entity pairs and context within the instances. However, this process is often accompanied by a large number of spurious correlation clues, i.e., there are spurious correlations between entity pairs or context and the final relation type, causing the model to collapse to a specific relation or make uniform predictions, resulting in model instability.
[0044] To identify the reasons for model instability, we modeled the open-domain relation extraction process using a structural causal model. (See attached...) Figure 1 As shown, open-domain relation extraction starts with an unlabeled dataset, using existing entity pairs or context for feature extraction or supervised signal modeling to learn relation representations. Finally, clustering is used to obtain potential relation categories. Through structural causal models, we can see that the dataset is a confounding factor; that is, the dataset creates backdoor paths between entity pairs or context and the final relation type. This can lead to the model potentially generating incorrect predictions by relying on spurious correlations within these backdoor paths.
[0045] 2. A relational representation learning method based on hierarchical entity ranking and generated text comparison
[0046] To eliminate the contamination introduced by the dataset during model training, we employ a causal intervention (do operator) approach, intervening in entity pairs and contextual backdoor paths. The probabilistic meaning of the do operator is to set the conditional probability of the intervened variable to 1, i.e., fixing the variable to a fixed value, and then studying its impact on the final result. (See attached diagram) Figure 1 In the code, do(e) replaces different contexts while keeping the entity pair unchanged, and do(c) replaces different entity pairs while keeping the context unchanged.
[0047] 2.1 Learning Hierarchical Entity Ranking
[0048] To control the impact of contextual backdoor paths, we need to intervene in the context and replace all possible entity pairs during training. However, in open-domain scenarios, we cannot determine which entities are feasible and which are not. Therefore, we leverage metadata about entities in a knowledge base to construct hierarchical relationships between entities and use this as a supervisory signal for representation learning. (See attached) Figure 2 As shown, for any instance in the unlabeled dataset, we construct a hierarchical tree by selecting any entity. Entities with the same meta-type are chosen as sibling entities, entities with the same second-level meta-type are chosen as neighbor entities, and other entities are considered negative examples. We then sample from the entity tree, taking one entity from each level and replacing it with its original context. The semantics of the instances replaced by sibling entities are considered the closest, followed by the semantics of the instances replaced by neighbor entities, with the other entities being the least close. This order is then used to rank and learn the relational relationships with the original relational instances, serving as a supervision signal for training the relational representation. The specific optimization objective is:
[0049]
[0050] Where θ represents the model parameters to be learned, and D(P, X) i () is a distance function that measures the generated instance X. i The distance between the original instance and the set of instances generated by X, m E is the edge distance of the loss function, and n=3 is the depth of the constructed hierarchical tree.
[0051] 2.2 Generated Text Comparison Learning
[0052] Specifically, for entity pair intervention, we need to fix the entities and select different contexts containing the same relationship for training. However, in real-world scenarios, it's impossible to know whether the contexts represent the same semantic relationship, and we cannot control the changes in the context. Therefore, we adopt a generative method to directly generate the text with controllable semantic relationships that we need. (See attached image) Figure 2As shown, for each relation instance in the dataset, we fix its entity pair and sample a relation triple containing that entity pair from the knowledge base as the original triple. We then sample the alias of the relation using metadata from the knowledge base to construct a triple with the same semantic relation but a different representation, which is the alias triple. In addition, we extract a triple from the knowledge base that has the same head entity (tail entity) but a different relation and tail entity (head entity), and concatenate it with the original triple as an extended triple. We also extract a different relation from the knowledge base and combine it with the original entity pair to form other triples.
[0053] The sampled triples are converted into strings using a fixed template. Finally, a pre-trained generative language model is used to generate the corresponding natural language text from these triples. After text generation, these texts are divided into positive and negative classes: alias triples and extended triples with the same relation are considered positive examples, while other triples and extended triples with different relations are considered negative examples. These positive and negative samples are compared with the original triples to train the model.
[0054] The specific forms of the ranking loss function and the contrastive loss function can be different, as long as they are functions that can calculate the loss by comparing the size of two values.
[0055] 3. Cluster-based open-domain relation extraction
[0056] Based on the relation representations obtained from the ranking and contrastive learning methods described above, we use the common clustering algorithm K-means to cluster instances in the unlabeled dataset and use the final clustering results as relation extraction results, where instances belonging to the same cluster have the same relation type.
[0057] Example:
[0058] Open-domain relation extraction tasks typically involve a large-scale, unlabeled relation dataset, where each instance is a sentence containing a pair of entities. Our method requires an additional knowledge base for relation hierarchy construction and comparative text construction. Specifically, we utilize the WikiData knowledge base.
[0059] Unlabeled datasets:
[0060] This dataset contains a large number of unlabeled relation instances, each containing a pair of given entities, for example:
[0061] 1. <e1> Wang Meng< / e1> yes <e2> Zhang San< / e2> "wife";
[0062] 2. <e1> Li Hua< / e1> married <e2> Zhao Si< / e2> ".
[0063] 3. <e1> Zhang San< / e1> yes <e2> Li Si< / e2> "son";
[0064] 4. <e1> Zhao Si< / e1> Daughter's name <e2> Wang Meng< / e2> "
[0065] 5. <e1> Zhang Kai< / e1> yes <e2> Zhang Jun< / e2> "older brother";
[0066] 6. "Zhang San is..." <e1> Zhang Jun< / e1> and <e2> Zhang Kai< / e2> "Parents".
[0067] The above six data points contain three types of relationships: the first two are spousal relationships, the middle two are parent-child relationships, and the last two are sibling relationships. Open-domain relationship extraction methods need to extract features or supervision signals from these unlabeled instances for model training, thereby finding relationship representations that can distinguish each category.
[0068] Implementation:
[0069] (i) For each relation instance in the unlabeled dataset, perform hierarchical entity sorting and comparative text generation, and use these as supervision signals for model training.
[0070] (II) Based on the hierarchical entity ranking learning method, we first sample any entity in the instance, link it to the knowledge base, and construct an entity hierarchy tree. We then sample entities from different levels and replace them in the instance to perform ranking learning. We train the model parameters.
[0071] (III) Based on the comparative learning method of generated text, we use entity pairs in the instance to link to the knowledge base and extract comparison relation triples. These extracted triples are then compared with the trained generative language model to generate corresponding natural language texts containing different relation triples. These generated texts are then compared to train the model parameters.
[0072] Based on the relation representations obtained from the training described above, we cluster the instances in the unlabeled dataset, and the clustering results are the open-domain relation extraction results. For example, in the dataset above, instances 1 and 2 are clustered into one class, 3 and 4 into another, and 5 and 6 into yet another.
[0073] Experimental data
[0074] Comparative tests were conducted using the T-Rex SPO dataset and the T-Rex DS dataset, respectively. The test results are shown in Table 1. It was found that the B-model of this invention... 3 The F1, V-measure F1, and ARI metrics are far superior to existing technologies.
[0075]
[0076] Table 1
[0077] The embodiments described above are merely illustrative of implementation methods of the present invention, and while the descriptions are detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the appended claims.
Claims
1. An open-domain relation extraction method based on hierarchical entity ranking and generated text comparison, comprising the following steps: 1) Input each instance in the unlabeled dataset into the trained encoder to obtain the relational representation of that instance; 2) Clustering is performed based on the aforementioned relation representation to obtain the relation extraction results; The trained encoder is obtained through the following steps: a) Extract an entity from any sample instance in the sample dataset, and select several knowledge entities in the knowledge base to build a hierarchical entity tree. In the hierarchical entity tree, the higher-level entities have a more similar metatype to the entity, while the lower-level entities have a greater difference in metatype from the entity. b) Select a knowledge entity at each level of the hierarchical entity tree, replace the selected knowledge entity with the sample instance, generate a semantic relationship order based on the hierarchical level of the replaced entity in the hierarchical tree, and perform ranking learning based on the semantic relationship order and the relationship instances in the sample instance, and calculate the ranking loss function. c) Based on the relation instance in the sample instance, obtain entity pairs, and combine them with the alias of the relation instance extracted from the knowledge base, knowledge relation instances that are different from the relation instance and entity pair, and knowledge entities to form original triples, alias triples, extended triples, and other triples; wherein, the original triples include: entity pairs and relation instances, the alias triples include: entity pairs and relation instance aliases, and the other triples include: entity pairs and knowledge relation instances that are different from the relation instances. Extended triples are obtained through the following steps: Retrieve the head entity, relation instance, and tail entity of the original triple; Extract knowledge entities that are the same as the head entity, knowledge relation instances that are different from the relation instance, and knowledge entities that are different from the tail entity from the knowledge base, or knowledge entities that are different from the head entity, knowledge relation instances that are different from the relation instance, and knowledge entities that are the same as the tail entity from the knowledge base, to form triples to be expanded. The triplet to be expanded is concatenated with the original triplet to obtain the expanded triplet; d) Convert the original triples, alias triples, extended triples and other triples into strings according to a fixed template, and combine them with a pre-trained generative language model to generate natural language text corresponding to the original triples, alias triples, extended triples and other triples. e) Divide the natural language text into positive and negative examples, and compare the positive and negative examples with the original triples to calculate the contrastive loss function. Among them, triples with the same relation in alias triples and extended triples are positive examples, and other triples and triples with different relations in extended triples are negative examples. f) Use the ranking loss function and the contrastive loss function to train the pre-trained encoder until a well-trained encoder is obtained.
2. The method as described in claim 1, characterized in that, The knowledge base includes: WikiData knowledge base.
3. The method as described in claim 1, characterized in that, The knowledge entities in the hierarchical entity tree include: entities with the same meta-type as the entity are sibling entities, entities with the same second-level meta-type as the entity are neighbor entities, and other entities.
4. The method as described in claim 1, characterized in that, Clustering methods include the K-means algorithm.
5. A storage medium storing a computer program, wherein, The computer program is configured to execute any of the methods described in claims 1-4 at runtime.
6. An electronic device comprising a memory and a processor, the memory storing a computer program, the processor being configured to run the computer program to perform the method as claimed in any one of claims 1-4.