A progressive confrontation-based entity set expansion method and system

Abstract

The present invention discloses a method and system for entity set expansion based on progressive adversarial learning, which belongs to the technical field of natural language processing. By constructing an expansion model and a boundary discrimination model based on contextual features, the entity category boundaries are adjusted and determined using multiple rounds of adversarial learning based on a small number of samples, thereby solving the problems of insufficient samples when determining boundaries and the inability to adjust dynamically.

Description

Technical Field

[0001] The present invention relates to an entity set expansion method and system, and in particular to an entity set expansion method based on progressive confrontation, belonging to the technical field of natural language processing. Background Art

[0002] In natural language processing, efficiently acquiring knowledge from natural text has long been a key research topic, crucial for other AI tasks such as knowledge graph construction, intelligent information retrieval, intelligent question answering, and machine reading comprehension. On-demand knowledge acquisition, which aims to rapidly and automatically acquire knowledge using only a small amount of example knowledge as a seed, holds significant application potential. Entity set expansion is an example of this on-demand acquisition of entity sets.

[0003] The entities referred to in the present invention are generally things that exist objectively and can be distinguished from each other, such as names of people, places, organizations, etc. Entity set expansion hopes to use only a small number of entities of a certain category as seed entities to quickly obtain other entities of the same category. Specifically, entity set expansion is a multi-round iterative process: each round of expansion uses the current entity set as a positive example, and by comparing the similarity between the context features of the new entity and the entities in the set, it is determined whether the new entity belongs to the category boundary (that is, it belongs to the same category as the entities in the set) or outside the category boundary (that is, it belongs to a different category than the entities in the set), and the entities within the boundary are added to the current entity set to form a larger set for the next round of iteration.

[0004] Because entity set expansion is a dynamic process, using only a small number of seed entities is insufficient to effectively determine the category boundaries. This is partly due to insufficient seeds for entity set expansion, namely, only a small number of seed entities within the category boundary serve as positive samples, while entities outside the boundary serve as negative samples. Furthermore, as the entity set expands during expansion, the resulting category boundaries often change dynamically. Traditional methods for determining category boundaries have two main approaches: one is to heuristically define an inter-entity distance metric based on statistical features or semantic representations and compare the distance between new entities and example entities to determine whether they fall within the category boundary. The other is to automatically learn a distance metric function to determine the category boundary using a small number of existing example samples or more supervisory signals obtained through other means. However, the first approach not only requires significant manual intervention, but the heuristically defined boundary often deviates significantly from the true boundary. The second approach, while capable of learning a boundary metric function, is often limited by the scarcity of samples and cannot effectively learn the true boundary. Furthermore, neither approach considers the dynamic nature of boundary determination and cannot be dynamically adjusted. Summary of the Invention

[0005] The purpose of the present invention is to provide an entity set expansion method and system based on progressive adversarial learning, which uses multiple rounds of adversarial learning to determine and adjust the category boundaries of entities, solving the problems of insufficient samples when determining boundaries and the inability to dynamically adjust them.

[0006] To achieve the above object, the present invention adopts the following technical solutions:

[0007] A method for expanding entity sets based on progressive adversarial learning includes the following steps:

[0008] Construct an expansion model and a boundary discrimination model. The expansion model expands new entities based on the contextual features of the entities, and the boundary discrimination model determines whether the new entities belong to the category boundary based on the contextual features of the new entities.

[0009] The expansion model and a set of multiple boundary discriminant models are combined to perform global and progressive boundary adjustment. That is, the first boundary discriminant model uses the new entities expanded in the first step of the expansion model as positive samples and performs local adversarial learning with the expansion model. Each subsequent boundary discriminant model copies the parameters of the previous boundary discriminant model and uses the new entities expanded in the latest step of the expansion model as positive samples and performs local adversarial learning with the expansion model. Each step of local adversarial learning learns the latest category boundary and the new entities that should be expanded.

[0010] The local adversarial learning method comprises the following steps: jointly performing local adversarial learning training on a boundary discriminant model and an expansion model, wherein the boundary discriminant model gives the boundary discriminant probability of the new entity and the known entity set and its corresponding boundary discriminant entropy based on the new entity expanded by the expansion model, and the boundary discriminant model is trained with the goal of maximizing the boundary discriminant entropy of the new entity and minimizing the boundary discriminant entropy of the entity set to learn the new category boundary; and the boundary discriminant probability of the new entity given by the trained boundary discriminant model is returned to the expansion model, and the expansion model is trained with the goal of minimizing the boundary discriminant entropy of the new entity to learn the new entity that should be expanded.

[0011] Furthermore, the extended model represents the relationship between entities and their contexts using a bipartite graph, and uses graph neural networks to obtain contextual features of new entities.

[0012] Furthermore, the boundary discrimination model represents the relationship between entities and their contexts using a bipartite graph, uses a graph neural network to obtain the contextual features of new entities, and uses a multi-layer perceptron to determine whether the new entity belongs to the category boundary.

[0013] Furthermore, in the first step of the expansion model, new entities are expanded based on the seed entity set, and in each subsequent step of expansion, new entities are expanded based on the seed set and the entities expanded in the previous steps and determined to belong to the category boundary.

[0014] Furthermore, when training the extended model, minimizing the boundary discriminant entropy of the new entity is simplified to equivalently maximizing the boundary discriminant probability of the new entity.

[0015] A progressive adversarial entity set expansion system, comprising:

[0016] The expansion model is used to expand new entities based on the contextual features of the entity. The boundary discrimination model determines whether the new entity belongs to the category boundary based on the contextual features of the new entity;

[0017] Boundary discrimination model, which is used to determine whether a new entity belongs to the category boundary based on its contextual features;

[0018] The joint expansion model and a set of multiple boundary discriminant models perform global progressive boundary adjustment. That is, the first boundary discriminant model uses the new entities expanded in the first step of the expansion model as positive samples and performs local adversarial learning with the expansion model. Each subsequent boundary discriminant model copies the parameters of the previous boundary discriminant model and uses the new entities expanded in the latest step of the expansion model as positive samples and performs local adversarial learning with the expansion model. Each step of local adversarial learning learns the latest category boundary and the new entity that should be expanded.

[0019] The local adversarial learning method comprises the following steps: jointly performing local adversarial learning training on a boundary discriminant model and an expansion model, wherein the boundary discriminant model gives the boundary discriminant probability of the new entity and the known entity set and its corresponding boundary discriminant entropy based on the new entity expanded by the expansion model, and the boundary discriminant model is trained with the goal of maximizing the boundary discriminant entropy of the new entity and minimizing the boundary discriminant entropy of the entity set to learn the new category boundary; and the boundary discriminant probability of the new entity given by the trained boundary discriminant model is returned to the expansion model, and the expansion model is trained with the goal of minimizing the boundary discriminant entropy of the new entity to learn the new entity that should be expanded.

[0020] The technical solution of the present invention first constructs a category boundary discrimination model based on contextual features, which can use contextual features to directly judge whether a new entity belongs to the boundary; then, through a local adversarial boundary learning method, in each round of entity expansion, the expansion model is required to expand positive examples as the goal, while the boundary discrimination model is to distinguish the newly expanded entities as the goal. The two are modeled through such adversarial learning goals, and eventually learn a good boundary discrimination model at the same time; finally, a global progressive boundary adjustment method is used. When learning the boundaries of a new round of expansion process, the boundaries learned in the previous round and the new expanded entities in this round are used to progressively adjust the original boundaries to dynamically adapt to the current expansion process. The local adversarial learning of the present invention in a single round of expansion process can make full use of the expanded new entities as additional examples to help establish the boundaries, solving the problem of insufficient samples when determining the boundaries; and the global progressive adjustment fully takes into account the dynamic nature of entity expansion, and gradually improves the learned boundaries for each expansion, so as to better adapt to the new round of expansion process. Compared with the existing technology, the present invention proposes a local learning method for entity expansion category boundaries, which effectively learns the expansion category boundaries through adversarial learning of the expansion model and the boundary discrimination model; and proposes a global boundary learning mechanism with progressive fine-tuning, which uses a set of boundary discrimination models to learn the boundaries of different expansion stages respectively, and continuously fine-tunes according to the new expansion process based on the learned boundaries. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 It is a unified framework diagram for entity set expansion based on progressive adversarial learning.

[0022] Figure 2 This is an example diagram of an entity and its surrounding context.

[0023] Figure 3 This is an example diagram of the global progressive boundary fine-tuning process. DETAILED DESCRIPTION

[0024] In order to make the technical solution of the present invention more clear and easy to understand, embodiments are given and described in detail with reference to the accompanying drawings as follows.

[0025] Figure 1 The figure shows a unified framework for entity set expansion based on progressive adversarial learning. During each expansion, the expansion model (bootstrapping network) strives to generate indistinguishable new entities, while the boundary discriminator model (discriminator network) strives to distinguish the new entities from the entities in the entity set, and feeds the degree of indistinguishability of the new entities as a reward to the expansion model for learning. This learning process is dynamically executed during multiple expansion processes.

[0026] A progressive adversarial entity set expansion method, comprising:

[0027] 1) The boundary discrimination model based on contextual features does not compare the similarity between the contexts of entities, but directly uses the entity context features to determine whether the new entity belongs to the category boundary.

[0028] 2) A global progressive boundary adjustment method addresses the dynamic changes in category boundaries during the expansion process. Multiple boundary discriminant models are used to learn the boundaries of different expansion stages. The new boundary starts from the boundary learned in the previous round and is re-adjusted to the new boundary through adversarial learning with the expansion model based on the entities expanded in this round.

[0029] The adversarial learning refers to a local adversarial boundary learning method. To address the problem of insufficient samples in boundary learning, when determining the boundary of a certain round of entity expansion, the new entities expanded by the expansion model are used as additional samples, and adversarial learning is used to jointly train the expansion model and the boundary discrimination model. The boundary discrimination model needs to try its best to distinguish between new entities and entities in the set, and the expansion model needs to try its best to confuse the distinction of the boundary discrimination model.

[0030] Each step of the above method is further described as follows:

[0031] 1. Boundary discrimination model based on contextual features

[0032] Design a boundary discrimination model to directly determine whether an entity belongs to a boundary. Specifically, the relationship between the entity and its context is represented by a bipartite graph (e.g. Figure 2 ), and uses a graph neural network to obtain the contextual feature encoding of entity associations, and uses a multi-layer perceptron to determine whether each entity belongs to the category.

[0033] 2. Global progressive boundary adjustment method

[0034] To address the dynamic changes in boundaries during the expansion process, a set of boundary discriminant models is used, each of which learns the boundaries of different expansion stages. Specifically, assuming that a boundary discriminant model has already learned the category boundary of the previous expansion process, the new boundary discriminant model will copy the parameters of the boundary discriminant model, thereby occupying the learned category boundary of the previous expansion process. The new boundary discriminant model then performs local adversarial learning with the expansion model to obtain the boundary of the current expansion stage.

[0035] The above-mentioned local adversarial learning refers to the local adversarial boundary learning method, which is explained as follows:

[0036] To address the problem of insufficient samples when learning boundaries, an adversarial learning mechanism is employed. Specifically, the new entities generated by the extended model are fully utilized as additional samples. The boundary discriminant model is required to distinguish between these new entities (e.g., {Thailand, NASA, DARPA}) and the entities in the entity set {China, Britain, Japan, France} as much as possible. At the same time, the extended model is required to extract as many new entities as possible that the boundary discriminant model cannot distinguish. Their learning objectives are as follows:

[0037]

[0038] Among them, D represents the boundary discrimination model, G represents the expansion model, c is the category of the seed entity, S c seed entities belonging to category c, It is the set of new entities expanded by the expansion model based on the seed entity in the first k-1 steps. It is an entity set consisting of the seed entity plus the expanded entity. refers to the new entity set to be expanded in step k, while e and e′ are entities sampled from different entity sets, p D (·) is the boundary discriminant probability output by the boundary discriminant model, slice (·) represents the entropy of the boundary discriminant probability, namely the boundary discriminant entropy, CE(.) represents the cross-entropy loss function, and E[·] represents the expectation function. The above learning objective consists of two parts, where the goal of D is to maximize the boundary discriminant entropy of the new entity and minimize the boundary discriminant entropy of the entity set, while the goal of G is to minimize the boundary discriminant entropy of the new entity, making it difficult to distinguish between the new entity and the entity set.

[0039] When learning the extended model, the indistinguishability of the new entity returned by the boundary discriminant model is used as a reward. Specifically, the extended model minimizes the boundary discriminant entropy of the new entity, which is equivalent to maximizing the boundary discriminant probability of the new entity. Learning is performed by maximizing the following formula:

[0040]

[0041] Among them, θ is the parameter of the extended model G, The parameter update gradient when the extended model is learned, p D (c|e′) is the boundary discrimination probability, is the probability of the extended model extending e′, and b is the baseline value (obtained by dividing 1 by the total number of categories |C|, i.e. ), log takes the natural constant as the base.

[0042] A specific embodiment is listed below:

[0043] 1) Local adversarial boundary learning

[0044] Scenario:

[0045] enter:

[0046] Seed entity set: {China, Britain};

[0047] The context surrounding the entity, such as Figure 2 , each entity may be associated with different contexts, and each context may be associated with different entities.

[0048] Implementation:

[0049] (1) First, expand the model to include new entities, such as {Thailand, NASA, DARPA}.

[0050] (2) Secondly, the boundary discrimination model is based on Figure 2 The context and entity association information shown are used to give the boundary discriminant probability of the new entity and the entity set and the corresponding boundary discriminant entropy. The boundary discriminant model is trained by maximizing the boundary discriminant entropy of the new entity and minimizing the boundary discriminant entropy of the entity set.

[0051] (3) Then, the trained boundary discrimination model returns the boundary discrimination probability of the new entity {Thailand, NASA, DARPA} to the extended model, and the extended model uses this probability as a reward for training.

[0052] (IV) Expand the model and expand new entities, such as {Thailand, France, Japan}, and then re-execute the steps starting from the second step until the set training conditions are met (such as loop execution 100 times).

[0053] 2) Global progressive boundary adjustment

[0054] Scenario:

[0055] enter:

[0056] Expand model G;

[0057] Seed entity set: {China,Britain};

[0058] The boundary discrimination model group {D1, D2, D3, ...}, such as Figure 3 As shown in Figure 2, different subscripts of D represent different boundary discrimination models.

[0059] Implementation:

[0060] (1) First, the expansion model expands the new entities in the first step based on the seed entity set, and performs adversarial learning with the first boundary discrimination model D1.

[0061] (2) Next, D2 copies the parameters of D1 and performs adversarial learning with the expanded model for the new entities expanded in the second step.

[0062] (3) Repeat the above steps, and let D i Copy D i-1 , and conduct adversarial learning with the extended model for the new entity expanded in step i.

[0063] Output: The learned boundary discriminant model group {D1,D2,D3,…}, as well as the expanded new entities such as {Thailand,USA,France,…}.

[0064] The above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit the same. Those skilled in the art may modify or replace the technical solutions of the present invention with equivalents. The scope of protection of the present invention shall be subject to the claims.

Claims

1. A progressive confrontation-based entity set expansion method, characterized in that: The following steps are involved: Construct an expansion model and a boundary discrimination model. The expansion model expands new entities based on the contextual features of the entities, and the boundary discrimination model determines whether the new entities belong to the category boundary based on the contextual features of the new entities. The expansion model and a set of multiple boundary discriminant models are combined to perform global and progressive boundary adjustment. That is, the first boundary discriminant model uses the new entities expanded in the first step of the expansion model as positive samples and performs local adversarial learning with the expansion model. Each subsequent boundary discriminant model copies the parameters of the previous boundary discriminant model and uses the new entities expanded in the latest step of the expansion model as positive samples and performs local adversarial learning with the expansion model. Each step of local adversarial learning learns the latest category boundary and the new entities that should be expanded. The local adversarial learning method comprises the following steps: jointly performing local adversarial learning training on a boundary discriminant model and an expansion model, wherein the boundary discriminant model gives the boundary discriminant probability of the new entity and the known entity set and its corresponding boundary discriminant entropy based on the new entity expanded by the expansion model, and the boundary discriminant model is trained with the goal of maximizing the boundary discriminant entropy of the new entity and minimizing the boundary discriminant entropy of the entity set to learn the new category boundary; and the boundary discriminant probability of the new entity given by the trained boundary discriminant model is returned to the expansion model, and the expansion model is trained with the goal of minimizing the boundary discriminant entropy of the new entity to learn the new entity that should be expanded.

2. The method according to claim 1, wherein The extended model represents the relationship between entities and their contexts using a bipartite graph, and uses graph neural networks to obtain contextual features of new entities.

3. The method according to claim 1, wherein The boundary discrimination model represents the relationship between entities and their contexts using a bipartite graph, uses a graph neural network to obtain the contextual features of new entities, and uses a multi-layer perceptron to determine whether the new entity belongs to the category boundary.

4. The method according to claim 1, wherein The first step of the expansion model is to expand new entities based on the seed entity set. Each subsequent step of expansion is to expand new entities based on the seed set and the entities expanded in the previous steps and determined to belong to the category boundary.

5. The method according to claim 4, wherein The objective function of local adversarial learning for the boundary discrimination model and the extended model is as follows: Among them, D represents the boundary discrimination model, G represents the expansion model, c represents the category of the seed entity, S c represents the seed entity belonging to category c, Represents the set of new entities expanded by the expansion model based on the seed entity in the first k-1 steps, Represents an entity set consisting of a seed entity plus an expanded entity. represents the new entity set to be expanded in step k, e and e′ represent entities sampled from different entity sets, and p D (·) represents the boundary discrimination probability output by the boundary discrimination model, H(·) represents the boundary discrimination entropy, CE(·) represents the cross entropy loss function, and E[·] represents the expectation function.

6. The method according to claim 5, wherein When training the extended model, minimizing the boundary discriminant entropy of the new entity is simplified to the equivalent maximization of the boundary discriminant probability of the new entity.

7. The method according to claim 6, wherein The extended model is learned according to the following maximization formula: Among them, θ represents the parameters of the extended model G, represents the parameter update gradient when the extended model is learned, p D (c|e′) represents the boundary discrimination probability, represents the probability of the extended model extending e′, b represents the baseline value, and log takes the natural constant as the base.

8. A progressive confrontation-based entity set expansion system, characterized in that: include: The expansion model is used to expand new entities based on the contextual features of the entity. The boundary discrimination model determines whether the new entity belongs to the category boundary based on the contextual features of the new entity; Boundary discrimination model, which is used to determine whether a new entity belongs to the category boundary based on its contextual features; The joint expansion model and a set of multiple boundary discriminant models perform global progressive boundary adjustment. That is, the first boundary discriminant model uses the new entities expanded in the first step of the expansion model as positive samples and performs local adversarial learning with the expansion model. Each subsequent boundary discriminant model copies the parameters of the previous boundary discriminant model and uses the new entities expanded in the latest step of the expansion model as positive samples and performs local adversarial learning with the expansion model. Each step of local adversarial learning learns the latest category boundary and the new entity that should be expanded. The local adversarial learning method comprises the following steps: jointly performing local adversarial learning training on a boundary discriminant model and an expansion model, wherein the boundary discriminant model gives the boundary discriminant probability of the new entity and the known entity set and its corresponding boundary discriminant entropy based on the new entity expanded by the expansion model, and the boundary discriminant model is trained with the goal of maximizing the boundary discriminant entropy of the new entity and minimizing the boundary discriminant entropy of the entity set to learn the new category boundary; and the boundary discriminant probability of the new entity given by the trained boundary discriminant model is returned to the expansion model, and the expansion model is trained with the goal of minimizing the boundary discriminant entropy of the new entity to learn the new entity that should be expanded.

9. The system according to claim 8, wherein The extended model represents the relationship between entities and their contexts using a bipartite graph, and uses graph neural networks to obtain contextual features of new entities.

10. The system according to claim 8, wherein The boundary discrimination model represents the relationship between entities and their contexts using a bipartite graph, uses a graph neural network to obtain the contextual features of new entities, and uses a multi-layer perceptron to determine whether the new entity belongs to the category boundary.

Images