Relationship extraction model optimization method, device and electronic device

Abstract

The present application provides a relationship extraction model optimization method, device and electronic device to construct a relationship extraction model including a key mention pair classifier and a relationship classifier. First, each entity pair in the sample document and the mention pairs of each entity pair are obtained. Then, based on the mention pairs in the sample document, the key mention pair classifier is optimized, and the key mention pairs are identified according to the optimized key mention pair classifier, and then the relationship classifier is optimized based on the key mention pairs, thereby completing the optimization training of the relationship extraction model. In this scheme, the influence of different mention pairs under entity pairs on model training is considered, and the focus is also on using key mention pairs to implement model training, which can further avoid the influence of irrelevant information on model training and improve the recognition accuracy of the model.

Description

Technical Field

[0001] The present application relates to the technical field of document extreme relationship extraction, and in particular to a relationship extraction model optimization method, device and electronic device. Background Art

[0002] Relation extraction is an important research direction in the field of natural language processing, and its goal is to extract the relationship between given entities from a given text. For decades, a large number of researchers have conducted extensive explorations on the problem of sentence-level relationship extraction. In recent years, with the leapfrog development of pre-trained language models in the field of natural language processing, the models have stronger learning capabilities, making it possible to perform relationship extraction tasks on longer and more complex texts. At the same time, since a large number of real-life relationship facts are expressed through document-level text, researchers have gradually paid attention to the research on document-level relationship extraction and proposed a variety of techniques for entity relationship classification directly based on document-level text, including document graph-based methods and pre-trained language model-based methods.

[0003] The document graph-based method constructs a document graph and uses a graph neural network (GNN) to model the information interaction between different document components (such as words, sentences, etc.). This method first needs to build a document graph containing different nodes such as words and sentences based on the structural characteristics of the document, then use GNN to encode the document, model the structural characteristics of the document, and use the final two-entity representation to classify the relationship. Since Transformer-based pre-trained language models such as BERT (Bidirectional Encoder Representations from Transformers) can be regarded as having a fully connected graph structure, there are also methods that directly use the pre-trained language model to model the input document, and use a variety of pooling methods to obtain the contextual representation of the entity for relationship classification. This method is relatively simple to implement and can also achieve good results.

[0004] However, existing document-level relationship extraction methods ignore the impact of different entity mentions (defining each occurrence of an entity in a text as a mention) on the final relationship contribution when dealing with scenarios where the same entity appears multiple times in a document, thereby interfering with model training and affecting the model's recognition results. Summary of the invention

[0005] The purpose of the present application includes, for example, providing a relationship extraction model optimization method, device and electronic device, which can consider the impact of different mention pairs under entity pairs on model training and improve the recognition accuracy of the model.

[0006] The embodiments of the present application can be implemented as follows:

[0007] In a first aspect, the present application provides a method for optimizing a relationship extraction model, the method comprising:

[0008] Constructing a relation extraction model, wherein the relation extraction model includes a key mention pair classifier and a relation classifier;

[0009] Obtain a sample document, wherein the sample document contains a plurality of entity pairs, and each of the entity pairs has at least one mention pair;

[0010] Importing the sample document into the key mention pair classifier, constructing a first loss function based on the key tag of each of the mention pairs and the key prediction result of the key mention pair classifier for each of the mention pairs, so as to optimize the key mention pair classifier;

[0011] The sample document is imported into the relationship classifier, each key mention pair predicted by the key mention pair classifier is obtained, and a second loss function is constructed based on the classification label of each key mention pair and the classification prediction result of the relationship classifier for each key mention pair to optimize the relationship classifier.

[0012] In an optional embodiment, after the step of constructing the relationship extraction model, the method further includes the step of pre-training a classifier for the key mentions, the step comprising:

[0013] Tagging each mention contained in each mention pair in the sample document with an identifier;

[0014] Inputting the marked sample documents into the key mention pair classifier for encoding, and obtaining the attention distribution and context representation of each mentioned identifier;

[0015] Based on the attention distribution and context representation of each of the mentioned identifiers, the probability that each of the mentioned pairs belongs to a key mention pair is calculated to achieve pre-training of the key mention pair classifier.

[0016] In an optional embodiment, the step of calculating the probability that each of the mention pairs belongs to a key mention pair based on the attention distribution and context representation of each of the mentioned identifiers to implement pre-training of the key mention pair classifier includes:

[0017] Based on the attention distribution and contextual representation of each of the mentioned identifiers, obtaining a contextual pooled representation of each mention pair;

[0018] Calculating the probability that each mention pair belongs to a key mention pair based on the contextual pooled representation of each mention pair;

[0019] Pre-training of the key mention pair classifier is achieved based on the calculated probability that each mention pair belongs to the key mention pair and the key tag of each mention pair.

[0020] In an optional embodiment, the key tag of each of the mentioned pairs is obtained by:

[0021] Obtaining a classification prediction result of each mention pair predicted by the relationship classifier, wherein the classification prediction result is a probability that each mention pair belongs to each classification type;

[0022] For each of the mentioned pairs, when the probability corresponding to the mentioned pair is greater than the average probability of all the mentioned pairs under the entity pair to which the mentioned pair belongs, the key tag of the mentioned pair is set to characterize the mentioned pair as a key mention pair; otherwise, the key tag of the mentioned pair is set to characterize that the mentioned pair is not a key mention pair.

[0023] In an optional embodiment, the first loss function includes a likelihood function of a key tag set for each of the mention pairs in the sample document and a first weight;

[0024] Among them, the absolute value of the first weight is smaller when the key prediction result of the key mention pair classifier for the mention pair is closer to its key tag, and the absolute value of the first weight is larger when the key prediction result of the key mention pair classifier for the mention pair is more different from its key tag.

[0025] In an optional embodiment, the second loss function includes a likelihood function of a classification label set for each of the key mention pairs in the sample document and a second weight;

[0026] Among them, the absolute value of the second weight is smaller when the classification prediction result of the relationship classifier for the key mention pair is closer to its classification label, and the absolute value of the second weight is larger when the classification prediction result of the relationship classifier for the key mention pair is more different from its classification label.

[0027] In an optional implementation, the classification label of each key mention pair is obtained by:

[0028] Obtaining a relationship type tag of an entity pair to which the key mention pair belongs;

[0029] When the relationship type of the entity pair is marked as a set type and the classification probability corresponding to the key mention pair is greater than or equal to the average probability of all key mention pairs under the entity pair to which the key mention pair belongs, the classification mark of the key mention pair is set to the set type.

[0030] In an optional embodiment, the method further comprises:

[0031] Obtain a document to be marked, wherein the document to be marked contains multiple entity pairs, and each entity pair has at least one mention pair;

[0032] Importing the to-be-labeled document into a relation extraction model, and identifying key mention pairs of each entity pair through the key mention pair classifier;

[0033] According to each of the key mention pairs and the relationship classifier, obtaining a classification type of each of the key mention pairs;

[0034] For each entity pair, the classification types of all key mention pairs of the entity pair are integrated to obtain the classification type of the entity pair.

[0035] In a second aspect, the present application provides a relationship extraction model optimization device, comprising:

[0036] A construction module, used to construct a relation extraction model, wherein the relation extraction model includes a key mention pair classifier and a relation classifier;

[0037] An acquisition module, configured to acquire a sample document, wherein the sample document contains a plurality of entity pairs, each of which has at least one mention pair;

[0038] A first optimization module, configured to import the sample document into the key mention pair classifier, and construct a first loss function based on the key tags of each of the mention pairs and the key prediction results of the key mention pair classifier for each of the mention pairs, so as to optimize the key mention pair classifier;

[0039] The second optimization module is used to import the sample document into the relationship classifier, obtain the key mention pairs predicted by the key mention pair classifier, and construct a second loss function based on the classification label of each key mention pair and the classification prediction result of the relationship classifier for each key mention pair to optimize the relationship classifier.

[0040] In a third aspect, the present application provides an electronic device comprising one or more storage media and one or more processors communicating with the storage media, wherein the one or more storage media store machine-executable instructions executable by the processor, and when the electronic device is running, the processor executes the machine-executable instructions to perform the method steps described in any one of the aforementioned embodiments.

[0041] The beneficial effects of the embodiments of the present application include, for example:

[0042] The present application provides a relationship extraction model optimization method, device and electronic device to construct a relationship extraction model including a key mention pair classifier and a relationship classifier. First, each entity pair in the sample document and the mention pairs of each entity pair are obtained. Then, based on the mention pairs in the sample document, the key mention pair classifier is optimized, and the key mention pairs are identified according to the optimized key mention pair classifier, and then the relationship classifier is optimized based on the key mention pairs, thereby completing the optimization training of the relationship extraction model. In this scheme, the influence of different mention pairs under entity pairs on model training is considered, and the focus is also on using key mention pairs to implement model training, which can further avoid the influence of irrelevant information on model training and improve the recognition accuracy of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0044] Figure 1 A flowchart of a method for optimizing a relationship extraction model provided in an embodiment of the present application;

[0045] Figure 2 A flowchart of a method for pre-training a classifier provided by the key embodiment of the present application;

[0046] Figure 3 for Figure 2 Flow chart of sub-steps included in step S203;

[0047] Figure 4 A flowchart of a relationship extraction method provided in an embodiment of the present application;

[0048] Figure 5 A structural block diagram of an electronic device provided in an embodiment of the present application;

[0049] Figure 6 A functional module block diagram of the relationship extraction model optimization device provided in an embodiment of the present application.

[0050] Icons: 110 - storage medium; 120 - processor; 130 - relation extraction model optimization device; 131 - construction module; 132 - acquisition module; 133 - first optimization module; 134 - second optimization module; 140 - communication interface. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations.

[0052] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for which protection is sought, but merely represents selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0053] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0054] In addition, the terms “first”, “second”, etc., if used, are merely used to distinguish between the descriptions and should not be understood as indicating or implying relative importance.

[0055] It should be noted that, in the absence of conflict, the features in the embodiments of the present application may be combined with each other.

[0056] See also Figure 1 , is a flow chart of the relationship extraction model optimization method provided in the embodiment of the present application. The method steps defined in the process related to the relationship extraction model optimization method can be implemented by a computer with data processing function, such as a personal computer, a tablet computer, etc. Figure 1 The specific process shown is described in detail.

[0057] S101, constructing a relationship extraction model, wherein the relationship extraction model includes a key mention pair classifier and a relationship classifier.

[0058] S102, obtaining a sample document, wherein the sample document contains a plurality of entity pairs, and each of the entity pairs has at least one mention pair.

[0059] S103, importing the sample document into the key mention pair classifier, constructing a first loss function based on the key tags of each of the mention pairs and the key prediction results of the key mention pair classifier for each of the mention pairs, so as to optimize the key mention pair classifier.

[0060] S104, importing the sample document into the relationship classifier, obtaining each key mention pair predicted by the key mention pair classifier, and constructing a second loss function based on the classification label of each key mention pair and the classification prediction result of the relationship classifier for each key mention pair, so as to optimize the relationship classifier.

[0061] In this embodiment, for a given input document For example, a sample document contains n entities The task requires to identify each entity pair (e h ,e t ) relationship type, where e h ,e t ∈ε and represent the head entity and tail entity respectively. The head entity and tail entity can be any name, place name, or work name in the document. The relationship type of the entity pair can be relationship information such as a person was born in a certain place, a person died in a certain place, etc.

[0062] It is a set of all predefined relationship types. Each entity e i In a given document, times, defining each occurrence of the entity as a mention, e i The set of all mentions of can be defined as Each entity pair to be predicted (e h ,e t ) may contain several mention pairs, which are defined as For the convenience of description, the following and Respectively abbreviated as m i and m j For each relationship type If the entity pair (e h ,e t ) exists, then there must be at least one mention pair of the entity pair that expresses this relationship.

[0063] After the above modeling, the simplest way to extract document-level relations is to directly use the training data of the mentioned pair relations to train a relation classifier. However, since each entity may appear many times in each document, it is very time-consuming and labor-intensive to directly label the mentioned pairs. At the same time, most of the existing document-level relation extraction data directly labels the relations for each entity pair in the document.

[0064] To solve this problem, in this embodiment, the relationship labels of different mention pairs are regarded as latent variables, so that the document-level relationship extraction is modeled as a multiple instance learning (MIL) problem. Specifically, a latent variable z∈{0,1} can be assigned to each mention pair to indicate whether it is a key mention pair. Among them, the key mention pair is a mention pair that contributes to the determination of the relationship type among multiple mention pairs under the entity pair. For example, based on the document information, two entities may constitute a mention pair, but the mention pair does not reflect a meaningful correspondence between the two entities, then the mention pair is not a key mention pair. The relationship probability generation formula for entity pairs is defined as follows:

[0065] p(y c =1|X,z)=max{p(y c,(i,j) =1|m i ,m j )·I(z (i,j) =1)}

[0066] Among them, I(z (i,j) =1) is the indicator function, y c =1 if and only if there is a relationship c between the entity pair (or mention pair). In this embodiment, the objective function of the constructed relationship extraction model can be as follows:

[0067]

[0068] However, considering that the above-mentioned key mention pairs are embodied in the form of latent variables, the objective function of the relation extraction model is difficult to optimize directly. Therefore, in this embodiment, the relation extraction model is divided into a key mention pair classifier and a relation classifier. The optimization process of the above-mentioned objective function is divided into two stages using the expectation maximization (EM) algorithm: the E-step, optimizing the key mention pair classifier, is achieved by optimizing the Kullback-Leibler divergence (KL) between the predicted key mention pair distribution and the true key mention pair distribution to obtain a tighter variational lower bound; the M-step, by maximizing the likelihood of the training sample data, the variational lower bound is maximized.

[0069] By converting the optimization of the above relation extraction model into a multi-instance learning problem, the relation classifier and the key mention pair classifier can be used to fit the distribution p θ (y c,(i,j) =1|m i ,m j ) and p ω (z (i,j) =1|m i,m j ). Substituting into the original objective function, the transformation is as follows:

[0070]

[0071] Among them, H(p ω (z|X)) is p ω The entropy of can be regarded as a constant term during optimization.

[0072] After the transformation shown above, the objective function that is difficult to optimize directly is transformed into two optimization stages for transformation. In the above step E, p ω (z|X) and p θ (z|X,y c ) to obtain a tighter variational lower bound. In the above M steps, the variational lower bound ∫p is optimized ω (z|X)log p θ (z,y c |X)dz+H(p ω (z|X)). Here p ω (z|X) and H(p ω (z|X)) are all considered constant terms. Therefore, we only need to consider optimizing p θ (z,y c |X) This part can be achieved by optimizing the likelihood function of the training data based on the pseudo-label strategy.

[0073] In this embodiment, after constructing the key mention pair classifier and the relationship classifier, the two classifiers may be pre-trained before formal optimization to obtain better initial parameters in advance.

[0074] See also Figure 2 In this embodiment, the pre-training of the key mention classifier can be achieved by:

[0075] S201, marking each mention contained in each mention pair in the sample document with an identifier.

[0076] S202, inputting the marked sample document into the key mention pair classifier for encoding, and obtaining the attention distribution and context representation of each mentioned identifier.

[0077] S203, based on the attention distribution and context representation of each of the mentioned identifiers, calculating the probability that each of the mentioned pairs belongs to a key mention pair, so as to implement pre-training of the key mention pair classifier.

[0078] In this embodiment, for the sample document, an identifier can be inserted before and after the mentions contained in each mention pair, where the identifier can be, for example, "*" or an identifier related to the entity type, "[entity_type]", etc. The processed sample document is input into a key mention pair classifier, where the key mention pair classifier can be constructed based on a pre-trained language model such as BERT.

[0079] After the key mentions are used to encode the classifier, the attention distribution A between tokens in the last layer of the classifier and the context representation of the document are obtained. Then, we can extract each mention m i The attention distribution of the identifier and context representation Based on the attention distribution and context representation of each mentioned identifier, the probability that each corresponding mention pair belongs to a key mention pair can be calculated.

[0080] See also Figure 3 In this embodiment, the probability that each mention pair belongs to a key mention pair can be calculated in the following way to achieve pre-training of the key mention pair classifier:

[0081] S2031, based on the attention distribution and contextual representation of each mentioned identifier, obtain a contextual pooled representation of each mention pair.

[0082] S2032: Calculate the probability that each mention pair belongs to a key mention pair based on the context pooling representation of each mention pair.

[0083] S2033: Pre-training the key mention pair classifier according to the calculated probability that each mention pair belongs to the key mention pair and the key tag of each mention pair.

[0084] In this embodiment, the attention distribution of each mentioned identifier can be and context representation According to the following formula, for each mention pair (m i ,m j ) obtains its context pooling representation as follows:

[0085]

[0086]

[0087]

[0088] Based on the contextual pooling representation of each mention pair, the representation of each mention pair belonging to the key mention pair can be obtained as follows:

[0089]

[0090] Then, the probability that each mention pair belongs to a key mention pair can be calculated according to the following calculation formula:

[0091] p ω (z (i,j) |m i ,m j )=σ(W k x (i,j)′ +b k )

[0092] Among them, σ represents the sigmoid activation function, and W and b are parameters in the BERT model.

[0093] The probability that each mention pair belongs to a key mention pair is calculated in the above manner. Combined with the key tags of each mention pair, the key mention pair classifier can be pre-trained.

[0094] In addition, the pre-training process for the relation classifier is similar to that for the key mention pair classifier. The mention-level relation classifier is similar in function to the traditional sentence-level relation classifier model, but unlike the sentence-level classifier, it processes longer document-level text. Considering that a lot of information irrelevant to relation classification may be introduced in the document, a local context pooling mechanism is used to obtain the representation of the mention pair when pre-training the relation classifier and the key mention pair classifier.

[0095] In this embodiment, the pre-training of the relation classifier can be achieved in the following ways:

[0096] Special identifiers (such as "*" or identifiers related to entity types, such as "[entity_type]", etc.) can be inserted before and after each mention in the sample text. The processed sample documents are imported into a relation classifier, wherein the relation classifier can be built based on a pre-trained language model such as BERT.

[0097] In this embodiment, the relationship classifier and the key mention pair classifier have different functions and behaviors. In order to prevent interference with each other's training, two BERT pre-trained language models with the same structure but different parameters are used to construct the relationship classifier and the key mention pair classifier. Optionally, in this embodiment, the PyTorch deep learning library and the HuggingfaceTransformers library can be used to implement the design of the BERT base pre-trained language model respectively.

[0098] It should be noted that in this embodiment, the models of the relationship classifier and the key mention pair classifier can also adopt a document-level relationship extraction model based on a document graph, such as GAIN (Graph Aggregation-and-Inference Network), or other document-level relationship extraction models based on a pre-trained language model, such as SSAN (Structured Self-Attention Network).

[0099] After the processed sample documents are imported into the relational classifier, the attention distribution A between the tokens in the last layer and the context representation of the text are obtained through encoding processing of the relational classifier.

[0100] Then extract each mention m i Attention distribution of previous identifiers and context representation

[0101] Based on the attention distribution and context representation of each mention, we get each mention pair (m i ,m j ) Its context pooling representation:

[0102]

[0103]

[0104]

[0105] The representation of relationship classification for each mention pair is as follows:

[0106]

[0107] Compute the probability that each mention pair belongs to each relation type:

[0108]

[0109] On this basis, the relationship classifier can be pre-trained by combining the calculated probabilities of each mention pair belonging to each relationship type and the classification labels of each mention pair belonging to each relationship type.

[0110] In this embodiment, after the pre-training of the relationship classifier and the key mention pair classifier is completed, the above-mentioned EM algorithm can be used to optimize the relationship classifier and the key mention pair classifier.

[0111] In the above step S130, when optimizing the key mention pair classifier, the key tags of each mention pair and the key prediction results of each mention pair by the key mention pair classifier are used, and the optimization is implemented based on the first loss function. ω (z|X) and p θ (z|X,y c ) of the KL divergence, and obtain a tighter variational lower bound to achieve it. Among them, p θ (z|X,y c ) represents the key tag of the mention pair, i.e., the distribution of the real key mention pairs, p ω (z|X) denotes the predicted key mention pair distribution.

[0112] However, there is no annotation of key mention pair information in the existing data. Therefore, in this embodiment, a pseudo-labeling mechanism is designed to obtain a pseudo-label of whether each mention pair is a key mention pair based on the relationship probability predicted by the relationship classifier and the true relationship annotation of the entity pair.

[0113] In this embodiment, specifically, the key tag of each mention pair can be obtained in the following way:

[0114] Obtain the classification prediction results of each mention pair predicted by the relation classifier, wherein the classification prediction results are the probabilities that each mention pair belongs to each classification type.

[0115] For each mention pair, when the probability corresponding to the mention pair is greater than the average probability of all mention pairs under the entity pair to which the mention pair belongs, the key tag of the mention pair is set to characterize the mention pair as a key mention pair; otherwise, the key tag of the mention pair is set to characterize that the mention pair is not a key mention pair.

[0116] In this embodiment, the key tags of each mention pair can be set in the following manner:

[0117]

[0118] in, When it is 1, it represents a key mention pair, and when it is 0, it represents not a key mention pair. Represents the average probability of all mention pairs under the entity pair. y c is an entity pair (e h ,e t ) relationship annotation.

[0119] In the general optimization process, the KL divergence can be reduced by optimizing the likelihood function of the key tags of the mentioned pairs, that is, the pseudo-label data. The loss function is as follows:

[0120]

[0121] Among them, p ω (z (i,j) ) is calculated as follows:

[0122]

[0123] However, considering that among a large number of mentions of the same entity pair, generally only a small number of mentions are key mentions, and most mentions are not key mentions, there is a problem of sample imbalance. Therefore, in order to alleviate the impact of sample imbalance, the first loss function for optimizing the key mention classifier finally constructed in this embodiment includes the likelihood function of the key tags set for each mention pair in the sample document and the first weight.

[0124] Among them, the absolute value of the first weight is smaller when the key prediction result of the key mention pair classifier for the mention pair is closer to its key tag, and the absolute value of the first weight is larger when the key prediction result of the key mention pair classifier for the mention pair is more different from its key tag.

[0125] The final constructed first loss function can be shown as follows:

[0126]

[0127] in, is the first weight, α ω and γ ω is a predefined hyperparameter.

[0128] The above expression means that p ω (z (i,j) ) is the probability that the mention pair predicted by the key mention pair classifier belongs to the key mention pair. When the pseudo-label of the mention pair is 1, the larger the value is, the higher the accuracy of the classifier's prediction is. After substituting the above-mentioned first weight, the smaller the absolute value of the first weight is, the lower its influence in the optimization process of the classifier. Conversely, for samples with lower accuracy predicted by the classifier, the larger the absolute value of the corresponding first weight is, and the higher its influence in the optimization process of the classifier. Therefore, the above idea of ​​Focal Loss is used to increase the classifier's attention to samples with incorrect predictions.

[0129] It should be noted that, in this embodiment, in addition to using the above-mentioned Focal Loss to avoid the data imbalance problem, other similar loss functions such as cross entropy can also be directly used.

[0130] In addition, in the above step S140, based on the classification labels of each key mention pair and the classification prediction results of each key mention pair by the relationship classifier, the relationship classifier is optimized based on the second loss function.

[0131] From the above, we can see that when executing, we need to optimize the variational lower bound ∫p ω (z|X)log p θ (z,y c |X)dz+H(p ω (z|X)) and can be transformed into a problem of optimizing the log-likelihood of the training data.

[0132] In this embodiment, similarly, the classification labels of the key mention pairs are obtained by setting pseudo labels of the key mention pairs. The classification labels of each key mention pair can be obtained in the following manner:

[0133] Get the relationship type tag of the entity pair to which the key mention pair belongs. When the relationship type tag of the entity pair is a set type and the classification probability corresponding to the key mention pair is greater than or equal to the average probability of the key mention pair under the entity pair to which the key mention pair belongs, the classification tag of the key mention pair is set to the set type.

[0134] In this embodiment, the probability that each entity pair belongs to each classification type can be converted into a representation of whether the entity pair belongs to each classification type. For example, when the entity pair belongs to a certain classification type, the relationship label of the entity pair for the classification type can be marked as 1, otherwise the relationship label of the entity pair for the classification type can be marked as 0.

[0135] Specifically, the classification tags of each mention pair can be identified as follows:

[0136]

[0137] in, y c is the classification label of the entity pair.

[0138] Similarly, in order to avoid the problem of data imbalance between different relationship categories, a second weight is added to the second loss function in the relationship classifier optimization process.

[0139] Specifically, the second loss function includes the likelihood function of the classification label set for each key mention pair in the sample document and the second weight. The absolute value of the second weight is smaller when the classification prediction result of the relationship classifier for the key mention pair is closer to its classification label, and the absolute value of the second weight is larger when the classification prediction result of the relationship classifier for the key mention pair is more different from its classification label.

[0140] In this embodiment, the second loss function finally constructed is as follows:

[0141]

[0142] Among them, α θand γ θ is a predefined hyperparameter, p θ (y c,(i,j) ) is calculated as follows:

[0143]

[0144] In this embodiment, the key mention classifier and the relationship classifier are optimized in the above manner. After multiple iterations of optimization, when the training reaches convergence, a relationship extraction model with excellent performance can be obtained.

[0145] The above process is the process of optimizing the relationship extraction model. After obtaining a relationship extraction model with excellent performance, relationship extraction of document-level text can be implemented based on the relationship extraction model.

[0146] See also Figure 4 In this embodiment, the relationship extraction can be implemented based on the optimized relationship extraction model in the following manner:

[0147] S301, obtaining a document to be marked, wherein the document to be marked includes a plurality of entity pairs, and each entity pair has at least one mention pair.

[0148] S302: Import the document to be marked into a relation extraction model, and identify the key mention pairs of each entity pair through the key mention pair classifier.

[0149] S303: Obtain a classification type of each of the key mention pairs according to each of the key mention pairs and the relationship classifier.

[0150] S304: For each entity pair, the classification types of all key mention pairs of the entity pair are integrated to obtain the classification type of the entity pair.

[0151] The document to be marked can be any document that needs to be subjected to relationship extraction. Similarly, the document to be marked contains multiple entity pairs, such as a person and a place, a person and a work, etc. Each entity pair has at least one mention pair, wherein each occurrence of an entity can be defined as a mention.

[0152] The key mention pair classifier in the optimized relationship extraction model can identify the key mention pairs of each mention pair, thereby eliminating the influence of non-key mention pairs on relationship determination.

[0153] On this basis, the classification type of each key mention pair can be predicted by the relational classifier, for example, a person was born in a certain place, a person has a certain work, etc.

[0154] For an entity pair containing at least two key mention pairs, if the classification types of the two key mention pairs are different, for example, one mention pair is that resident A was born in place A, and the other mention pair is that resident A died in place A. It can be seen that the classification types of different mention pairs under the entity pair of resident A and place A are different. Therefore, the classification types of the two key mention pairs under the entity pair can be combined to obtain the classification type of the entity pair, that is, the entity pair has the two classification types of the two key mention pairs.

[0155] The relationship extraction model optimization and relationship extraction method provided in this embodiment can utilize the existing document-level relationship extraction dataset annotated at the entity pair level, and achieve efficient training in the case of missing data annotation by modeling the distribution of the latent variable of key mention pairs. In addition, focusing on the classification prediction of key mention pairs can eliminate the interference of irrelevant information on classification and improve the model training effect.

[0156] See also Figure 5 , is a schematic diagram of exemplary components of an electronic device provided in an embodiment of the present application, and the electronic device may be the above-mentioned personal computer, laptop computer, etc. The electronic device may include a storage medium 110, a processor 120, a relationship extraction model optimization device 130 and a communication interface 140. In the present embodiment, the storage medium 110 and the processor 120 are both located in the electronic device and the two are separately arranged. However, it should be understood that the storage medium 110 may also be independent of the electronic device and may be accessed by the processor 120 through a bus interface. Alternatively, the storage medium 110 may also be integrated into the processor 120, for example, it may be a cache and / or a general register.

[0157] The relationship extraction model optimization device 130 can be understood as the above-mentioned electronic device, or the processor 120 of the electronic device, or can be understood as a software function module independent of the above-mentioned electronic device or processor 120 that implements the above-mentioned relationship extraction model optimization method under the control of the electronic device.

[0158] like Figure 6 As shown, the above-mentioned relation extraction model optimization device 130 may include a construction module 131, an acquisition module 132, a first optimization module 133 and a second optimization module 134. The functions of each functional module of the relation extraction model optimization device 130 are respectively described in detail below.

[0159] A construction module 131 is used to construct a relationship extraction model, wherein the relationship extraction model includes a key mention pair classifier and a relationship classifier;

[0160] It can be understood that the construction module 131 can be used to execute the above step S101. The detailed implementation of the construction module 131 can refer to the above content related to step S101.

[0161] An acquisition module 132 is used to acquire a sample document, wherein the sample document contains a plurality of entity pairs, each of which has at least one mention pair;

[0162] It can be understood that the acquisition module 132 can be used to execute the above step S102. The detailed implementation of the acquisition module 132 can refer to the above content related to step S102.

[0163] A first optimization module 133, configured to import the sample document into the key mention pair classifier, and construct a first loss function based on the key tags of each of the mention pairs and the key prediction results of the key mention pair classifier for each of the mention pairs, so as to optimize the key mention pair classifier;

[0164] It can be understood that the first optimization module 133 can be used to execute the above step S103. The detailed implementation of the first optimization module 133 can refer to the above content related to step S103.

[0165] The second optimization module 134 is used to import the sample document into the relationship classifier, obtain the key mention pairs predicted by the key mention pair classifier, and construct a second loss function based on the classification label of each key mention pair and the classification prediction result of the relationship classifier for each key mention pair to optimize the relationship classifier.

[0166] It can be understood that the second optimization module 134 can be used to execute the above step S104. The detailed implementation of the second optimization module 134 can refer to the above content related to step S104.

[0167] In a possible implementation, the relationship extraction model optimization device 130 further includes a pre-training module for pre-training a key mention pair classifier, wherein the pre-training module is used to:

[0168] Tagging each mention contained in each mention pair in the sample document with an identifier;

[0169] Inputting the marked sample documents into the key mention pair classifier for encoding, and obtaining the attention distribution and context representation of each mentioned identifier;

[0170] Based on the attention distribution and context representation of each of the mentioned identifiers, the probability that each of the mentioned pairs belongs to a key mention pair is calculated to achieve pre-training of the key mention pair classifier.

[0171] In a possible implementation, the above pre-training module may be specifically used for:

[0172] Based on the attention distribution and contextual representation of each of the mentioned identifiers, obtaining a contextual pooled representation of each mention pair;

[0173] Calculating the probability that each mention pair belongs to a key mention pair based on the contextual pooled representation of each mention pair;

[0174] Pre-training of the key mention pair classifier is achieved based on the calculated probability that each mention pair belongs to the key mention pair and the key tag of each mention pair.

[0175] In a possible implementation, the key tag of each of the mentioned pairs is obtained in the following manner:

[0176] Obtaining a classification prediction result of each mention pair predicted by the relationship classifier, wherein the classification prediction result is a probability that each mention pair belongs to each classification type;

[0177] For each of the mentioned pairs, when the probability corresponding to the mentioned pair is greater than the average probability of all the mentioned pairs under the entity pair to which the mentioned pair belongs, the key tag of the mentioned pair is set to characterize the mentioned pair as a key mention pair; otherwise, the key tag of the mentioned pair is set to characterize that the mentioned pair is not a key mention pair.

[0178] In a possible implementation manner, the first loss function includes a likelihood function of a key tag set for each of the mention pairs in the sample document and a first weight;

[0179] Among them, the absolute value of the first weight is smaller when the key prediction result of the key mention pair classifier for the mention pair is closer to its key tag, and the absolute value of the first weight is larger when the key prediction result of the key mention pair classifier for the mention pair is more different from its key tag.

[0180] In a possible implementation, the second loss function includes a likelihood function of a classification label set for each of the key mention pairs in the sample document and a second weight;

[0181] Among them, the absolute value of the second weight is smaller when the classification prediction result of the relationship classifier for the key mention pair is closer to its classification label, and the absolute value of the second weight is larger when the classification prediction result of the relationship classifier for the key mention pair is more different from its classification label.

[0182] In a possible implementation, the classification label of each key mention pair is obtained in the following manner:

[0183] Obtaining a relationship type tag of an entity pair to which the key mention pair belongs;

[0184] When the relationship type mark of the entity pair is a set type and the classification probability corresponding to the key mention pair is greater than or equal to the average probability of all key mention pairs under the entity pair to which the key mention pair belongs, the classification mark of the key mention pair is set to the set type.

[0185] In a possible implementation, the relationship extraction model optimization device 130 further includes a relationship extraction module, which can be used to:

[0186] Obtain a document to be marked, wherein the document to be marked contains multiple entity pairs, and each entity pair has at least one mention pair;

[0187] Importing the to-be-labeled document into a relation extraction model, and identifying key mention pairs of each entity pair through the key mention pair classifier;

[0188] According to each of the key mention pairs and the relationship classifier, obtaining a classification type of each of the key mention pairs;

[0189] For each entity pair, the classification types of all key mention pairs of the entity pair are integrated to obtain the classification type of the entity pair.

[0190] For descriptions of the processing flow of each module in the device and the interaction flow between each module, reference may be made to the relevant descriptions in the above method embodiment, which will not be described in detail here.

[0191] Furthermore, an embodiment of the present application also provides a computer-readable storage medium, which stores machine-executable instructions. When the machine-executable instructions are executed, the relationship extraction model optimization method provided in the above embodiment is implemented.

[0192] Specifically, the computer-readable storage medium can be a general storage medium, such as a mobile disk, a hard disk, etc. When the computer program on the computer-readable storage medium is executed, the above-mentioned relationship extraction model optimization method can be executed. Regarding the process involved when the computer-readable storage medium and its executable instructions are executed, reference can be made to the relevant description in the above-mentioned method embodiment, which will not be described in detail here.

[0193] In summary, the present application provides a relationship extraction model optimization method, device and electronic device to construct a relationship extraction model including a key mention pair classifier and a relationship classifier. First, obtain each entity pair in the sample document and the mention pairs of each entity pair. Then, based on the mention pairs in the sample document, optimize the key mention pair classifier, identify the key mention pairs according to the optimized key mention pair classifier, and then optimize the relationship classifier based on the key mention pairs, thereby completing the optimization training of the relationship extraction model. In this scheme, the influence of different mention pairs under entity pairs on model training is considered, and the focus is also on using key mention pairs to implement model training, which can further avoid the influence of irrelevant information on model training and improve the recognition accuracy of the model.

[0194] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A method for optimizing a relationship extraction model, characterized in that: The method comprises: Constructing a relation extraction model, wherein the relation extraction model includes a key mention pair classifier and a relation classifier; Acquire a sample document, wherein the sample document contains a plurality of entity pairs, each of the entity pairs has at least one mention pair, and the key mention pair is a mention pair that contributes to determining a relationship type among a plurality of mention pairs under the entity pair; Importing the sample document into the key mention pair classifier, constructing a first loss function based on the key tag of each of the mention pairs and the key prediction result of the key mention pair classifier for each of the mention pairs, so as to optimize the key mention pair classifier; Importing the sample document into the relationship classifier, obtaining each key mention pair predicted by the key mention pair classifier, and constructing a second loss function based on the classification label of each key mention pair and the classification prediction result of the relationship classifier for each key mention pair, so as to optimize the relationship classifier; After the step of constructing the relationship extraction model, the method further includes the step of pre-training the classifier for the key mentions, which step includes: Tagging each mention contained in each mention pair in the sample document with an identifier; Inputting the marked sample documents into the key mention pair classifier for encoding, and obtaining the attention distribution and context representation of each mentioned identifier; Based on the attention distribution and context representation of each of the mentioned identifiers, the probability that each of the mentioned pairs belongs to a key mention pair is calculated to achieve pre-training of the key mention pair classifier.

2. The method for optimizing the relationship extraction model according to claim 1, characterized in that: The step of calculating the probability that each of the mentioned pairs belongs to a key mention pair based on the attention distribution and context representation of each of the mentioned identifiers to implement pre-training of the key mention pair classifier includes: Based on the attention distribution and contextual representation of each of the mentioned identifiers, obtaining a contextual pooled representation of each mention pair; Calculating the probability that each mention pair belongs to a key mention pair based on the contextual pooled representation of each mention pair; Pre-training of the key mention pair classifier is achieved based on the calculated probability that each mention pair belongs to the key mention pair and the key tag of each mention pair.

3. The method for optimizing the relationship extraction model according to claim 1, characterized in that: The key tag of each mentioned pair is obtained by: Obtaining a classification prediction result of each mention pair predicted by the relationship classifier, wherein the classification prediction result is a probability that each mention pair belongs to each classification type; For each of the mentioned pairs, when the probability corresponding to the mentioned pair is greater than the average probability of all the mentioned pairs under the entity pair to which the mentioned pair belongs, the key tag of the mentioned pair is set to characterize the mentioned pair as a key mention pair; otherwise, the key tag of the mentioned pair is set to characterize that the mentioned pair is not a key mention pair.

4. The method for optimizing a relation extraction model according to claim 1, wherein: The first loss function includes a likelihood function of a key tag set for each of the reference pairs in the sample document and a first weight; Among them, the absolute value of the first weight is smaller when the key prediction result of the key mention pair classifier for the mention pair is closer to its key tag, and the absolute value of the first weight is larger when the key prediction result of the key mention pair classifier for the mention pair is more different from its key tag.

5. The method for optimizing a relation extraction model according to claim 1, wherein: The second loss function includes a likelihood function of the classification label set for each of the key mention pairs in the sample document and a second weight; Among them, the absolute value of the second weight is smaller when the classification prediction result of the relationship classifier for the key mention pair is closer to its classification label, and the absolute value of the second weight is larger when the classification prediction result of the relationship classifier for the key mention pair is more different from its classification label.

6. The method for optimizing a relation extraction model according to claim 1, characterized in that: The classification label of each key mention pair is obtained in the following way: Obtaining a relationship type tag of an entity pair to which the key mention pair belongs; When the relationship type of the entity pair is marked as a set type and the classification probability corresponding to the key mention pair is greater than or equal to the average probability of all key mention pairs under the entity pair to which the key mention pair belongs, the classification mark of the key mention pair is set to the set type.

7. The method for optimizing a relationship extraction model according to any one of claims 1 to 6, characterized in that: The method further comprises: Obtain a document to be marked, wherein the document to be marked contains multiple entity pairs, and each entity pair has at least one mention pair; Importing the to-be-labeled document into a relation extraction model, and identifying key mention pairs of each entity pair through the key mention pair classifier; According to each of the key mention pairs and the relationship classifier, obtaining a classification type of each of the key mention pairs; For each entity pair, the classification types of all key mention pairs of the entity pair are integrated to obtain the classification type of the entity pair.

8. A relationship extraction model optimization device, characterized in that: include: A construction module, used to construct a relation extraction model, wherein the relation extraction model includes a key mention pair classifier and a relation classifier; An acquisition module is used to acquire a sample document, wherein the sample document contains a plurality of entity pairs, each of the entity pairs has at least one mention pair, and the key mention pair is a mention pair that contributes to the determination of the relationship type among the plurality of mention pairs under the entity pair; A first optimization module, configured to import the sample document into the key mention pair classifier, and construct a first loss function based on the key tags of each of the mention pairs and the key prediction results of the key mention pair classifier for each of the mention pairs, so as to optimize the key mention pair classifier; A second optimization module is used to import the sample document into the relationship classifier, obtain each key mention pair predicted by the key mention pair classifier, and construct a second loss function based on the classification label of each key mention pair and the classification prediction result of the relationship classifier for each key mention pair, so as to optimize the relationship classifier; A pre-training module is used to mark the mentions contained in each mention pair in the sample document with an identifier, input the marked sample document into the key mention pair classifier for encoding, obtain the attention distribution and context representation of each identifier of the mention, and calculate the probability that each mention pair belongs to a key mention pair based on the attention distribution and context representation of each identifier of the mention, so as to realize the pre-training of the key mention pair classifier.

9. An electronic device, characterized in that: The electronic device comprises one or more storage media and one or more processors communicating with the storage media, wherein the one or more storage media store machine executable instructions executable by the processor, and when the electronic device is running, the processor executes the machine executable instructions to execute the method described in any one of claims 1 to 7.

Images