A method for joint entity relationship extraction based on multi-feature fusion based on semantic enhancement

Abstract

The present invention discloses a method for joint extraction of entity relations based on semantically enhanced multi-feature fusion. By regarding the task of joint extraction of entity relations as a function that maps an object through a subject with a relation as a condition, the extraction idea of ​​first identifying the head entity and then identifying the tail entity under each relation is adopted. The head entity information is first identified, and the features with enhanced sequence dependency information after RNN encoding are used to identify the start and end positions of the head entity using a pointer network. The head entity and its entity type are used as prior information, and multiple features are fused with enhanced features to obtain a fusion vector with enhanced semantic expression ability, reducing the model's attention to semantically irrelevant entities. This method adopts an attention mechanism to make it easier for words in a sentence to capture the global features that have been added with the head entity information, strengthen feature dependency, and improve the performance of extracting relationship triplets in complex overlapping scenarios.

Description

Technical Field

[0001] The present invention belongs to the field of natural language processing information extraction and deep learning, and in particular relates to a method for joint entity relationship extraction based on semantic enhancement and multi-feature fusion. Background Art

[0002] With the rapid development of computers and the Internet, the amount of data generated and created by humans has exploded. How to efficiently utilize these massive amounts of heterogeneous data is a problem that needs to be solved urgently and is also one of the important contents of information extraction (IE) research. Entity-Relation Extraction (RE) is an important part of the information extraction field and has been widely used in search engines, intelligent recommendations, machine translation, and question-answering services. It includes two subtasks: entity recognition and relationship extraction. Entity recognition is one of the most basic tasks in the field of information extraction. It aims to identify words with specific meanings in text, such as names of people, places, and organizations. The relationship extraction task is based on the named entity recognition task, extracting the semantic relationships between entities and constructing them in the form of relationship triples.

[0003] Scholars at home and abroad have conducted significant research and exploration on the task of entity relationship extraction. The first category is pipeline entity relationship extraction methods based on deep learning. These methods do not require manual feature extraction and can automatically capture features of the input text through neural networks, which are then used for entity relationship extraction research. However, the relationship extraction results of pipeline extraction schemes are overly dependent on the results of entity recognition, ignoring the relationship between the two processes, which to some extent affects the effectiveness of relationship extraction. At the same time, errors in the entity recognition process are also propagated to the relationship extraction process. The second category is joint entity relationship extraction methods based on deep learning. Joint relationship extraction methods use a single model to combine the two processes of named entity recognition and relationship extraction, so that they can be jointly optimized in a unified model. Although these methods have achieved good results, they cannot completely solve the problem of overlapping triples, and when more complex overlapping situations exist in sentences, these methods still cannot achieve satisfactory results. However, recent research has found that the following problems are common in current entity relationship extraction methods: 1) The extraction model has poor semantic expression capabilities. Most existing extraction models use the BERT pre-trained model to obtain encoding vectors. Although it can provide a general word vector representation that includes the full context for the joint model, its direct use may ignore more fine-grained local dependency information, resulting in poor semantic feature expression of the model. 2) There is a large amount of relationship overlap in the corpus. Relationship overlap means that an entity participates in multiple relationship triples. There may be only one entity pair and relationship in the data (Simple), or there may be multiple relationships between an entity and another entity (EPO), or there may be multiple relationships between an entity and different entities (SEO). This increases the complexity of the corpus data, making it impossible for existing classifiers to effectively extract multiple triplets at the same time. Summary of the Invention

[0004] The purpose of this invention is to propose a semantically enhanced multi-feature fusion entity relation extraction model (MFF-WSE) to construct entity relation extraction tasks to solve the current common problems of poor semantic expression ability of models and a large number of relationship overlaps in corpora. By considering the entity relation extraction task as a function f that maps the subject to the object with the relationship as a condition, r(s)→o, adopting the extraction strategy of first identifying the head entity and then identifying the tail entity under each relationship, avoids the limitation that the subject and object can only be mapped to a single relationship and can effectively solve the problem of entity overlap. The proposed method first identifies the head entity information, uses the features with enhanced sequence dependency information after RNN encoding, and uses a pointer network to identify the start and end positions of the head entity. The head entity and its entity type are used as prior information, and multiple features are fused with enhanced features to obtain a fusion vector with enhanced semantic expression capabilities, reducing the model's attention to semantically irrelevant entities. Unlike traditional fusion methods that simply perform addition or concatenation, this method also uses an attention mechanism to make it easier for words in the sentence to capture the global features that incorporate the head entity information, strengthen feature dependencies, and thus improve the model's performance in extracting relation triplets in complex overlapping scenarios.

[0005] Therefore, the main steps of the model of the proposed method are as follows:

[0006] Step 1: Data encoding: Input the data to be used for the entity relationship extraction task into the pre-trained model to obtain the sentence shared encoding vector matrix and entity type vector representation of the data.

[0007] Step 2: Contextual information enhancement: The shared encoding vector matrix obtained in step 1 is input into the contextual information enhancement module. A bidirectional long short-term memory network (BiLSTM) is used to retain and transfer key features in the input data through a gating mechanism, thereby performing deep feature extraction on the sentence vector to learn the dependencies on the observation sequence and improve the accuracy of head entity recognition.

[0008] Step 3: Head entity information identification; the enhanced vector enhanced by context information in step 2 is input into the head entity information identification module to identify all head entities and their entity types in the sentence. Specifically, by inputting the enhanced vector of step 2 into two identical independent binary classifiers (0 / 1) for decoding, the probability of each token as the start and end position is calculated, and then the probability value is judged to be greater than the preset threshold to determine whether it is an entity boundary. If it is greater than the threshold, it is marked as 1, which is an entity boundary; otherwise, it is marked as 0, which is not an entity boundary. After obtaining the entity boundary, the shared encoding vector is intercepted according to the starting position marked as 1 by the binary classifier to obtain the vector representation of the current head entity. The head entity vector is then input into the entity type sigmoid classifier for type recognition operation to obtain the entity type of the corresponding entity.

[0009] Step 4: Multi-feature fusion; In this step, the head entity related information and the shared encoding vector are fused to obtain the fused features used to mark the tail entities and relations. Specifically, first, according to the entity type obtained in step 3, the corresponding entity type vector is selected from step 1 and the head entity vector intercepted in step 3 is calculated to calculate the average encoding representation, and then the splicing and fusion are performed. In order to enable the sentence to better learn the head entity prior information and reduce the learning difficulty of the model, different attention weights are given to the words in the sentence in this step. By using the attention mechanism, the words in the sentence can more easily capture the global features that have been added with the head entity information, thereby strengthening feature dependence.

[0010] Step 5: Identify tail entities and relations. The fusion vector constructed in Step 4 is used to identify tail entities and relations in the same manner as the head entity in Step 3. The difference is that this step uses a multi-layer binary classifier with the number of layers being the same as the predefined number of relation types. This allows all possible tail entities under each relation to be identified, based on the head entity.

[0011] Compared with the prior art, the present invention has the following beneficial effects:

[0012] 1) By introducing a contextual information enhancement module after the encoding layer to strengthen the sequence information dependency, the key features of the input data can be retained and transferred to obtain a more fine-grained feature representation;

[0013] 2) In the head entity information recognition module, entity type prediction and vector fusion are introduced to strengthen the representation ability of entity information vectors and improve the accuracy of tail entity and relationship recognition;

[0014] 3) An attention mechanism is introduced in the multi-feature fusion layer to calculate the probability that each word in the fusion vector participates in the feature representation of the current task. This allows the model to focus on the important parts of the text related to the task and ignore other content irrelevant to the target task. This enhances the entity expression ability for the recognition of relations and tail entities and improves the model's generalization ability.

[0015] 4) Based on the identification of the head entity, all possible tail entities under each relationship can be identified by adopting the pointer network decoding method, which can more conveniently solve the problem of relationship overlap. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 The overall flow chart of the method designed by the present invention.

[0017] Figure 2 Schematic diagram of the Transformer structure used in the BERT model, which is a pre-trained word vector generation model.

[0018] Figure 3 This is the LSTM model structure diagram.

[0019] Figure 4 This is the BiLSTM model structure diagram. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solutions and features of the present invention more clearly understood, the present invention will be further described in detail below in combination with specific implementation examples and with reference to the accompanying drawings.

[0021] Step 1: Data encoding;

[0022] Traditional text encoding models lack semantic representation capabilities. BERT, proposed by Devlin, is an autoencoding language model that uses a bidirectional language model to obtain word vector representations of input text. This model addresses the shortcomings of distributed word vector representations such as Word2Vec and Glove that cannot effectively obtain high-dimensional features. The BERT model implements bidirectional feature representation of text. It uses large-scale unlabeled corpus for training and learns deep representations by jointly training the left and right contexts of each word, obtaining a representation of text that contains rich semantic information. It also accelerates model training through parallel computing. The Transformer structure of the BERT pre-training model is as follows: Figure 2 shown.

[0023] The present invention uses the pre-trained BERT model to encode the input single sentence x, and the Transformer structure is represented as Trans(x). Define the sentence set x = {x1, x2, ...x n}、Entity type set T={t1,t2,...,t m}, for each sentence x i and each entity type t i The encoding operation is shown in formula (1) and formula (2):

[0024] h0=w token +w Pos (1)

[0025] e=Trans(h i-1 ), i∈[1,N] (2)

[0026] where w token and w Pos They represent the word embedding matrix and position embedding matrix respectively. After n layers of Transformer encoding, N is the maximum length of the word vector. The dimension of the vector representation of a single word is the dimension of the BERT hidden layer output, which defaults to 768. The value output by the last layer is the final result of the text encoding; the output sentence vector represents h x and entity type vector representation h type, define the sentence vector as H = {e1, e2, e3, ..., e n}.

[0027] Step 2: Contextual information enhancement;

[0028] Since the BERT encoder uses a Transformer structure, position-related information is weakened during training, while position information is very important in sequence labeling tasks. Recurrent neural networks (RNNs) are often used in the field of natural language processing to learn serialized nonlinear features. To address the problems of RNNs being unable to capture long-distance information and gradient disappearance, the Long Short Term Memory (LSTM) network has made some improvements to RNNs by introducing memory units and gating mechanisms, achieving better results in relation extraction tasks. The structure of the Long Short Term Memory network mainly consists of an input gate, an output gate, and a forget gate, and this structure is used to achieve long-term memory capabilities. LSTM models such as Figure 3 shown.

[0029] The LSTM model formula is as follows:

[0030] i t =σ(W i ·[h t-1 , x t ]+b i ) (3)

[0031] j t =σ(W j ·[h t-1 , x t ]+b j ) (4)

[0032] k t =σ(W k ·[h t-1 , x t ]+b k ) (5)

[0033]

[0034]

[0035] h t =k t tanh(c t ) (8)

[0036] Where W in formula (3-6) i and b i Represent different weight matrices and biases, x t represents the variable input at time t, ht-1 represents the hidden state at time t-1, h t represents the hidden state of the hidden layer at time t, c t represents the cell state at time t, Represents the candidate cell state at time t. t 、j t 、k t They refer to the input gate, forget gate, and output gate respectively, σ is the sigmoid activation function, and tanh is the hyperbolic tangent activation function.

[0037] A unidirectional LSTM network can only retain relevant information about data in the same direction, but real-world applications often require considering the temporal information before and after a sentence. Therefore, this module uses a bidirectional long short-term memory network (BiLSTM) to perform deep feature extraction on sentence vectors to learn the dependencies on the observation sequence and improve the accuracy of head entity recognition. The structure of BiLSTM is as follows: Figure 4 As shown in , it is composed of two unidirectional networks. During forward or backward propagation, the hidden layer output before the corresponding direction will be retained, and the output vectors of the two directions are finally concatenated as the final output, as shown in formula (9).

[0038]

[0039] in represents the hidden layer output of the forward propagation, is the hidden layer output of the backward propagation.

[0040] Step 3: Header entity information identification;

[0041] The head entity information recognition module aims to identify all head entities and their entity types in a sentence. First, a pointer network is used to detect the output of the context information enhancement module and mark the start and end positions corresponding to the main word. Specifically, the H-encoded vector finally output by BiLSTM is input into two identical independent binary classifiers (0 / 1) for decoding, and the probability of each token being the start and end position is calculated. Then, it is determined whether the probability value is greater than a preset threshold to determine whether it is an entity boundary. If it is greater than the threshold, it is marked as 1, indicating an entity boundary; otherwise, it is marked as 0, indicating that it is not an entity boundary. The following formula (10-11) is the binary classifier calculation formula.

[0042]

[0043]

[0044] where h iis the vector of the i-th word in the input sentence after being processed by the context enhancement module, W(·) and b(·) represent the training weight and bias vector respectively, and is the output probability value of the i-th token vector calculated by the binary classifier. If this value is greater than the preset threshold, the position of the token is marked as 1; otherwise, it is marked as 0. This threshold can be adjusted during model training to ultimately find an optimal threshold. For text containing multiple head entities, this model uses the nearest match principle. That is, for each starting position, it matches the nearest ending position from the right, and identifies the text segment corresponding to the starting and ending positions as a head entity.

[0045] After obtaining the entity boundary, the shared encoding vector h x According to the starting position marked as 1 by the binary classifier, it is intercepted to obtain the vector representation h of the current head entity sub . Set the head entity vector h sub Input into the entity type classifier for type recognition operation. The recognition formula is as follows.

[0046] P type =sigmoid(W type h sub +b type ) (12)

[0047] where h sub is the entity vector intercepted from the identified head entity boundary, P type It is the probability of the predicted type, and the final result is the entity type corresponding to the maximum probability.

[0048] Step 4: Multi-feature fusion;

[0049] To strengthen the dependency between the model's head entity recognition and tail entity recognition, the head entity-related information and the sentence vector are fused in this step to obtain fused features for marking the tail entities and relationships. Previous entity relationship extraction models only used addition and multiplication when fusing relationship features. Although these methods are simple, they destroy the semantics of the original features and blur the focus of the model's learning. Therefore, in order to enable sentences to better learn the head entity prior information and reduce the learning difficulty of the model, different attention weights are assigned to the words in the sentence at this layer. By using the attention mechanism, the words in the sentence can more easily capture the global features that incorporate the head entity information, strengthening feature dependency.

[0050] Specifically, first, the corresponding entity type vector and the head entity vector are selected according to the entity type, and the average encoding representation is calculated, and then fused. Since simply adding the entity vector and the type vector will destroy the original semantic information of the two vectors, a splicing method is used in this step to perform semantic expansion fusion, and finally the output vector h of the module is obtained. cat , calculated as follows.

[0051]

[0052]

[0053] h cat =[h sub' :h type ] (15)

[0054] in Represent the start and end position word vectors corresponding to the k-th head entity, Indicates the i-th word vector representation of the current type, h sub' 、h type 、h cat They are head entity vector, entity type vector and head entity information vector.

[0055] Then, in order to calculate the correlation between each word in the sentence and the head entity information, we will use the scaled dot product method to calculate it, as shown in formula (16-18).

[0056]

[0057] α i =softmax(e i ) (17)

[0058] M=[α i h s :h cat ] (18)

[0059] where α i is the calculated j-th word and head entity information h cat The relevance score is multiplied by the original vector and combined with the head entity information h cat The tail entity-relation feature M is obtained by splicing, which integrates the prior information of the head entity.

[0060] Step 5: Identification of tail entities and relations;

[0061] The tail entity-relationship recognition layer identifies all possible tail entities within each relationship, building upon the identification of the head entity. This step employs a multi-layer binary classifier, with the number of layers equal to the predefined number of relationship types. This layer takes as input the sentence features M that incorporate prior information about the head entity. When decoding M, each detected head entity is simultaneously labeled with a corresponding tail entity. The specific operation is shown in Equations (19-20).

[0062]

[0063]

[0064] in and They represent the probability values ​​of the i-th token in the sentence being the start and end positions of the tail entity under the current head entity information, respectively. W(·) and b(·) represent the training weight and bias vector, respectively.

[0065] Experimental equipment and required environment

[0066] The experiment uses a GTX 2080Ti graphics card to run the code, with 11G video memory, and is conducted on a Linux CentOS platform. Python 3.6 / Keras 2.2.4 / Tensorflow 1.1.14 is used.

[0067] Experimental results and analysis

[0068] The datasets used for experimental evaluation of relation extraction tasks primarily include the New York Times (NYT) dataset and the WebNLG dataset. The NYT dataset, originally a news corpus generated using distant supervision methods, is widely used for overlapping entity relation extraction. It contains 24 predefined relations, with sentences longer than 100 and those without triples filtered out. The final training, test, and validation sets contain 56,195, 5,000, and 5,000 sentences, respectively. Originally created for natural language generation tasks, it has also been adapted for overlapping entity relation extraction. The WebNLG dataset contains 246 predefined relations, with each sentence containing multiple triples. Its training, test, and validation sets contain 5,019, 500, and 703 sentences, respectively. By analyzing and statistically analyzing the data from the NYT and WebNLG datasets, we summarize the number of sentences classified as Normal, SEO, and EPO in each dataset. Table 1 shows that 34.13% of the NYT dataset sentences contain overlapping triples, with EPO sentences accounting for 51% and SEO sentences accounting for 76.8%. 68.2% of the WebNLG dataset contains overlapping triples, of which 0.06% are EPO sentences and 99.5% are SEO sentences.

[0069] The evaluation metrics used follow those used in the Wei et al. paper, assuming that a predicted triple is considered correct only when the head entity, tail entity, and relation are all correct. The extracted relation triplets are evaluated using three metrics: precision (P), recall (R), and F1 score, as shown in Equations (25-27).

[0070]

[0071]

[0072]

[0073] Among them, T P To predict the number of correct triplets, F P is the number of incorrectly predicted triplets, T P +F P is the number of all triples extracted, F N is the number of correct triplets not extracted, T P +F N is the total number of triplets in the dataset. Therefore, precision is the ratio of correctly extracted triplets to the total number of extracted triplets, and recall is defined as the ratio of the number of correct triplets to all triplets in the dataset.

[0074] The first set of experiments compared the proposed model with other related joint extraction models. The NovelTagging, ETL-Span, CopyRE, and CasRel models, which are commonly used in this field, were compared with the proposed MFF-WSE model on the NYT and WebNLG datasets. The experimental results are shown in Table 2.

[0075] Among several comparative extraction models, the CopyRE model uses multiple dynamic decoders to extract relation triplets. Due to the limitations of the copy mechanism, the model has a lower accuracy rate. In addition, due to the inherent limitations of RNN expansion, the generated triples are limited. The NovelTagging model uses a single-layer tagging method, which can only assign one label to each word. It does not take into account that an entity may have different relationships with multiple entities in the context, so it cannot identify overlapping triplets. Moreover, since the model considers all entities belonging to a single relation type, the model has high precision and low recall. The ELT-Span model can extract relation triplets in sentences, and the extraction results on WebNLG have been greatly improved, but it cannot handle the EPO problem. A new pointer tagging framework is constructed on the CasRel model, and the extraction effect reaches the best in the field, but there are problems with exposure bias and error propagation. Compared with the CasRel model, our model's F1 values ​​on the two datasets increased by 1.4% and 0.5%, respectively. This indicates that using contextual features on the original embeddings generated by the pre-trained model to enhance the accuracy of head entity recognition has a certain improvement, further alleviating the error propagation problem and making the model more accurate. By extracting entity information and introducing the attention mechanism for multi-feature fusion, the representation ability of sentence embeddings is enhanced, enabling the model to generate more triplets, ultimately achieving the optimal F1 value for our model.

[0076] The second set of experiments explored the model's extraction performance on sentences of varying complexity. Experiments were conducted on sentences with varying numbers of triples on the NYT and WebNLG datasets. The experimental results are shown in Table 3. As can be seen, the model's F1 score initially increases and then decreases as the number of triples increases. However, the experimental results show that the model's extraction performance on sentences of all five levels of complexity is superior to the baseline models CasRel and ETL-Span. These results demonstrate that this chapter's model can more effectively model and extract triples from sentences containing multiple triples, demonstrating stronger generalization capabilities.

[0077] To further explore the model's extraction performance across different overlapping triple types, the third set of experiments used the NYT and WebNLG datasets to classify sentences into three categories: Normal, EPO, and SEO. The F1 scores of the extraction performance of this model, the baseline CasRel model, and the ETL-Span model were compared on these three sentence types. The experimental results are shown in Table 4. These results demonstrate that the model achieves optimal extraction performance for different triple types, demonstrating its ability to more effectively address the overlapping triple problem.

[0078] Table 1 shows the analysis and statistics of the data in the NYT and WebNLG datasets.

[0079] Table 2 shows the comparison of the precision (P), recall (R), and F1 value of the joint extraction model with other related entity relations on the NYT and WebNLG datasets.

[0080] Table 3 compares the F1 values ​​of sentences with different numbers of triples on the NYT and WebNLG datasets.

[0081] Table 4 compares the F1 value of the extraction performance on sentences with different overlapping types on the NYT and WebNLG datasets.

[0082] Table 1 Experimental data statistics

[0083]

[0084] Table 2 Comparison experiment with existing extraction models

[0085]

[0086] Table 3 Experimental results on sentences with different numbers of triples

[0087]

[0088] Table 4 Experimental results on sentences with different overlapping types

[0089]

Claims

1. A method for entity relationship joint extraction based on semantic enhancement and multi-feature fusion, characterized by: The steps include: Step 1: Data encoding: Input the data to be used for the entity relationship extraction task into the pre-trained model to obtain the sentence shared encoding vector matrix and entity type vector representation of the data; Step 2: Contextual information enhancement: The shared encoding vector matrix obtained in step 1 is input into the contextual information enhancement module. A bidirectional long short-term memory (BiLSTM) network is used to retain and transfer key features in the input data through a gating mechanism. This allows for deep feature extraction of sentence vectors, learning dependencies on observation sequences and improving the accuracy of head entity recognition. Step 3: Head entity information recognition: The enhanced vector enhanced with contextual information in step 2 is input into the head entity information recognition module to identify all head entities and their entity types in the sentence; Step 4: Multi-feature fusion: fuse the head entity related information and the shared encoding vector to obtain the fused features used to mark the tail entities and relations; Step 5: Identification of tail entities and relations; The fusion vector constructed in step 4 is used to identify the tail entities and relations. A multi-layer binary classifier is used, with the number of layers being the predefined number of relation types. This allows all possible tail entities under each relation to be identified based on the head entity. In step 3, the enhanced vector from step 2 is input into two identical independent binary classifiers 0 / 1 for decoding, and the probability of each token being the start and end position is calculated. Then, the probability value is judged to be greater than a preset threshold to determine whether it is an entity boundary. If it is greater than the threshold, it is marked as 1, indicating an entity boundary; otherwise, it is marked as 0, indicating that it is not an entity boundary. After obtaining the entity boundary, the shared encoding vector is intercepted according to the starting position marked as 1 by the binary classifier to obtain the vector representation of the current head entity. The head entity vector is then input into the entity type sigmoid classifier for type recognition to obtain the entity type of the corresponding entity. In step 4, first, based on the entity type obtained in step 3, the corresponding entity type vector is selected from step 1 and the head entity vector extracted in step 3 to calculate the average encoding representation, and then concatenate and fuse them. Different attention weights are assigned to the words in the sentence. By using the attention mechanism, the words in the sentence can more easily capture the global features that have been added with the head entity information, thereby strengthening feature dependence.

Images