An entity relationship extraction method based on hybrid pruning and attention graph convolutional network

Abstract

The present invention discloses an entity relationship extraction method based on hybrid pruning and attention graph convolutional network. In order to learn the rich semantic information representing the entity relationship in the dependency syntax tree corresponding to the input sentence, a hybrid pruning strategy of the dependency syntax tree combining "hard pruning" and "soft pruning" is designed to learn the adjacency matrix corresponding to the entity dependency relationship. Then, based on the densely connected network and combined with the idea of the RGCN network, each hidden layer in the network is modeled to guide the generation of neural implicit representations of the input sentence and the key text entities therein. Finally, the feedforward neural network is applied to the embedded representation formed after the sentence and entity are spliced to perform the text entity relationship classification task. The experimental results on the PubMed dataset show that the proposed HP-GCN model can improve the quality of entity relationship extraction in text and achieve advanced performance.

Description

Technical Field

[0001] The present invention belongs to the field of natural language processing, and specifically relates to the design of an entity relationship extraction method based on a hybrid pruning strategy and an attention graph convolutional network. Background Art

[0002] Text entity relationship extraction is an important research direction in the field of natural language processing. Its main task is to identify and extract the semantic relationship between two given entities in the text.<entity1,relation,entity2> Triples have extensive application value in the practical application of natural language processing; text entity relationship extraction is mainly divided into sentence-level relationship extraction, document-level relationship extraction, and corpus-level relationship extraction. The present invention relates to a text entity relationship extraction method that captures the semantic and structural information in text sentences through a dependency syntactic tree structure, and then encodes this through a graph convolutional network (GCN) with excellent representation learning capabilities to achieve text sentence-level entity relationship extraction.

[0003] Zhang et al. incorporated pruned dependency trees into GCN for relation extraction. Reference: Zhang Y, Qi P, Manning C D. Graph convolution over pruned dependency trees improves relation extraction[J].arXiv preprint arXiv:1809.10185, 2018. While pruned trees can effectively process long sentences, they inevitably introduce noise and information loss. This inspired us to explore more effective pruning strategies from the perspectives of pruning rules and deep neural network weight learning, while further optimizing the structural design of the GCN network. Wang et al. introduced the Multi-Head Graph Attention Network (MultiGAT), which automatically assigns weights to edges based on the importance between two nodes. Multiple substructures learn different feature information separately, but it does not effectively consider the relationship transformations between related target entity pairs, making the model lack the ability to model different relationship types. Reference: Wang E, Wang F, Yang Z, et al. Extracting biomedical relations via a multi-head attention based graph convolutional network[C] / / 2020 IEEE International Conference on Bioinformatics and Biomedicine (BIBM). IEEE, 2020: 793-798. In recent years, with the emergence of pre-trained language models such as BERT (Bidirectional Encoder Representation from Transformers) and its variants, the paradigm of combining pre-training with fine-tuning has brought many natural language processing tasks to a new stage. Lee et al. used biomedical data from PubMed to train BERT, obtaining the BioBERT model suitable for the biomedical field and achieving state-of-the-art results on multiple corpora.References: Lee J, Yoon W, Kim S, et al. BioBERT: a pre-trained biomedical language representation model for biomedical text mining [J]. Bioinformatics, 2020, 36 (4): 1234-1240. Su et al. used BioBERT for word representation and used the attention mechanism to further extract global semantic features from the output of the last layer of BioBERT. References: Su P, Vijay-Shanker K. Investigation of bert model on biomedical relation extraction based on revised fine-tuning mechanism [C] / / 2020 IEEE International Conference on Bioinformatics and Biomedicine (BIBM). IEEE, 2020: 2522-2529. Although the above works have made some progress, they are not good at simultaneously modeling the semantic and global structural information related to the target relationship in long sentences. Summary of the Invention

[0004] The purpose of the present invention is to solve the problems faced by existing methods of integrating dependency syntactic tree structures into graph convolutional networks to realize text entity relationship extraction, such as excessive pruning, difficulty in effectively grasping the structural information of long sentences, automatically assigning weights to the dependency relationships between different words, and enabling multiple substructures to learn the interactive information of different embedding representation subspaces respectively. An entity relationship extraction method based on a hybrid pruning strategy and an attention graph convolutional network is proposed.

[0005] The technical solution of the present invention is: an entity relationship extraction method based on a hybrid pruning strategy and an attention graph convolutional network, comprising the following steps:

[0006] S1. Based on the input text sentence x, a "hard pruning" strategy based on the dependency syntax tree is designed. The three dependency structures of shortest dependency path (SDP), lowest common ancestor (LCA subtree), and pruned tree are comprehensively considered to obtain the key nodes and their dependency relationships in the original dependency syntax tree corresponding to x, and the above tree structure is converted into a hard dependency adjacency matrix A. hard .

[0007] S2. Based on the input text sentence x, we explore the "soft pruning" strategy based on the dependency syntax tree. We use the attention-driven graph convolutional network to automatically learn the weight representations of all dependency relationships in the original dependency syntax tree corresponding to x, and convert the above tree structure into a soft dependency adjacency matrix A. soft .

[0008] S3, by weighting A hard and A soft Combine and learn the dependency adjacency matrix A corresponding to x;

[0009] S4. Based on the densely connected network, define the initial representation of each layer of nodes in the network Further combining the idea of RGCN network, each hidden layer in the network Conduct modeling;

[0010] S5: Linearly combine the representations of N different densely connected layers in the densely connected network and output them, thereby guiding the generation of the neural implicit representation h of the input sentence sent and the neural implicit representation of key entities

[0011] S6. Finally, the feedforward neural network is applied to the embedding representation h formed by concatenating the sentence and the entity fcon , perform entity relationship classification tasks, and use stochastic gradient descent method to solve the HP-GCN model.

[0012] Furthermore, step S1 is specifically as follows:

[0013] In order to learn the rich semantic information of entity relationships in the dependency syntax tree corresponding to the input sentence, a hybrid pruning strategy of the dependency syntax tree combining "hard pruning" and "soft pruning" is designed, and the corresponding adjacency matrix A is constructed.

[0014] Specifically, for the "hard pruning" strategy based on the dependency syntax tree, the idea of Entity Path-centric Pruning will be followed. According to the shortest dependency path (SDP), the nearest common ancestor (LCA subtree) and the pruned tree structures, the common subtree path between the target entity pairs is obtained, and the key nodes and their dependency edges are retained, thereby constructing the corresponding hard dependency adjacency matrix A. hard .

[0015] On the one hand, since the complete dependency syntactic tree corresponding to a sentence x is not essential for entity relationship extraction, excessive dependency paths and dependency relationships can actually hinder relationship extraction performance. On the other hand, practice has shown that most of the key information about a relationship can be obtained through the above structure. Therefore, this heuristic method can effectively remove the redundant information caused by excessive dependency paths in x, while retaining the more representative dependency path information that is closely related to entity relationships.

[0016] Furthermore, step S2 is specifically as follows:

[0017] In the construction of soft dependency adjacency matrix A through attention-driven graph convolutional network softt When , the original dependency syntax tree corresponding to x needs to be converted into a weighted graph with fully connected edges. soft corresponds to a fully connected graph, where each element Represents the dependency edge weight from node i to node j. Represents a fully connected graph G (1) . A soft It can be constructed by using the self-attention mechanism, which has the advantage of being able to capture the interaction between any two positions in a single sequence. Of course, the multi-head attention mechanism is also used to calculate A. soft , which enables the model to give multi-layer attention to the text sentence x from multiple perspectives, focusing on the interactive information from different word embedding representation subspaces. The specific calculation involves a query and a set of key-value pairs. The result value can be obtained by weighting, where the weight can be completed by the query function with the corresponding key. The calculation formula is as follows:

[0018]

[0019] Among them, Q and K both represent the set representation of the l-1th layer in the HP-GCN model. and is the projection parameter matrix, is the adjacency matrix of the t-th attention guide, corresponding to the t-th head. Here, at most N matrices can be constructed, where N is a hyperparameter.

[0020] Furthermore, step S3 is specifically as follows:

[0021] By weighting A hard and A soft Combine and construct the dependency adjacency matrix A corresponding to x. The specific calculation is:

[0022] A=γA hard +(1-γ)A soft (2)

[0023] Among them, γ is the proportional coefficient, which means A hard The proportion in the calculation of A.

[0024] Furthermore, step S4 is specifically as follows:

[0025] In a densely connected network, any layer has direct connections to all previous layers. Mathematically, we first It is defined as the concatenation of the initial node representation and the node representation generated by the 1st, ..., l-1th layers, that is,

[0026]

[0027] In practice, each densely connected layer contains L network sub-layers. The dimension of these network sub-layers is d hidden Depends on L and the input feature dimension d. In the HP-GCN model, set d hidden = d / L. For example, if there are 3 network sublayers in a densely connected network and the input dimension is 300, then the hidden dimension of each network sublayer is: Next, the outputs of each network sublayer are concatenated to form a new representation. Therefore, the output dimension is 300 (3×100). Unlike GCN models, where the latent dimension is greater than or equal to the input dimension, the HP-GCN model shrinks the latent dimension as the number of layers increases, improving the parameter efficiency of networks similar to DenseNets.

[0028] Since there are N different attention-guided adjacency matrices in the HP-GCN model, N independent dense connection layers are required. Furthermore, combining the RGCN network idea, each hidden layer in the network Modeling (for the t-th matrix A (t) ):

[0029]

[0030] Among them, j = 1, ..., N, is used to select the adjacency matrix A guided by attention under the dependency relationship r∈R (t) The column dimension of the weight matrix of each sub-layer is increased by d hidden ,Right now, Among them, d (l) =d+d hidden ×(l-1).

[0031] Furthermore, step S5 is specifically as follows:

[0032] The representations of N different densely connected layers in a densely connected network are connected through a linear combination layer. Formally, the output of the linear combination layer can be defined as:

[0033] h comb =W comb h out +b comb (5)

[0034] Among them, h out is the output of concatenating the outputs of N independent densely connected layers, that is is the weight matrix in the linear transformation, b comb is the bias vector in the linear transformation.

[0035] Furthermore, step S6 is specifically as follows:

[0036] After applying the HP-GCN model on the dependency tree, we can get the implicit representation of all words (tokens). Entity relationship extraction aims to predict the relationship between entities based on these representations. The sentence representation and entity representation are concatenated to get the final representation for classification. First, we need to get the sentence representation h sent , which can be calculated by the following formula:

[0037] h sent =f(h mask )=f(HP-GCN(x)) (6)

[0038] Among them, h mask represents the implicit representation of the masked set. Here, masking refers to selecting only the representations of words in the sentence that are not entity words. Represents a maximum pooling function that can map n output vectors into 1 sentence vector. Similarly, the neural implicit representation of the entity can also be obtained. For the i-th entity, its neural implicit representation is The calculation formula is:

[0039]

[0040] in, Represents the neural implicit representation of the i-th entity. Then, the sentence and entity representations are concatenated to form a new embedding representation.

[0041] Apply a feed-forward neural network (FFNN) to the new concatenated representation, which takes the following form:

[0042]

[0043] Among them, h fcon It will eventually be used as input to the logistic regression classifier to perform text entity relationship classification prediction.

[0044] The stochastic gradient descent algorithm is used to solve the HP-GCN model, and the parameters are continuously updated until convergence stops.

[0045] The beneficial effect of the present invention is that the present invention designs a hybrid pruning strategy of dependency syntax tree that combines "hard pruning" and "soft pruning" in the method of integrating dependency syntax tree structure into entity relationship extraction for graph convolutional network to learn the adjacency matrix corresponding to the entity dependency path, thereby solving the over-pruning problem faced when using dependency syntax tree structure to characterize the rich semantic information of sentences. Based on the densely connected network, the idea of RGCN network is introduced, and the representation of each hidden layer in the above network is explored. The dependency relationship between words in different sentences is automatically assigned weights, so that multiple substructures can learn the interactive information of different embedding representation subspaces respectively, which can more effectively grasp the structural information of long sentences. The feedforward neural network is applied to the embedded representation formed after the sentence and entity are spliced, so as to perform the entity relationship classification task. The experimental results on the PubMed dataset show that the proposed HP-GCN method can improve the quality of entity relationship extraction and achieve advanced performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 This is the overall architecture diagram of the relationship extraction method based on the hybrid pruning strategy and attention graph convolutional network provided by the present invention.

[0047] Figure 2 This is the attention guidance layer diagram in an embodiment of the present invention. DETAILED DESCRIPTION

[0048] The embodiments of the present invention will be further described below with reference to the accompanying drawings.

[0049] The present invention provides an entity relationship extraction method based on a hybrid pruning strategy and an attention graph convolutional network, such as Figure 1 As shown, the following steps are included:

[0050] S1. In order to learn the rich semantic information of entity relationships in the dependency syntax tree corresponding to the input sentence, a hybrid pruning strategy for the dependency syntax tree combining "hard pruning" and "soft pruning" is designed, and the corresponding adjacency matrix A is constructed.

[0051] Specifically, for the "hard pruning" strategy based on the dependency syntax tree, the idea of entity path-centric pruning will be followed. According to the shortest dependency path (SDP), the lowest common ancestor tree (LCA subtree) and the pruned tree structure, the common subtree path between the target entity pairs is obtained, and the key nodes and their dependency edges are retained, thereby constructing the corresponding hard dependency adjacency matrix A. hard .

[0052] On the one hand, since the complete dependency syntactic tree corresponding to a sentence x is not essential for entity relationship extraction, excessive dependency paths and dependency relationships can actually hinder relationship extraction performance. On the other hand, practice has shown that most of the key information about a relationship can be obtained through the above structure. Therefore, this heuristic method can effectively remove the redundant information caused by excessive dependency paths in x, while retaining the more representative dependency path information that is closely related to entity relationships.

[0053] S2. Based on the input text, input sentence x and explore the "soft pruning" strategy based on the dependency syntax tree. The attention-driven graph convolutional network is used to automatically learn the weight representations corresponding to all dependency relationships in the original dependency syntax tree corresponding to x, and convert the above tree structure into a soft dependency adjacency matrix A. soft .

[0054] In the construction of soft dependency adjacency matrix A through attention-driven graph convolutional network soft When , the original dependency syntax tree corresponding to x needs to be converted into a weighted graph with fully connected edges. soft corresponds to a fully connected graph, where each element Represents the dependency edge weight from node i to node j. Represents a fully connected graph G (1) . A soft It can be constructed by using the self-attention mechanism, which has the advantage of being able to capture the interaction between any two positions in a single sequence. Of course, the multi-head attention mechanism is also used to calculate A. soft , which enables the model to give multi-layer attention to the text sentence x from multiple perspectives, focusing on the interactive information from different word embedding representation subspaces. The specific calculation involves a query and a set of key-value pairs. The result value can be obtained by weighting, where the weight can be completed by the query function with the corresponding key. The calculation formula is as follows:

[0055]

[0056] Among them, Q and K both represent the set representation of the l-1th layer in the HP-GCN model. and is the projection parameter matrix, is the adjacency matrix of the t-th attention guide, corresponding to the t-th head. Here, at most N matrices can be constructed, where N is a hyperparameter.

[0057] S3, by weighting A hard and A soft Combine and learn the dependency adjacency matrix A corresponding to x.

[0058] The specific calculation is:

[0059] A=γA hard +(1-γ)A soft (2)

[0060] Among them, γ is the proportional coefficient, which means A hard The proportion in the calculation of A.

[0061] S4. Based on the densely connected network, define the initial representation of each layer of nodes in the network Further combining the idea of RGCN network, each hidden layer in the network Perform modeling.

[0062] Mathematically, first It is defined as the concatenation of the initial node representation and the node representation generated by the 1st, ..., l-1th layers, that is,

[0063]

[0064] In practice, each densely connected layer contains L network sub-layers. The dimension of these network sub-layers is d hidden Depends on L and the input feature dimension d. In the HP-GCN model, set d hidden = d / L. For example, if there are 3 network sublayers in a densely connected network and the input dimension is 300, then the hidden dimension of each network sublayer is: Next, the outputs of each network sublayer are concatenated to form a new representation. Therefore, the output dimension is 300 (3×100). Unlike GCN models, where the latent dimension is greater than or equal to the input dimension, the HP-GCN model shrinks the latent dimension as the number of layers increases, improving the parameter efficiency of networks similar to DenseNets.

[0065] Since there are N different attention-guided adjacency matrices in the HP-GCN model, N independent dense connection layers are required. Furthermore, combining the RGCN network idea, each hidden layer in the network Modeling (for the t-th matrix A (t) ):

[0066]

[0067] Among them, j = 1, ..., N, is used to select the adjacency matrix A guided by attention under the dependency relationship r∈R (t) The column dimension of the weight matrix of each sub-layer is increased by d hidden ,Right now, Among them, d (l) =d+d hidden ×(l-1).

[0068] S5: Linearly combine the representations of N different densely connected layers in the densely connected network and output them, thereby guiding the generation of the neural implicit representation h of the input sentence sent and the neural implicit representation of key entities

[0069] Formally, the output of the linear combination layer can be defined as:

[0070] h comb =W comb h out +b comb (5)

[0071] Among them, h out is the output of concatenating the outputs of N independent densely connected layers, that is is the weight matrix in the linear transformation, b comb is the bias vector in the linear transformation.

[0072] S6. Apply the feedforward neural network to the embedding representation h formed by concatenating sentences and entities fcon , perform entity relationship classification tasks, and use stochastic gradient descent method to solve the HP-GCN model.

[0073] After applying the HP-GCN model on the dependency tree, implicit representations of all words (tokens) are obtained. Entity relation extraction aims to predict the relationships between entities based on these representations.

[0074] The sentence representation and entity representation are concatenated to obtain the final representation for classification. First, the sentence representation h sent , which can be calculated by the following formula:

[0075] h sent =f(h maxk )=f(HP-GCN(x)) (6)

[0076] Among them, h mask represents the implicit representation of the masked set. Here, masking refers to selecting only the word representations that are not entity words in the sentence. Represents a maximum pooling function that can map n output vectors into 1 sentence vector. Similarly, the neural implicit representation of the entity can also be obtained. For the i-th entity, its neural implicit representation is The calculation formula is:

[0077]

[0078] in, Represents the neural implicit representation of the i-th entity. Then, the sentence and entity representations are concatenated to form a new embedding representation.

[0079] Apply a feed-forward neural network (FNN) to the new concatenated representation, which is as follows:

[0080]

[0081] Among them, h fcon It will eventually be used as input to the logistic regression classifier to perform entity relationship classification prediction.

[0082] The stochastic gradient descent algorithm is used to solve the HP-GCN model, and the parameters are continuously updated until convergence stops.

[0083] Compared with existing entity relationship extraction algorithms, this invention has greatly improved the accuracy, as shown in the following:

[0084] Experiments were conducted on the PubMed dataset and compared with six existing advanced entity relationship extraction algorithms, including Feature-Based, SPTree, Graph LSTM-EMBED, Graph LSTM-FULL, Bidir DAG LSTM, and GSGLSTM. The reported performance results are shown in the following table, where T and B represent the drug-gene-mutation interaction entity relationship triple knowledge and drug-mutation interaction entity relationship bigram knowledge, respectively. Single and Cross represent the reported entity relationship extraction accuracy on single-sentence and cross-sentence instances, respectively. Compared with the comparison method, the accuracy test results of the present invention have been significantly improved. As shown in the following table:

[0085]

[0086] Those skilled in the art will appreciate that the embodiments described herein are intended to help readers understand the principles of the present invention, and it should be understood that the scope of protection of the present invention is not limited to such specific descriptions and embodiments. Those skilled in the art can make various other specific variations and combinations based on the technical teachings disclosed in the present invention without departing from the essence of the present invention, and such variations and combinations are still within the scope of protection of the present invention.

Claims

1. A method for entity relationship extraction based on hybrid pruning and attention graph convolutional network, characterized by: The following steps are involved: S1. Based on the input text sentence x, a "hard pruning" strategy based on the dependency syntax tree is designed. For the "hard pruning" strategy based on the dependency syntax tree, the heuristic pruning idea centered on the key entity path is followed. According to the three structures of shortest dependency path, nearest common ancestor and pruned tree, the common subtree path between the target entity pairs is obtained, and the key nodes and their dependency edges are retained, thereby constructing the corresponding hard dependency adjacency matrix A. hard ; S2. Based on the input text sentence x, we explore the "soft pruning" strategy based on the dependency syntax tree. We use the attention-driven graph convolutional network to automatically learn the weight representations of all dependency relationships in the original dependency syntax tree corresponding to x, and convert the above tree structure into a soft dependency adjacency matrix A. soft ; In the construction of soft dependency adjacency matrix A through attention-driven graph convolutional network soft When , the original dependency syntax tree corresponding to x is converted into a weighted graph with fully connected edges; each A soft corresponds to a fully connected graph, where each element Represents the dependency edge weight from node i to node j; Represents a fully connected graph G (1) ; A soft It can be constructed by using the self-attention mechanism, which has the advantage of being able to capture the interaction between any two positions in a single sequence; of course, by also using the multi-head attention mechanism to calculate A soft , which enables the model to give multi-layer attention to the text sentence x from multiple perspectives, and jointly focus on the interactive information from different word embedding representation subspaces; the specific calculation includes a query and a set of key-value pairs, and the result value can be obtained by weighting, where the weight can be completed by the query function with the corresponding key. The calculation formula is as follows: Among them, Q and K both represent the set representation of the l-1 layer in the HP-GCN model; and is the projection parameter matrix, is the adjacency matrix of the t-th attention guide, corresponding to the t-th head; here, at most N matrices can be constructed, where N is a hyperparameter; S3, by weighting A hard and A soft Combine and learn the dependency adjacency matrix A corresponding to x; S4. Based on the densely connected network, define the initial representation of each layer of nodes in the network Further combining the idea of RGCN network, each hidden layer in the network Conduct modeling; S5: Linearly combine the representations of N different densely connected layers in the densely connected network and output them, thereby guiding the generation of the neural implicit representation h of the input sentence sent and the neural implicit representation of key entities S6. Finally, the feedforward neural network is applied to the embedding representation h formed by concatenating the sentence and the entity fcon , perform entity relationship classification tasks, and use stochastic gradient descent method to solve the HP-GCN model.

2. The entity relationship extraction method based on hybrid pruning and attention graph convolutional network according to claim 1 is characterized in that: The step S3 is specifically as follows: By weighting A hard and A soft Combine and construct the dependency adjacency matrix A corresponding to x. The specific calculation is: A=γA hard +(1-γ)A soft Among them, γ is the proportional coefficient, which means A hard The proportion in the calculation of A.

3. The entity relationship extraction method based on hybrid pruning and attention graph convolutional network according to claim 2 is characterized in that: The step S4 is specifically as follows: In a densely connected network, any layer has direct connections to all previous layers. Mathematically, first It is defined as the concatenation of the initial node representation and the node representation generated by the 1st, ..., l-1th layers, that is, In practice, each densely connected layer contains L network sub-layers with dimensions d hidden Depends on L and the input feature dimension d. In the HP-GCN model, set d hidden = d / L; Then, the output of each network sub-layer is concatenated to form a new representation; Therefore, the output dimension is 300. Unlike the GCN model where the latent dimension is greater than or equal to the input dimension, the HP-GCN model reduces the latent dimension as the number of layers increases to improve the parameter efficiency of the DenseNets-like network; Since there are N different attention-guided adjacency matrices in the HP-GCN model, N independent dense connection layers are required; further combining the RGCN network idea, each hidden layer h in the network i (l) For modeling, for the t-th matrix A (t) : Among them, j = 1, ..., N, is used to select the adjacency matrix A guided by attention under the dependency relationship r∈R (t) Related weight matrices and bias terms; the column dimension of the weight matrix of each sub-layer is increased by d hidden ,Right now Among them, d (l) =d+d hidden ×(l-1).

4. The entity relationship extraction method based on hybrid pruning and attention graph convolutional network according to claim 3 is characterized in that: The step S5 is specifically as follows: The representations of N different densely connected layers in a densely connected network are connected through a linear combination layer. Formally, the output of the linear combination layer can be defined as: h comb =W comb h out +b comb Among them, h out is the output of concatenating the outputs of N independent densely connected layers, that is is the weight matrix in the linear transformation, b comb is the bias vector in the linear transformation.

5. The entity relationship extraction method based on hybrid pruning and attention graph convolutional network according to claim 4 is characterized in that: The step S6 is specifically as follows: After applying the HP-GCN model on the dependency tree, implicit representations of all words can be obtained. Entity relationship extraction aims to predict the relationship between entities based on these representations. The sentence representation and entity representation are concatenated to obtain the final representation for classification. First, we need to obtain the sentence representation h sent , which can be calculated by the following formula: h sent =f(h mask )=f(HP-GCN(x)) Among them, h mask represents the implicit representation of the masked set, where masking refers to selecting only the word representations that are not entity words in the sentence. represents a maximum pooling function that can map n output vectors into 1 sentence vector; for the i-th entity, its neural implicit representation The calculation formula is: in, Represent the neural implicit representation of the i-th entity. Then, concatenate the sentence and entity representations to form a new embedding representation; Applying a feedforward neural network to the new concatenated representation takes the following form: Among them, h fcon This will eventually be used as input to a logistic regression classifier to perform entity relationship classification predictions; The stochastic gradient descent algorithm is used to solve the HP-GCN model, and the parameters are continuously updated until convergence stops.

Images