A method and device for extracting relations based on entity pairs and shortest dependency paths
By combining the information of entity pairs with the shortest dependency path, using graph neural network to process medical text, the problem of existing models relying on external information but not internal information is solved, and the accuracy and reliability of medical text classification is improved, and it is suitable for knowledge The construction of the map.
Patent Information
- Application Number
- CN202411304023.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-19
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2044-09-19
AI Technical Summary
Existing models rely too much on external information when processing medical texts and fail to effectively combine internal information and external information, resulting in insufficient prediction accuracy.
The relationship extraction method based on entity pairs and the shortest dependency path is adopted. By obtaining word embeddings, sentence embeddings, and shortest dependency path embeddings, and fusing them into the graph neural network, the entity pair graph is constructed, combining the multi-head attention mechanism and one-dimensional volume The accumulation layer processes the shortest dependency path to obtain more accurate feature information.
It significantly improves the accuracy and reliability of sentence classification prediction, can effectively utilize context information, is suitable for multi-classification problems in medical literature, and has broad application potential.
Smart Images

Figure CN118821779B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of knowledge graph technology, and in particular to a method, device and computer-readable storage medium for extracting relationships based on entity pairs and shortest dependency paths. Background Art
[0002] With the increasing number of biomedical literature, how to better utilize biomedical literature has become a difficult problem. Knowledge graph is a carrier that can display the relationship between different entities. Through knowledge graph, it is convenient for biomedical workers to find the connection between each other. Relationship extraction is an indispensable part of knowledge graph production. Its essence is to extract relevant information between two entities in a sentence and identify the relationship.
[0003] Early relationship extraction tasks mainly relied on rules to extract relationships, and extracted the relationship between entities through text syntax and semantic patterns. However, this method requires manual and continuous updating of expressions, and the constructed rules may only be applicable to the current task. DiMex proposed in the prior art uses regular expressions to extract mutations and genes, and then determines the relationship type through sentence structure. Next, researchers used machine learning methods to automatically identify entities and determine the relationship between entities. The prior art also proposed the use of fully automatic machine learning methods to identify disease-related point mutations from biomedical literature libraries and determine relationships, and added statistical features, distance features, and sentiment features to its relationship recognition model to help machine learning relationships between entities.
[0004] Nowadays, with the emergence of deep learning, more and more researchers have begun to use it to extract relations. Today's deep learning models are mainly divided into two categories: (1) Pre-trained language models that use large-scale text datasets for pre-training to learn the deep semantic and syntactic information of the language. In the prior art, the BioBERT model is proposed based on the BERT model, which is trained using article abstracts from Wikipedia, BooksCorpus, and PubMed. It is also the most widely used medical pre-trained language model. Other researchers have obtained a GPT variant, the BioGPT model, based on GPT-2 and a large amount of biological field-specific data for pre-training. (2) Deep learning models that enrich relational information based on sentence features by adding entity information features, dependency features, and other features. In the prior art, BioBERT is used to enrich entity relationship information using cross-text content between entities. In the prior art, a dependency-driven attention convolutional network is also designed based on the dependency tree, considering local connections and global connections respectively. The former only uses entity dependency words such as head words and child nodes, while the latter considers the connection between entity pairs and uses the shortest dependency path as the connection node. The researchers proposed a cross-sentence n-arg sentence graph LSTM, which aims to transmit dependency information through a long short-term memory network.
[0005] However, when processing medical texts, existing models often only use entity information and ignore the supplement of other dependent information to entity information. They mainly rely on a single source of features and do not consider the connection between entity pairs, which limits the comprehensiveness and accuracy of the model's predictions. Summary of the invention
[0006] To this end, the technical problem to be solved by the present invention is to overcome the problem that the existing model is too dependent on external information, but fails to combine internal information with external information, resulting in insufficient model prediction accuracy.
[0007] In order to solve the above technical problems, the present invention provides a relationship extraction method based on entity pairs and shortest dependency paths, comprising:
[0008] Input multiple sentences into the embedding layer of the relation extraction model to obtain word embedding and sentence embedding of each sentence; obtain the first entity embedding and the second entity embedding of each sentence based on the word embedding, wherein the first entity embedding and the second entity embedding constitute an entity pair; use the sentence embedding as the context representation of each sentence;
[0009] The shortest dependency path of entity pairs in each sentence is obtained and input into the embedding layer of the relation extraction model to obtain the embedding of the shortest dependency path; the embedding of the shortest dependency path is input into the shortest dependency path processing module composed of a multi-head attention mechanism and a one-dimensional convolutional layer to obtain the shortest dependency path representation;
[0010] The first entity embedding, the second entity embedding and the shortest dependency path representation of each sentence are fused to obtain the fused features of each sentence; the fused features of each sentence are fused with the context representation to obtain the sentence semantic information of each sentence;
[0011] The fused features of multiple sentences are input into the graph neural network, and the fused features are used as nodes to construct an entity pair graph to obtain the graph topology information of each sentence.
[0012] The sentence semantic information and graph topology information of each sentence are fused and input into the classification layer, and the predicted classification probability of each sentence is output to classify the sentences.
[0013] Preferably, BioBERT is used as the embedding layer of the relation extraction model.
[0014] Preferably, multiple sentences are input into the embedding layer of the relation extraction model, and the formula for obtaining entity embedding is:
[0015] ;
[0016] in, represents entity embedding, Indicates the position where the entity starts. Indicates the end position of the entity. represents the word embedding output by the embedding layer; It is a linear change layer and a dropout layer.
[0017] Preferably, the process of obtaining the shortest dependency path of entity pairs in each sentence includes:
[0018] The ScispaCy toolkit is used to perform part-of-speech analysis on each sentence after word segmentation to obtain the dependency relationship between words in the sentence and obtain a dependency tree with ROOT as the root node; the breadth-first algorithm is used to find the position of entity pairs in each sentence starting from the root node of the dependency tree, and the shortest dependency path of entity pairs in each sentence is calculated based on the path.
[0019] Preferably, the embedding of the shortest dependency path is input into the shortest dependency path processing module composed of a multi-head attention mechanism and a one-dimensional convolutional layer to obtain the shortest dependency path representation, and the formula is:
[0020] ;
[0021] in, is the shortest dependency path representation, is the embedding of the shortest dependency path, represents the multi-head attention mechanism, Represents a one-dimensional convolutional layer.
[0022] Preferably, the sentence semantic information of each sentence is obtained by fusing the fusion feature of each sentence with the context representation, and the formula is:
[0023] ;
[0024] in, is the sentence semantic information, For context, To fuse features, FCLayer is a linear change layer and a dropout layer. For splicing operation.
[0025] Preferably, the graph neural network is represented as ,in is the node set composed of fusion features, Represents the adjacency matrix; if the entity pairs of two fused features have the same entity, the adjacency matrix The element representing the connection relationship between the two fusion features is set to 1;
[0026] The fused features of multiple sentences are input into the graph neural network, and the fused features are used as nodes to construct an entity pair graph to obtain the graph topology information of each sentence. The formula is:
[0027] ;
[0028] in, is the graph topology information, To fusion features, Represents a graph neural network.
[0029] Preferably, the sentence semantic information and graph topology information of each sentence are fused and input into the classification layer, and the predicted classification probability of each sentence is output to classify the sentences. The formula is:
[0030] ;
[0031] in, represents the target category, represents the predicted probability distribution of each category, Indicates the target category The predicted classification probability, represents the softmax function, represents a linear transformation layer, Indicates noise, is the sentence semantic information, It is the graph topology information.
[0032] The present invention also provides a relationship extraction device based on entity pairs and shortest dependency paths, comprising:
[0033] An embedding module is used to input multiple sentences into an embedding layer of a relation extraction model to obtain a word embedding and a sentence embedding of each sentence; obtain a first entity embedding and a second entity embedding of each sentence based on the word embedding, wherein the first entity embedding and the second entity embedding constitute an entity pair; and use the sentence embedding as a context representation of each sentence;
[0034] The shortest dependency path generation module is used to obtain the shortest dependency path of entity pairs in each sentence and input it into the embedding layer of the relation extraction model to obtain the embedding of the shortest dependency path; the embedding of the shortest dependency path is input into the shortest dependency path processing module composed of a multi-head attention mechanism and a one-dimensional convolutional layer to obtain the shortest dependency path representation;
[0035] A fusion module is used to fuse the first entity embedding, the second entity embedding and the shortest dependency path representation of each sentence to obtain a fusion feature of each sentence; and to fuse the fusion feature of each sentence with the context representation to obtain the sentence semantic information of each sentence;
[0036] The graph neural network module is used to input the fusion features of multiple sentences into the graph neural network, use the fusion features as nodes, construct an entity pair graph, and obtain the graph topology information of each sentence;
[0037] The classification module is used to fuse the sentence semantic information and graph topology information of each sentence and input them into the classification layer, output the predicted classification probability of each sentence, and classify the sentences.
[0038] The present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned method for extracting relationships based on entity pairs and shortest dependency paths are implemented.
[0039] The above technical solution of the present invention has the following beneficial effects compared with the prior art:
[0040] The present invention discloses a method for extracting a relationship based on entity pairs and shortest dependency paths. By extracting the embedded features of the shortest path through the shortest dependency path processing module, more accurate shortest dependency path information can be obtained. The first entity embedding, the second entity embedding and the shortest dependency path representation in each sentence are fused to obtain the fusion features of each sentence. In the graph neural network, the fusion features of each sentence are used as nodes to construct an entity pair graph, and the entity information and the shortest dependency path information are transmitted, which is helpful for learning and utilizing the same entity. The present invention combines the shortest dependency path with the entity pair graph, overcomes the problem that the existing model is too dependent on external information and fails to combine internal information with external information. The entity pair graph enables the entity pairs to learn entity information and context information from each other, and uses the shortest dependency path as a feature, effectively utilizing the context information of the sentence, and then obtaining more effective relationship information from the context information, significantly improving the accuracy and reliability of sentence classification prediction. The present invention has a good effect on the multi-classification problem of medical literature, and has obtained a good classification effect for a variety of medical relationships, which will help the construction of knowledge graphs and has broad development prospects and application potential. BRIEF DESCRIPTION OF THE DRAWINGS
[0041] In order to make the content of the present invention more clearly understood, the present invention is further described in detail below according to specific embodiments of the present invention in conjunction with the accompanying drawings, wherein:
[0042] Figure 1 It is a flow chart of a method for extracting a relationship based on entity pairs and the shortest dependency path of the present invention;
[0043] Figure 2 It is the structural diagram of the relation extraction model;
[0044] Figure 3 It is a sentence entity pair instance graph. DETAILED DESCRIPTION
[0045] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it, but the embodiments are not intended to limit the present invention.
[0046] The present invention provides a relationship extraction method based on entity pairs and the shortest dependency paths. It adopts an end-to-end deep learning framework that integrates multiple information. It not only takes into account the dependency relationship between entities, but also considers integrating the topological information of the entity pair graph. First, BioBERT is used as the embedding layer to obtain word embedding, and the sentence is parsed to obtain the shortest dependency path of the entity and the entity information to obtain the fusion feature. On the one hand, the fusion feature is fused with the context representation to obtain the sentence semantic information. On the other hand, the fusion feature is used to obtain the graph topology information through the graph neural network. Finally, the information obtained by combining the two is predicted through the linear layer. Figure 1 As shown, the specific steps of the present invention include:
[0047] S1. Build a relation extraction model. The structure of the relation extraction model is referenced Figure 2 shown.
[0048] S2. Input multiple sentences into the embedding layer of the relation extraction model to obtain word embedding and sentence embedding of each sentence; obtain the first entity embedding and the second entity embedding of each sentence based on the word embedding, wherein the first entity embedding and the second entity embedding constitute an entity pair; and use the sentence embedding as the context representation of each sentence.
[0049] Preferably, this embodiment uses BioBERT as the embedding layer. Med-7 and Bio-Epidemiology-NER are mainly trained on medical cases, while BioBERT is trained using 4.5 billion words from PubMed document abstracts and 13.5 billion words from PubMed Central document full texts, which meets the requirements of the present invention: extracting relations from medical literature.
[0050] The output of the embedding layer includes sentence embeddings and word embedding , where sentence embedding is the context representation.
[0051] In order to obtain more accurate entity semantic features, the entity position is specially encoded, and the entity average embedding method is used to represent the entity embedding. The formula is:
[0052] ;
[0053] in, represents entity embedding, Indicates the position where the entity starts. Indicates the end position of the entity. Represents word embedding; FCLayer is a linear change layer and a dropout layer, which are used to control the output size of entity embedding. The dropout layer is used to prevent overfitting.
[0054] Input multiple sentences into the embedding layer of the relation extraction model to obtain the first entity embedding of each sentence , second entity embedding and context representation , the first entity embedding and the second entity embedding constitute an entity pair.
[0055] S3. After preprocessing the sentences, the shortest dependency path of the entity pairs in each sentence is obtained; the shortest dependency path of the entity pairs in each sentence is input into the embedding layer of the relation extraction model to obtain the embedding of the shortest dependency path; the embedding of the shortest dependency path is input into the shortest dependency path processing module composed of a multi-head attention mechanism and a one-dimensional convolutional layer to obtain the shortest dependency path representation.
[0056] Specifically, S3 includes the following steps:
[0057] S301, obtaining the shortest dependency path of entity pairs in each sentence, including:
[0058] The ScispaCy toolkit is used to perform part-of-speech analysis on each sentence after word segmentation to obtain the dependency relationship between words in the sentence and obtain a dependency tree with ROOT as the root node; the breadth-first algorithm is used to find the position of entity pairs in each sentence starting from the root node of the dependency tree, and the shortest dependency path of entity pairs in each sentence is calculated based on the path.
[0059] The parsing results of the ScispaCy toolkit are the head word, token, category, and child node of each token. The category is used to distinguish the marked entity pairs. When the entity is parsed into multiple tokens, it is convenient to mark all entity positions and their head words and child nodes, and also to find the shortest dependency path between two entities.
[0060] The dependency tree can be used to obtain dependency information related to the entity, and the shortest dependency path between entity pairs in the sentence can be obtained in the subsequent steps.
[0061] When encountering an entity pair across sentences, the root node ROOT is often not 1, which may result in the two entities being unable to find the shortest dependency path. To address this problem, this embodiment selects the head word as an alternative word to be added.
[0062] S302: Input the shortest dependency path of entity pairs in each sentence into the embedding layer of the relation extraction model to obtain the embedding of the shortest dependency path. .
[0063] S303, the embedding of the shortest dependency path is input into the shortest dependency path processing module composed of a multi-head attention mechanism and a one-dimensional convolutional layer to obtain the shortest dependency path representation, the formula is:
[0064] ;
[0065] in, is the shortest dependency path representation, is the embedding of the shortest dependency path, represents the multi-head attention mechanism, Represents a one-dimensional convolutional layer.
[0066] The shortest dependency path processing module composed of a multi-head attention mechanism and a one-dimensional convolutional layer can better capture global and local features compared to the average pooling method. The multi-head attention mechanism can capture global features, calculate the correlation between each position in the sequence and all other positions through the self-attention mechanism, and identify long-distance dependencies. The one-dimensional convolutional layer slides on the sequence data through the convolution kernel, which can identify short-distance patterns and features. Average pooling only performs average operations in local areas and cannot capture long-distance dependencies. Therefore, the shortest dependency path processing module can better process sequence data and has stronger feature extraction and representation capabilities than simple average pooling.
[0067] S4. The first entity embedding, the second entity embedding and the shortest dependency path representation of each sentence are fused to obtain the fused features of each sentence. ; After fusing the fusion features of each sentence with the context representation, the sentence semantic information of each sentence is obtained. The formula is:
[0068] ;
[0069] in, is the sentence semantic information, For context, To fuse features, FCLayer is a linear change layer and a dropout layer. For splicing operation.
[0070] S5. Input the fused features of multiple sentences into the graph neural network, use the fused features as nodes, construct an entity pair graph, and obtain the graph topology information of each sentence.
[0071] The mainstream graph neural network mainly uses words in a single sentence as nodes of the graph, obtains the dependency tree through NLP tools, and applies it to the graph neural network. This embodiment selects the fusion feature composed of entity pairs and shortest dependency path representation as the node information of the graph, and obtains the graph topology information through the entity pair graph.
[0072] Reference Figure 3As shown, there is an entity pair (e1:TREM2; e2:Alzheimer's Disease) in sentence A, an entity pair (e1:R47H; e2:AD(Alzheimer's Disease)) in sentence B, and an entity pair (e1:R47H; e2:TREM2) in sentence C. It can be clearly seen that there are common entities between each two sentences A, B, and C, and it can be inferred from the entity pair relationship in sentences A and B that the entity in C also has a certain relationship with them. In this embodiment, each entity pair is regarded as a node, and other information about the entity of this entity pair can be obtained through the graph neural network.
[0073] The graph neural network is represented as ,in is the node set composed of fusion features, and the edge Represents entity pair and entity pairs If there are the same entities between the entity pairs of two fused features, the adjacency matrix The elements representing the connection between the two fused features are set to 1.
[0074] Using the graph attention network as the graph network, the fused features of multiple sentences are input into the graph neural network, and the fused features are used as nodes to construct the entity pair graph to obtain the graph topology information of each sentence. The formula is:
[0075] ;
[0076] in, is the graph topology information, To fusion features, Represents a graph neural network.
[0077] The graph topology information can be used to obtain the contextual information of entities in different sentences and their dependency information, which will be more helpful for the model to classify relationships.
[0078] S6. The sentence semantic information and graph topology information of each sentence are fused and input into the classification layer, and the predicted classification probability of each sentence is output to classify the sentences. The formula is:
[0079] ;
[0080] in, represents the target category, represents the predicted probability distribution of each category, Indicates the target category The predicted classification probability, represents the softmax function, represents a linear transformation layer, Indicates noise.
[0081] In order to verify the effect of the relationship extraction method based on entity pairs and shortest dependency paths provided by the present invention, Table 1 is a comparison between the present invention and other existing models on different data sets.
[0082] Table 1. Comparison of the present invention with other existing models on different data sets
[0083]
[0084] As can be seen from Table 1, the present invention achieves Micro-F1 scores of 96.68% and 96.03% on the S4 and BioRED datasets, respectively, which is a significant improvement over other existing models. The S4 dataset is a dataset of Yang et al., obtained after data preprocessing.
[0085] In summary, the present invention discloses a method for extracting a relationship based on entity pairs and the shortest dependency path, which extracts the embedded features of the shortest path through the shortest dependency path processing module, and can obtain more accurate shortest dependency path information, and fuses the first entity embedding, the second entity embedding and the shortest dependency path representation in each sentence to obtain the fusion features of each sentence, and uses the fusion features of each sentence as nodes in the graph neural network to construct an entity pair graph, and transmit the entity information and the shortest dependency path information, which is helpful for learning and utilizing the same entity. The present invention combines the shortest dependency path with the entity pair graph, overcomes the problem that the existing model is too dependent on external information, but fails to combine internal information with external information, enables the entity pairs to learn entity information and context information from each other through the entity pair graph, and uses the shortest dependency path as a feature, effectively utilizing the context information of the sentence, and then can obtain more effective relationship information from the context information, significantly improving the accuracy and reliability of sentence classification prediction. The present invention has a good effect on the multi-classification problem of medical literature, and has obtained a good classification effect for a variety of medical relationships, which will help the construction of knowledge graphs and has broad development prospects and application potential.
[0086] Based on the above-mentioned method for extracting a relationship based on entity pairs and the shortest dependency path, this embodiment further provides a device for extracting a relationship based on entity pairs and the shortest dependency path, including:
[0087] An embedding module is used to input multiple sentences into an embedding layer of a relation extraction model to obtain a word embedding and a sentence embedding of each sentence; obtain a first entity embedding and a second entity embedding of each sentence based on the word embedding, wherein the first entity embedding and the second entity embedding constitute an entity pair; and use the sentence embedding as a context representation of each sentence;
[0088] The shortest dependency path generation module is used to obtain the shortest dependency path of entity pairs in each sentence after preprocessing the sentences; the shortest dependency path of entity pairs in each sentence is input into the embedding layer of the relation extraction model to obtain the embedding of the shortest dependency path; the embedding of the shortest dependency path is input into the shortest dependency path processing module composed of a multi-head attention mechanism and a one-dimensional convolutional layer to obtain the shortest dependency path representation;
[0089] A fusion module is used to fuse the first entity embedding, the second entity embedding and the shortest dependency path representation of each sentence to obtain a fusion feature of each sentence; and to fuse the fusion feature of each sentence with the context representation to obtain the sentence semantic information of each sentence;
[0090] The graph neural network module is used to input the fusion features of multiple sentences into the graph neural network, use the fusion features as nodes, construct an entity pair graph, and obtain the graph topology information of each sentence; in the graph neural network, if the entity pairs of two fusion features have the same entity, it is set to 1 in the adjacency matrix;
[0091] The classification module is used to fuse the sentence semantic information and graph topology information of each sentence and input them into the classification layer, output the predicted classification probability of each sentence, and classify the sentences.
[0092] This embodiment also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned method for extracting relationships based on entity pairs and shortest dependency paths are implemented.
[0093] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.
[0094] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0095] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0096] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0097] Obviously, the above embodiments are merely examples for the purpose of clear explanation and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived therefrom are still within the scope of protection of the present invention.
Claims
1. A relationship extraction method based on entity pairs and shortest dependency paths, characterized in that: include: Input multiple sentences into the embedding layer of the relation extraction model to obtain word embedding and sentence embedding for each sentence; Based on word embedding, obtain the first entity embedding and the second entity embedding of each sentence, wherein the first entity embedding and the second entity embedding constitute an entity pair; use the sentence embedding as the context representation of each sentence; Among them, multiple sentences are input into the embedding layer of the relation extraction model, and the formula for entity embedding is obtained as follows: ; in, represents entity embedding, Indicates the position where the entity starts. Indicates the end position of the entity. represents the word embedding output by the embedding layer; It is a linear change layer and a dropout layer; The shortest dependency path of entity pairs in each sentence is obtained and input into the embedding layer of the relation extraction model to obtain the embedding of the shortest dependency path; the embedding of the shortest dependency path is input into the shortest dependency path processing module composed of a multi-head attention mechanism and a one-dimensional convolutional layer to obtain the shortest dependency path representation; The first entity embedding, the second entity embedding and the shortest dependency path representation of each sentence are fused to obtain the fused features of each sentence; the fused features of each sentence are fused with the context representation to obtain the sentence semantic information of each sentence; The fused features of multiple sentences are input into the graph neural network, and the fused features are used as nodes to construct an entity pair graph to obtain the graph topology information of each sentence. The sentence semantic information and graph topology information of each sentence are fused and input into the classification layer, and the predicted classification probability of each sentence is output to classify the sentences.
2. According to claim 1, a relationship extraction method based on entity pairs and shortest dependency paths is characterized in that: BioBERT is used as the embedding layer of the relation extraction model.
3. The method for extracting a relationship based on entity pairs and shortest dependency paths according to claim 1, characterized in that: The process of obtaining the shortest dependency path of entity pairs in each sentence includes: The ScispaCy toolkit is used to perform part-of-speech analysis on each sentence after word segmentation to obtain the dependency relationship between words in the sentence and obtain a dependency tree with ROOT as the root node; the breadth-first algorithm is used to find the position of entity pairs in each sentence starting from the root node of the dependency tree, and the shortest dependency path of entity pairs in each sentence is calculated based on the path.
4. The method for extracting a relationship based on entity pairs and shortest dependency paths according to claim 1, characterized in that: The embedding of the shortest dependency path is input into the shortest dependency path processing module composed of a multi-head attention mechanism and a one-dimensional convolutional layer to obtain the shortest dependency path representation, the formula is: ; in, is the shortest dependency path representation, is the embedding of the shortest dependency path, represents the multi-head attention mechanism, Represents a one-dimensional convolutional layer.
5. The method for extracting a relationship based on entity pairs and shortest dependency paths according to claim 1, characterized in that: The sentence semantic information of each sentence is obtained by fusing the fusion features of each sentence with the context representation. The formula is: ; in, is the sentence semantic information, For context, To fuse features, FCLayer is a linear change layer and a dropout layer. For splicing operation.
6. The method for extracting a relationship based on entity pairs and shortest dependency paths according to claim 1, characterized in that: The graph neural network is represented as ,in is the node set composed of fusion features, Represents the adjacency matrix; if the entity pairs of two fused features have the same entity, the adjacency matrix The element representing the connection relationship between the two fusion features is set to 1; The fused features of multiple sentences are input into the graph neural network, and the fused features are used as nodes to construct an entity pair graph to obtain the graph topology information of each sentence. The formula is: ; in, is the graph topology information, To fusion features, Represents a graph neural network.
7. The method for extracting a relationship based on entity pairs and shortest dependency paths according to claim 1, characterized in that: The sentence semantic information and graph topology information of each sentence are fused and input into the classification layer, and the predicted classification probability of each sentence is output to classify the sentences. The formula is: ; in, represents the target category, represents the predicted probability distribution of each category, Indicates the target category The predicted classification probability, represents the softmax function, represents a linear transformation layer, Indicates noise, is the sentence semantic information, It is the graph topology information.
8. A relationship extraction device based on entity pairs and shortest dependency paths, characterized in that: include: The embedding module is used to input multiple sentences into the embedding layer of the relation extraction model to obtain the word embedding and sentence embedding of each sentence; Based on word embedding, obtain the first entity embedding and the second entity embedding of each sentence, wherein the first entity embedding and the second entity embedding constitute an entity pair; use the sentence embedding as the context representation of each sentence; Among them, multiple sentences are input into the embedding layer of the relation extraction model, and the formula for entity embedding is obtained as follows: ; in, represents entity embedding, Indicates the position where the entity starts. Indicates the end position of the entity. represents the word embedding output by the embedding layer; It is a linear change layer and a dropout layer; The shortest dependency path generation module is used to obtain the shortest dependency path of entity pairs in each sentence and input it into the embedding layer of the relation extraction model to obtain the embedding of the shortest dependency path; the embedding of the shortest dependency path is input into the shortest dependency path processing module composed of a multi-head attention mechanism and a one-dimensional convolutional layer to obtain the shortest dependency path representation; A fusion module is used to fuse the first entity embedding, the second entity embedding and the shortest dependency path representation of each sentence to obtain a fusion feature of each sentence; and to fuse the fusion feature of each sentence with the context representation to obtain the sentence semantic information of each sentence; The graph neural network module is used to input the fusion features of multiple sentences into the graph neural network, use the fusion features as nodes, construct an entity pair graph, and obtain the graph topology information of each sentence; in the graph neural network, if the entity pairs of two fusion features have the same entity, it is set to 1 in the adjacency matrix; The classification module is used to fuse the sentence semantic information and graph topology information of each sentence and input them into the classification layer, output the predicted classification probability of each sentence, and classify the sentences.
9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of a method for extracting relationships based on entity pairs and shortest dependency paths as described in any one of claims 1 to 7 are implemented.
Citation Information
Patent Citations
Aspect-level sentiment analysis method and device based on syntax and graph convolutional network
CN114417823A
Biomedical relationship extraction method and device based on multi-head attention and graph convolutional network combined with R-Drop mechanism, and medium
CN114547298A
