A knowledge-enhanced drug entity relation joint extraction method and system

Abstract

The application discloses a kind of based on knowledge enhancement's drug entity relationship joint extraction method, comprising: obtaining drug entity interaction relationship dataset, the drug entity interaction relationship dataset is preprocessed, to obtain the drug entity interaction relationship dataset after preprocessing, for each drug entity in the drug entity interaction relationship dataset after preprocessing, the entity description information and interaction relationship subgraph information corresponding to the drug entity are obtained, entity description information and interaction relationship subgraph information are preprocessed, to obtain entity description feature and interaction relationship subgraph feature, the drug entity interaction relationship dataset after preprocessing and its external knowledge information are input into pre-trained entity relationship joint extraction model, to obtain the final drug entity relationship extraction result.The application can solve the technical problem that the method using conventional deep learning model is not suitable for this special field of biomedical.

Description

Technical Field

[0001] This invention relates to the field of drug entity relation joint extraction technology, and more specifically, to a method and system for drug entity relation joint extraction based on knowledge enhancement. Background Technology

[0002] With the rapid development of deep learning, more and more people are applying its methods to the biomedical field. Furthermore, the rapid development of information technology has led to an explosive growth in biomedical texts, which contain richer biomedical information. Among these, drug-drug interactions (DDIs) are one of the most closely watched pieces of information. Obtaining DDIs directly from literature allows for faster and more direct access to the latest information, necessitating faster and better drug entity relationship extraction technologies, such as entity recognition and relationship extraction.

[0003] Existing methods for extracting information from DDIs are mainly implemented in the following two ways: The first is to use traditional deep learning models to learn text features, which usually include parts of speech, sentence components, syntactic analysis, etc. By learning these features, each drug entity pair is classified and DDI relationships are predicted; The second is to add some information from external knowledge bases to the deep learning model to help the model obtain more accurate results.

[0004] However, both of the aforementioned existing methods for extracting drug entity relationships have some significant technical problems. The first method relies excessively on the structural information of the sentence itself, failing to fully integrate the valuable information from the biomedical knowledge base, making it unsuitable for the specific field of biomedicine. The second method, on the other hand, fails to adequately combine entity description information and drug-drug interaction subgraph information from the knowledge base, easily leading to knowledge interference and omissions of relevant knowledge, resulting in inaccurate extraction results. Summary of the Invention

[0005] To address the aforementioned deficiencies or improvement needs of existing technologies, this invention provides a method and system for joint extraction of drug entity relationships based on knowledge enhancement. Its purpose is to solve the technical problem that existing methods using traditional deep learning models rely excessively on the structural information of the sentences themselves, failing to fully integrate the beneficial information in the biomedical knowledge base, thus making the extraction method unsuitable for the specific field of biomedicine. Furthermore, existing methods that add external knowledge base information to deep learning models fail to fully integrate entity description information and drug interaction subgraph information in the knowledge base, easily leading to knowledge interference and omission of relevant knowledge, resulting in inaccurate extraction results.

[0006] To achieve the above objectives, according to one aspect of the present invention, a method for joint extraction of drug entity relationships based on knowledge enhancement is provided, comprising the following steps:

[0007] (1) Obtain the drug entity interaction relationship dataset and preprocess the drug entity interaction relationship dataset to obtain the preprocessed drug entity interaction relationship dataset.

[0008] (2) For each drug entity in the drug entity interaction relationship dataset after preprocessing in step (1), obtain the entity description information and interaction relationship subgraph information corresponding to the drug entity, preprocess the entity description information and interaction relationship subgraph information to obtain entity description features and interaction relationship subgraph features. The entity description features and interaction relationship subgraph features constitute the external knowledge information corresponding to the drug entity. All external knowledge information corresponding to the drug entities constitutes the external knowledge information of the drug entity interaction relationship dataset.

[0009] (3) Input the preprocessed drug entity interaction relationship dataset obtained in step (1) and the external knowledge information of the drug entity interaction relationship dataset obtained in step (2) into the pre-trained entity relationship joint extraction model to obtain the final drug entity relationship extraction result.

[0010] Preferably, step (1) specifically involves first executing the instruction xml.dom.minidom.parse to obtain a Document Object Model (DOM) tree, and then obtaining the content of the DDIExtaction 2013 dataset based on the DOM tree; then, obtaining the interaction relationships of multiple drug entities based on the obtained content of the DDIExtaction 2013 dataset; then, further obtaining multiple biomedical text sentences, the positions of drug entities in each biomedical text sentence, the interaction relationship type between any pair of drug entities, and the drug entities contained in each biomedical text sentence based on the obtained interaction relationship of multiple drug entities; finally, obtaining the drug interaction relationship triplet for each pair of drug entities based on the interaction relationship type between each pair of drug entities and the drug entities contained in each biomedical text sentence, and the drug interaction relationship triplet for all drug entities constitutes the preprocessed drug entity interaction relationship dataset.

[0011] Preferably, step (2) specifically involves: first, using the DrugBank knowledge base to obtain the entity description information and interaction relationship subgraph information corresponding to each drug entity; then, for each drug entity, using the BioBERT-Basev1.0 pre-trained word vector model to process the entity description information corresponding to the drug entity to obtain the entity description features corresponding to the drug entity; then, for each drug entity, using the GCN model to process the interaction relationship subgraph information corresponding to the drug entity to obtain the interaction relationship subgraph features corresponding to the drug entity; finally, for each drug entity, obtaining the external knowledge information corresponding to the drug entity based on its corresponding entity description features and interaction relationship subgraph features. The external knowledge information corresponding to all drug entities constitutes the external knowledge information of the drug entity interaction relationship dataset.

[0012] Preferably, the process of obtaining entity description information and interaction relationship subgraph features corresponding to each drug entity using the DrugBank knowledge base specifically involves using the xml.dom.minidom.parse command to obtain the DOM tree of the DrugBank knowledge base, obtaining the content of DrugBank based on the DOM tree, and then obtaining the relevant information of the drug entities in the DrugBank knowledge base. This relevant information includes the entity description, synonyms, near-synonyms, and interaction relationships with other drug entities of the drug entities; then, based on the drug entities in the drug entity interaction relationship dataset preprocessed in step (1), the relevant information of the drug entities in the DrugBank knowledge base is synonymously matched; and then, based on the relevant information of the matched drug entities in the DrugBank knowledge base, the entity description information and interaction relationship subgraph information are obtained.

[0013] The process of obtaining the interaction subgraph features corresponding to each drug entity using the DrugBank knowledge base specifically involves using K-means clustering to map the interaction relationships of drug entities in the DrugBank knowledge base to the corresponding four relationships (Advice, Int, Effect, and Mechanism) in the DDIExtaction 2013 dataset, thereby obtaining the interaction subgraph information of drug entities in the drug entity interaction relationship dataset; finally, GCN is used to transform the interaction subgraph information of drug entities in the drug entity interaction relationship dataset into interaction subgraph features.

[0014] Preferably, the entity relationship joint extraction model includes a mask attention module, two identical BERT models, and a feature fusion module.

[0015] The input to the mask attention module is a mask matrix of size n·n and an interaction relationship subgraph feature of size n·h1 obtained in step (2). The output is the knowledge-masked interaction relationship subgraph feature of size n·h1. The mask matrix is ​​designed based on the positional relationship of the drug entity in the biomedical text sentence and is used to mask the influence of the interaction relationship subgraph of the drug entity on other drug entities in order to obtain the knowledge-masked interaction relationship subgraph feature. n represents the length of the biomedical text sentence in the drug entity interaction relationship dataset after preprocessing in step (1), and h1 represents the vector dimension of the interaction relationship subgraph feature.

[0016] Each BERT model uses a 12-layer Transformer encoder. The first BERT model takes as input the drug entity interaction dataset obtained in step (1) and all entity description features corresponding to all drug entities in all biomedical text sentences obtained in step (2). The second BERT model takes as input the drug entity interaction dataset obtained in step (1) and the interaction subgraph features after knowledge masking obtained by the mask attention module. The output of the first BERT model is an N·h2-dimensional feature vector that integrates entity description features, and the output of the second BERT model is an N·h2-dimensional feature vector that integrates interaction subgraph features. N represents the total number of biomedical text sentences extracted in step (1), and h2 represents the dimension of the feature vector output by each BERT model.

[0017] The feature fusion module fuses the N·h2-dimensional feature vector from the first BERT model output, which incorporates entity description features, and the N·h2-dimensional feature vector from the second BERT model output, which incorporates interaction relationship subgraph features, to obtain the prediction model result.

[0018] Preferably, the specific network structure of the feature fusion module is as follows:

[0019] The first layer is the feature concatenation layer. Its inputs are two feature vectors of size N·h2 that integrate entity description features and feature vectors of size N·h2 that integrate interaction relationship subgraph features, respectively. The output is a feature vector of size N·(2*h2) that concatenates these two different feature vectors.

[0020] The second layer is a fully connected layer. Its input is a feature vector of size N·(2*h2) that is a concatenation of two different features from the output of the first layer. This layer uses the sigmoid activation function and outputs the entity relationship prediction result.

[0021] Preferably, the entity relation joint extraction model is trained through the following steps:

[0022] (3-1) Obtain the drug entity interaction relationship dataset, preprocess the drug entity interaction relationship dataset to obtain the preprocessed drug entity interaction relationship dataset, for each drug entity in the preprocessed drug entity interaction relationship dataset, obtain the entity description information and interaction relationship subgraph information corresponding to the drug entity, preprocess the entity description information and interaction relationship subgraph information to obtain entity description features and interaction relationship subgraph features, the entity description features and interaction relationship subgraph features constitute the external knowledge information corresponding to the drug entity, the external knowledge information corresponding to all drug entities constitutes the external knowledge information of the drug entity interaction relationship dataset, merge the drug entity interaction relationship dataset and its external knowledge information, and randomly divide the merged result into training set, validation set and test set in a ratio of 8:1:1.

[0023] (3-2) Initialize the weight parameters of the BERT model to obtain the initialized BERT model;

[0024] (3-3) Input the training set obtained in step (3-1) into the BERT model initialized in step (3-2) for training to obtain the output value that integrates entity description features and interaction relationship subgraph features.

[0025] (3-4) The output values ​​obtained in step (3-3) are concatenated for features and input into the feature fusion module to predict the final output result P of the i-th biomedical text sentence at the j-th position in the training set partitioned in step (3-1) of the entity relation joint extraction model. ij , where i∈[1,N], j∈[1,n], N represents the number of biomedical text sentences in the training set, and n represents the maximum length of each biomedical text sentence;

[0026] (3-5) The final output y of the i-th biomedical text sentence at the j-th position in the training set partitioned in step (3-1) according to step (3-4). ij We obtain the predicted value of entity e at the j-th position of the i-th biomedical text sentence in the training set partitioned in step (3-1). And the predicted value of the j-th positional relation r of the i-th biomedical text sentence in the training set partitioned in step (3-1). according to Calculate the loss function of the joint entity-relation extraction model, and use this loss function to iteratively train the joint entity-relation extraction model until the loss function of the joint entity-relation extraction model reaches its minimum value.

[0027] (3-6) Use the validation set divided in step (3-1) to iteratively validate the entity relation joint extraction model after iterative training until the classification accuracy reaches the optimal level, thereby obtaining the trained entity relation joint extraction model.

[0028] Preferably, the calculation formula for step (3-4) is:

[0029]

[0030]

[0031] in This represents the final output of the i-th biomedical text sentence at the j-th position in the training set partitioned by step (3-1), which incorporates entity description features d. d represents the entity description features corresponding to each drug entity obtained in step (3-3). This represents the final output of the i-th biomedical text sentence at the j-th position in the training set partitioned by step (3-1), which incorporates the interaction relation subgraph feature k; where k represents the interaction relation subgraph feature corresponding to each drug entity obtained in step (3-3); p ij This represents the fused feature at the j-th position of the i-th biomedical text sentence in the training set partitioned in step (3-1), after feature concatenation of the output values ​​obtained in step (3-3). ij This represents the final output of the i-th biomedical text sentence at the j-th position in the training set partitioned in step (3-1), and Concatenate represents the concatenation function.

[0032] Preferably, the loss function Loss of the entity relation joint extraction model is:

[0033]

[0034] Loss e The cross-entropy between the predicted value and the true label of an entity is specifically expressed as:

[0035]

[0036] in This represents the predicted value of entity e at the j-th position of the i-th biomedical text sentence in the training set partitioned in step (3-1). This represents the true label of entity e at the j-th position of the i-th biomedical text sentence in the training set partitioned in step (3-1);

[0037] Loss r The cross-entropy between the predicted relation value and the true label is specifically expressed as:

[0038]

[0039] in This represents the predicted value of the j-th positional relation r of the i-th biomedical text sentence in the training set partitioned in step (3-1). Let r represent the true label of the j-th positional relation of the i-th biomedical text sentence in the training set partitioned in step (3-1).

[0040] According to another aspect of the present invention, a knowledge-enhanced drug entity relation joint extraction system is provided, comprising:

[0041] The first module is used to obtain a drug entity interaction relationship dataset and preprocess the drug entity interaction relationship dataset to obtain a preprocessed drug entity interaction relationship dataset.

[0042] The second module is used to obtain the entity description information and interaction subgraph information corresponding to each drug entity in the drug entity interaction relationship dataset preprocessed by the first module, and to preprocess the entity description information and interaction subgraph information to obtain entity description features and interaction subgraph features. The entity description features and interaction subgraph features constitute the external knowledge information corresponding to the drug entity. The external knowledge information corresponding to all drug entities constitutes the external knowledge information of the drug entity interaction relationship dataset.

[0043] The third module is used to input the preprocessed drug entity interaction relationship dataset obtained from the first module and the external knowledge information of the drug entity interaction relationship dataset obtained from the second module into the pre-trained entity relationship joint extraction model to obtain the final drug entity relationship extraction result.

[0044] In summary, compared with the prior art, the above-described technical solutions conceived by this invention can achieve the following beneficial effects:

[0045] (1) Since the present invention adopts steps (3-2) to (3-3), it uses the BERT model as the basis to fully learn the features, entity description features, and interaction relationship subgraph features in biomedical text sentences. Therefore, it can solve the technical problem that the existing first method is not suitable for the special field of biomedicine because the model relies too much on the structural information of the sentence itself and fails to fully combine the beneficial information in the biomedical knowledge base.

[0046] (2) Since the present invention adopts step (3-4), it uses a feature fusion module to fuse entity description features and interaction relationship subgraph features at the end of the model and then predicts the results, avoiding the mutual influence between different features and causing knowledge interference. Therefore, it can solve the technical problem that the existing second method is prone to knowledge interference and omission of some relevant knowledge due to the failure to fully combine entity description information and drug interaction subgraph interaction information in the knowledge base, resulting in inaccurate extraction results.

[0047] (3) Since the present invention adopts step (3), it uses the entity relationship joint extraction model to perform the two tasks of entity recognition and relationship extraction at the same time, making full use of entity information, thus having the advantage of reducing entity redundancy. Attached Figure Description

[0048] Figure 1 This is a flowchart of the knowledge-enhanced drug entity relationship joint extraction method of the present invention;

[0049] Figure 2 This is a framework diagram of the entity relationship joint extraction model used in the method of this invention. Detailed Implementation

[0050] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0051] The basic idea of ​​this invention is to provide a knowledge-enhanced drug entity relation joint extraction method and system. It uses a drug entity interaction relation dataset to compare the performance of the knowledge-enhanced drug entity relation joint extraction method with other methods. It uses the DrugBank knowledge base as an external knowledge base, providing entity description information and interaction subgraph information for each drug entity, thus enhancing model performance. The entity relation joint extraction model fully considers entity information and reduces entity redundancy. Finally, a feature fusion module fuses entity description features and interaction subgraph features, reducing knowledge noise and further enhancing model performance.

[0052] like Figure 1 As shown, this invention provides a method for joint extraction of drug entity relationships based on knowledge enhancement, comprising the following steps:

[0053] (1) Obtain the drug entity interaction relationship dataset and preprocess the drug entity interaction relationship dataset to obtain the preprocessed drug entity interaction relationship dataset.

[0054] Specifically, the drug entity interaction dataset obtained in this invention is the DDIExtaction 2013 dataset, which is often used to train and evaluate the performance of drug interaction extraction models. This dataset is a semantic annotation corpus for drug interactions, with all annotated texts sourced from DrugBank documents and MedLine article summaries. The dataset contains four relation types: Advice, Int, Effect, and Mechanism. This invention requires extracting drug entity interactions from biomedical text; therefore, the DDIExtaction 2013 dataset needs to be preprocessed to remove irrelevant information, extracting only the biomedical text sentences, the locations of drug entities within each sentence, the interaction type between each pair of drug entities, and the drug entities contained in the biomedical text sentence, resulting in the preprocessed drug entity interaction dataset.

[0055] More specifically, since the downloaded original DDIExtaction 2013 dataset is in XML format, the model cannot directly read the file to obtain its content. Therefore, this step first executes the command `xml.dom.minidom.parse` to obtain a Document Object Model (DOM) tree, and then retrieves the content of the DDIExtaction 2013 dataset based on this DOM tree. Next, based on the retrieved content of the DDIExtaction 2013 dataset, the interaction relationships of multiple drug entities are obtained. Then, based on the obtained interaction relationships of multiple drug entities, multiple biomedical text sentences, the positions of drug entities in each biomedical text sentence, the interaction relationship type between any pair of drug entities, and the drug entities contained in each biomedical text sentence are further obtained. Finally, based on the interaction relationship type between each pair of drug entities and the drug entities contained in each biomedical text sentence, drug interaction relationship triples for each pair of drug entities are obtained. The drug interaction relationship triples of all drug entities constitute the preprocessed drug entity interaction relationship dataset.

[0056] The advantage of this step is that it processes each pair of drug entities and the interaction relationship between them into a drug interaction triplet, which increases the correlation between drug entities and interaction relationships.

[0057] (2) For each drug entity in the drug entity interaction relationship dataset after preprocessing in step (1), obtain the entity description information and interaction relationship subgraph information corresponding to the drug entity, preprocess the entity description information and interaction relationship subgraph information to obtain entity description features and interaction relationship subgraph features. The entity description features and interaction relationship subgraph features constitute the external knowledge information corresponding to the drug entity. All external knowledge information corresponding to the drug entities constitutes the external knowledge information of the drug entity interaction relationship dataset.

[0058] Specifically, this step involves the following steps: First, using the DrugBank knowledge base, we obtain the entity description information and interaction subgraph information corresponding to each drug entity. Then, for each drug entity, we process the entity description information using a BioBERT-Basev1.0 pre-trained word vector model to obtain the entity description features. Next, for each drug entity, we process the interaction subgraph information using a GCN model to obtain the interaction subgraph features. Finally, for each drug entity, we obtain the external knowledge information corresponding to the drug entity based on its entity description features and interaction subgraph features. All the external knowledge information corresponding to the drug entities constitutes the external knowledge information of the drug entity interaction dataset.

[0059] The process of obtaining entity description information and interaction relationship subgraph features corresponding to each drug entity using the DrugBank knowledge base is as follows: Since the data in the DrugBank knowledge base is stored in XML format, this step also uses the xml.dom.minidom.parse command to obtain the DOM tree of the DrugBank knowledge base. Based on the DOM tree, the content of DrugBank is obtained, and then the relevant information of the drug entities in the DrugBank knowledge base is obtained. This relevant information includes the entity description, synonyms, near-synonyms, and interaction relationships with other drug entities of the drug entities. Then, based on the drug entities in the drug entity interaction relationship dataset preprocessed in step (1), the relevant information of the drug entities in the DrugBank knowledge base is synonymously matched. Then, based on the relevant information of the matched drug entities in the DrugBank knowledge base, the entity description information and interaction relationship subgraph information are obtained.

[0060] The process of obtaining the interaction subgraph features corresponding to each drug entity using the DrugBank knowledge base is as follows: Since the interaction relationship of drug entities in the DrugBank knowledge base is described in a sentence, this step uses the K-means clustering method to map the interaction relationship of drug entities in the DrugBank knowledge base to the four relationships Advice, Int, Effect, and Mechanism in the DDIExtaction 2013 dataset, thus obtaining the interaction subgraph information of drug entities in the drug entity interaction relationship dataset; finally, GCN is used to convert the interaction subgraph information of drug entities in the drug entity interaction relationship dataset into interaction subgraph features.

[0061] The advantage of this step is that it utilizes the DrugBank knowledge base to provide entity description information and interaction relationship subgraph information, thus providing valuable information from biomedical knowledge bases.

[0062] (3) Input the preprocessed drug entity interaction relationship dataset obtained in step (1) and the external knowledge information of the drug entity interaction relationship dataset obtained in step (2) into the pre-trained entity relationship joint extraction model to obtain the final drug entity relationship extraction result.

[0063] like Figure 2 As shown, the entity relationship joint extraction model of the present invention includes a mask attention module, two identical BERT models, and a feature fusion module.

[0064] The input to the mask attention module is a mask matrix of size n·n and an interaction relationship subgraph feature of size n·h1 obtained in step (2). The output is the knowledge-masked interaction relationship subgraph feature of size n·h1. The mask matrix is ​​designed based on the positional relationship of the drug entity in the biomedical text sentence and is used to mask the influence of the interaction relationship subgraph of the drug entity on other drug entities in order to obtain the knowledge-masked interaction relationship subgraph feature. n represents the length of the biomedical text sentence in the drug entity interaction relationship dataset after preprocessing in step (1), and h1 represents the vector dimension of the interaction relationship subgraph feature.

[0065] Each BERT model uses a 12-layer Transformer encoder. The first BERT model takes as input the drug entity interaction dataset obtained in step (1) and all entity description features corresponding to all drug entities in all biomedical text sentences obtained in step (2). The second BERT model takes as input the drug entity interaction dataset obtained in step (1) and the knowledge-masked interaction subgraph features obtained by the mask attention module. The output of the first BERT model is an N·h2-dimensional feature vector that integrates entity description features, and the output of the second BERT model is an N·h2-dimensional feature vector that integrates interaction subgraph features. N represents the total number of biomedical text sentences extracted in step (1), and h2 represents the dimension of the feature vector output by each BERT model. Here, h2 is usually taken as 128.

[0066] The feature fusion module fuses the N·h2-dimensional feature vector from the first BERT model output, which incorporates entity description features, and the N·h2-dimensional feature vector from the second BERT model output, which incorporates interaction relationship subgraph features, to obtain the prediction model result.

[0067] For the feature fusion module, its specific network structure is as follows:

[0068] The first layer is the feature concatenation layer. Its inputs are two feature vectors of size N·h2 that integrate entity description features and feature vectors of size N·h2 that integrate interaction relationship subgraph features, respectively. The output is a feature vector of size N·(2*h2) that concatenates these two different feature vectors.

[0069] The second layer is a fully connected layer. Its input is a feature vector of size N·(2*h2) that is a concatenation of two different features from the output of the first layer. This layer uses the sigmoid activation function and outputs the entity relationship prediction result.

[0070] Specifically, the entity relation joint extraction model in this invention is trained through the following steps:

[0071] (3-1) Obtain the drug entity interaction relationship dataset, preprocess the drug entity interaction relationship dataset to obtain the preprocessed drug entity interaction relationship dataset, for each drug entity in the preprocessed drug entity interaction relationship dataset, obtain the entity description information and interaction relationship subgraph information corresponding to the drug entity, preprocess the entity description information and interaction relationship subgraph information to obtain entity description features and interaction relationship subgraph features, the entity description features and interaction relationship subgraph features constitute the external knowledge information corresponding to the drug entity, the external knowledge information corresponding to all drug entities constitutes the external knowledge information of the drug entity interaction relationship dataset, merge the drug entity interaction relationship dataset and its external knowledge information, and randomly divide the merged result into training set, validation set and test set in a ratio of 8:1:1.

[0072] Specifically, the drug entity interaction dataset used in this step is the DDIExtaction 2013 dataset, which includes four types of relationships: Int, Advice, Effect, and Mechanism. The original DDIExtaction 2013 dataset is divided into a training set of 77% and a test set of 23%. The training set contains 572 articles annotated in DrugBank and 147 abstracts in Medline, while the test set contains 158 articles annotated in DrugBank and 33 abstracts in Medline.

[0073] It should be noted that the processing procedure in this step is exactly the same as the corresponding procedures in steps (1) and (2) above, and will not be repeated here.

[0074] (3-2) Initialize the weight parameters of the BERT model to obtain the initialized BERT model;

[0075] Specifically, the initial values ​​of the weight parameters of the BERT model are obtained by pre-training using the paper published in PubMed and PMC, BioBERT-Base v1.0, with an initial learning rate of lr = 0.00005. In order to accelerate the decrease of the loss value, this step uses Adam-Optimizer as the optimizer, because this optimizer has the advantages of momentum and adaptive learning rate.

[0076] (3-3) Input the training set obtained in step (3-1) into the BERT model initialized in step (3-2) for training to obtain the output value that integrates entity description features and interaction relationship subgraph features.

[0077] (3-4) The output values ​​obtained in step (3-3) are concatenated for features and input into the feature fusion module to predict the final output result P of the i-th biomedical text sentence at the j-th position in the training set partitioned in step (3-1) of the entity relation joint extraction model. ij , where i∈[1,N], j∈[1,n], N represents the number of biomedical text sentences (i.e. samples) in the training set, and n represents the maximum length of each biomedical text sentence;

[0078] Specifically, the calculation formula for this step is:

[0079]

[0080]

[0081] in This represents the final output of the i-th biomedical text sentence at the j-th position in the training set partitioned by step (3-1), which incorporates entity description features d. d represents the entity description features corresponding to each drug entity obtained in step (3-3). This represents the final output of the i-th biomedical text sentence at the j-th position in the training set partitioned by step (3-1), which incorporates the interaction relation subgraph feature k; where k represents the interaction relation subgraph feature corresponding to each drug entity obtained in step (3-3); p ij This represents the fused feature at the j-th position of the i-th biomedical text sentence in the training set partitioned in step (3-1), after feature concatenation of the output values ​​obtained in step (3-3). ij This represents the final output of the i-th biomedical text sentence at the j-th position in the training set partitioned in step (3-1), and Concatenate represents the concatenation function.

[0082] The advantage of steps (3-3) to (3-4) above is that they integrate external knowledge information while avoiding interference between different knowledge, thus making better use of the knowledge in the knowledge base.

[0083] (3-5) The final output y of the i-th biomedical text sentence at the j-th position in the training set partitioned in step (3-1) according to step (3-4). ij We obtain the predicted value of entity e at the j-th position of the i-th biomedical text sentence in the training set partitioned in step (3-1). And the predicted value of the j-th positional relation r of the i-th biomedical text sentence in the training set partitioned in step (3-1). according to Calculate the loss function of the joint entity-relation extraction model, and use this loss function to iteratively train the joint entity-relation extraction model until the loss function of the joint entity-relation extraction model reaches its minimum value.

[0084] Specifically, the loss function of the entity-relation joint extraction model is obtained by averaging the cross-entropy loss between the entity prediction and the true label, and the cross-entropy loss between the relation prediction and the true label.

[0085] The cross-entropy between the predicted entity value and the true label is specifically expressed as:

[0086]

[0087] in This represents the predicted value of entity e at the j-th position of the i-th biomedical text sentence in the training set partitioned in step (3-1). This represents the true label of entity e at the j-th position of the i-th biomedical text sentence in the training set partitioned in step (3-1).

[0088] The cross-entropy between the predicted relation value and the true label is specifically expressed as:

[0089]

[0090] in This represents the predicted value of the j-th positional relation r of the i-th biomedical text sentence in the training set partitioned in step (3-1). Let r represent the true label of the j-th positional relation of the i-th biomedical text sentence in the training set partitioned in step (3-1).

[0091] The loss function (Loss) for the entity relation joint extraction model is:

[0092]

[0093] (3-6) Use the validation set divided in step (3-1) to iteratively validate the entity relation joint extraction model after iterative training until the classification accuracy reaches the optimal level, thereby obtaining the trained entity relation joint extraction model.

[0094] The advantages of steps (3-2) to (3-5) above are that BioBERT-Base v1.0 is used as the BERT model initialization parameter, which speeds up the model parameter fitting and quickly obtains the final model result; and the mask attention module is used to mask the influence of the interaction relationship subgraph of the drug entity on other drug entities.

[0095] Experimental results

[0096] This invention provides a performance evaluation of the knowledge-enhanced drug entity relationship joint extraction system. In this embodiment, evaluation experiments were conducted on the drug entity interaction relationships in the DDIExtaction2013 dataset. In these experiments, Predict, Recall, and F1 scores were used as performance metrics, and a 5-fold cross-validation comparison experiment was performed with seven other models. These seven methods are: CNN-based, SCNN, SDP RNN, JointAB-LSTM, BERE, SciBERT CNN, and 3DGT-DDI.

[0097] The specific experimental parameter settings provided by this invention are shown in Table 1 below.

[0098] Table 1 Hyperparameter settings for experimental execution

[0099] parameter set up Batch size 8 Learning rate 1e-5 Vector dimension size 128

[0100] The performance test results of the method of the present invention on the drug entity interaction relationship dataset are shown in Table 2 below.

[0101] Table 2. Performance test results of the method of the present invention on the drug-entity interaction dataset.

[0102]

[0103]

[0104] As can be seen from the experimental results in Table 2 above, the knowledge-enhanced drug entity relationship joint extraction system provided by this invention outperforms other comparative methods in the test performance on the drug entity interaction relationship dataset. Specifically, compared with all comparative methods, the method of this invention has performance improvements of at least 3.05% and 2.51% in Predict and F1 scores, respectively, achieving a high standard of classification effect.

[0105] Compared with related technologies, this invention proposes a knowledge-enhanced drug entity relation joint extraction method and system. The method inputs biomedical text sentences from the drug entity interaction relation dataset, the drug entity positions within each biomedical text sentence, the interaction relation type between each pair of drug entities, and the entity description information and interaction subgraph information of each drug entity contained in the biomedical text sentence into the drug entity relation joint extraction model to learn the drug entity features and interaction relation information between drug entities. Furthermore, by comparing several well-known machine learning models for drug interaction relation extraction, the knowledge-enhanced drug entity relation joint extraction method and system provided by this invention outperforms other comparative models on the drug entity interaction relation dataset, achieving high stability and high accuracy. The knowledge-enhanced drug entity relation joint extraction method and system provided by this invention also have the advantages of high accuracy and short training time.

[0106] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A knowledge-enhanced method for joint extraction of drug entity relationships, characterized in that, The method comprises the following steps: (1) obtaining a drug entity interaction relationship data set, preprocessing the drug entity interaction relationship data set to obtain a preprocessed drug entity interaction relationship data set; (2) for each drug entity in the preprocessed drug entity interaction relationship data set of step (1), obtaining the entity description information and the interaction relationship subgraph information corresponding to the drug entity, preprocessing the entity description information and the interaction relationship subgraph information to obtain entity description features and interaction relationship subgraph features, the entity description features and the interaction relationship subgraph features forming external knowledge information corresponding to the drug entity, and all the external knowledge information corresponding to the drug entities forming external knowledge information of the drug entity interaction relationship data set; (3) inputting the preprocessed drug entity interaction relationship data set obtained in step (1) and the external knowledge information of the drug entity interaction relationship data set obtained in step (2) into a pre-trained entity relationship joint extraction model to obtain a final drug entity relationship extraction result; the entity relationship joint extraction model comprises a mask attention module, two identical BERT models, and a feature fusion module; The input of the mask attention module is a mask matrix with a size of n*n and the interaction relationship subgraph features with a size of n*h1 obtained in step (2), and the output is the knowledge masked interaction relationship subgraph features with a size of n*h1, wherein the mask matrix is designed according to the positional relationship of the drug entity in the biomedical text sentence, and is used to mask the influence of the interaction relationship subgraph of the drug entity on other drug entities to obtain the knowledge masked interaction relationship subgraph features, n represents the length of the biomedical text sentence in the preprocessed drug entity interaction relationship data set of step (1), and h1 represents the vector dimension of the interaction relationship subgraph features; Each BERT model uses a 12-layer Transformer encoder, wherein the first BERT model inputs the drug entity interaction relationship data set obtained in step (1) and all the entity description features corresponding to all the drug entities in all the biomedical text sentences obtained in step (2); the second BERT model inputs the drug entity interaction relationship data set obtained in step (1) and the knowledge masked interaction relationship subgraph features obtained by the mask attention module; The output of the first BERT model is a feature vector with a size of N*h2 fused with the entity description features, the output of the second BERT model is a feature vector with a size of N*h2 fused with the interaction relationship subgraph features, N represents the total number of biomedical text sentences extracted in step (1), and h2 represents the feature vector dimension of the output of each BERT model; The feature fusion module fuses the feature vector with a size of N*h2 output by the first BERT model and the feature vector with a size of N*h2 output by the second BERT model to obtain a prediction model result.

2. The knowledge enhancement-based joint extraction method of drug entity relationships according to claim 1, characterized in that, The step (1) is specifically that, first, the instruction xml.dom.minidom.parse is executed to obtain a document object model (DOM) tree, and the content of the DDIExtaction 2013 dataset is obtained according to the DOM tree; then, the interaction relationship of a plurality of drug entities is obtained according to the content of the DDIExtaction 2013 dataset; then, a plurality of biomedical text sentences, the position of a drug entity in each biomedical text sentence, the interaction relationship type between any pair of drug entities in all drug entities, and the drug entity contained in each biomedical text sentence are further obtained according to the interaction relationship of the plurality of drug entities; and finally, the drug interaction relationship triple of each pair of drug entities is obtained according to the interaction relationship type between each pair of drug entities and the drug entity contained in each biomedical text sentence, and the drug interaction relationship triples of all drug entities constitute the preprocessed drug entity interaction relationship dataset.

3. The knowledge enhancement-based joint extraction method of drug entity relationships according to claim 2, characterized in that, The step (2) is specifically that, first, the entity description information and the interaction relationship subgraph information corresponding to each drug entity are obtained using the DrugBank knowledge base; then, for each drug entity, the entity description information corresponding to the drug entity is processed using the BioBERT-Base v1.0 pre-training word vector model to obtain the entity description feature corresponding to the drug entity; then, for each drug entity, the interaction relationship subgraph information corresponding to the drug entity is processed using the GCN model to obtain the interaction relationship subgraph feature corresponding to the drug entity; and finally, for each drug entity, the external knowledge information corresponding to the drug entity is obtained according to the entity description feature and the interaction relationship subgraph feature corresponding to the drug entity, and the external knowledge information corresponding to all drug entities constitutes the external knowledge information of the drug entity interaction relationship dataset.

4. The knowledge-enhanced joint extraction method for drug entity relationships according to claim 3, characterized in that, The process of obtaining the entity description information and the interaction relationship subgraph feature corresponding to each drug entity using the DrugBank knowledge base is specifically that, the xml.dom.minidom.parse instruction is used to obtain the DOM tree of the DrugBank knowledge base, the content of the DrugBank is obtained according to the DOM tree, and the related information of the drug entities in the DrugBank knowledge base is further obtained, which includes the entity description, the synonym, the near-synonym, and the interaction relationship with other drug entities of the drug entity; then, the related information of the drug entities in the DrugBank knowledge base is matched with the drug entities in the preprocessed drug entity interaction relationship dataset in step (1); and then, the entity description information and the interaction relationship subgraph information are obtained according to the matched related information of the drug entities in the DrugBank knowledge base. The process of obtaining the interaction relationship subgraph feature corresponding to each drug entity using the DrugBank knowledge base is as follows: the interaction relationship of the drug entity in the DrugBank knowledge base is mapped to the Advice, Int, Effect, and Mechanism 4 relationships in the DDIExtaction 2013 dataset using the K-means clustering method, and the interaction relationship subgraph information of the drug entity in the drug entity interaction relationship dataset is obtained. Finally, the GCN is used to convert the interaction relationship subgraph information of the drug entity in the drug entity interaction relationship dataset into interaction relationship subgraph features.

5. The knowledge enhancement-based joint extraction method of drug entity relationships according to claim 4, characterized in that, The specific network structure of the feature fusion module is as follows: The first layer is a feature splicing layer, which inputs two BERT model output feature vectors of size N·h2 that fuse entity description features and size N·h2 that fuse interaction relationship subgraph features, and outputs an N·(2*h2) feature vector that splices the two different feature vectors; The second layer is a fully connected layer, which inputs the N·(2*h2) feature vector spliced with two different features output by the first layer, uses a sigmoid activation function, and outputs an entity relationship prediction result.

6. The knowledge enhancement-based joint extraction method of drug entity relationships according to claim 1, characterized in that, The entity relationship joint extraction model is trained by the following steps: (3-1) Obtain the drug entity interaction relationship dataset, preprocess the drug entity interaction relationship dataset to obtain the preprocessed drug entity interaction relationship dataset, obtain the entity description information and interaction relationship subgraph information corresponding to each drug entity in the preprocessed drug entity interaction relationship dataset, preprocess the entity description information and interaction relationship subgraph information to obtain the entity description feature and interaction relationship subgraph feature, the entity description feature and interaction relationship subgraph feature constitute the external knowledge information corresponding to the drug entity, and all drug entity corresponding external knowledge information constitutes the external knowledge information of the drug entity interaction relationship dataset. The drug entity interaction relationship dataset and its external knowledge information are combined, and the combined result is randomly divided into a training set, a validation set, and a test set according to a ratio of 8:1:1; (3-2) initialize the weight parameters of the Bert model to obtain the initialized Bert model; (3-3) input the training set divided in step (3-1) into the BERT model initialized in step (3-2) for training to obtain output values that fuse entity description features and interaction relationship subgraph features; (3-4) The output value obtained in step (3-3) is subjected to feature splicing and input into a feature fusion module to predict the final output result P of the jth position of the ith biomedical text sentence in the training set divided in step (3-1) of the entity relationship joint extraction model ij wherein , N represents the number of biomedical text sentences in the training set, and n represents the maximum length of each biomedical text sentence. (3-5) the final output result of the jth position of the ith biomedical text sentence in the training set divided in step (3-1) obtained according to step (3-4) , to obtain the prediction value of the jth position entity e of the ith biomedical text sentence in the training set divided in step (3-1) , and the jth position relationship r prediction value of the ith biomedical text sentence in the training set divided in step (3-1) , according to , , the loss function of the entity relationship joint extraction model is calculated, and the entity relationship joint extraction model is iteratively trained using the loss function until the loss function of the entity relationship joint extraction model reaches a minimum value. (3-6) use the validation set divided in step (3-1) to iteratively validate the iteratively trained entity relationship joint extraction model until the classification accuracy reaches the optimal value, thereby obtaining the trained entity relationship joint extraction model.

7. The knowledge-enhanced drug entity relationship joint extraction method according to claim 6, wherein the calculation formula of step (3-4) is: ​ ， wherein represents the final output result of the jth position of the ith biomedical text sentence in the training set divided in step (3-1) fused with the entity description feature d, represents the entity description feature corresponding to each drug entity obtained in step (3-3); represents the final output result of the jth position of the ith biomedical text sentence in the training set divided in step (3-1) fused with the interaction relationship subgraph feature k, represents the interaction relationship subgraph feature corresponding to each drug entity obtained in step (3-3); represents the fusion feature of the jth position of the ith biomedical text sentence in the training set divided in step (3-1) after feature splicing is performed on the output value obtained in step (3-3), represents the final output result of the jth position of the ith biomedical text sentence in the training set divided in step (3-1), represents a splicing function.

8. The knowledge enhancement-based joint extraction method of drug entity relationships according to claim 7, characterized in that, A loss function Loss of the entity relation joint extraction model is: ， where Loss e represents the cross-entropy between the entity prediction and the true label, which is specifically represented as: ， wherein represents the predicted value of the jth position entity e of the ith biomedical text sentence in the training set divided in step (3-1), represents the true label of the jth position entity e of the ith biomedical text sentence in the training set divided in step (3-1). Loss r denotes the cross-entropy between the relation prediction and the true label, which is specifically expressed as: ， wherein represents the jth position relation r prediction value of the ith biomedical text sentence in the training set divided in step (3-1), represents the jth position relation r true label of the ith biomedical text sentence in the training set divided in step (3-1).

9. A knowledge-enhanced drug entity relation joint extraction system based on the knowledge-enhanced drug entity relation joint extraction method of any one of claims 1 to 8. Comprise: The first module is used for obtaining a drug entity interaction relation data set, preprocessing the drug entity interaction relation data set to obtain a preprocessed drug entity interaction relation data set; The second module is used for obtaining, for each drug entity in the preprocessed drug entity interaction relation data set of the first module, entity description information and interaction relation subgraph information corresponding to the drug entity, preprocessing the entity description information and the interaction relation subgraph information to obtain entity description features and interaction relation subgraph features, the entity description features and the interaction relation subgraph features constituting external knowledge information corresponding to the drug entity, and all the external knowledge information of the drug entities constituting external knowledge information of the drug entity interaction relation data set; The third module is used for inputting the preprocessed drug entity interaction relation data set obtained by the first module and the external knowledge information of the drug entity interaction relation data set obtained by the second module into a pre-trained entity relation joint extraction model to obtain a final drug entity relation extraction result.

Images