Knowledge enhancement neural network model-based adverse drug event detection method and system

Abstract

The invention discloses a drug adverse event detection method and system based on a knowledge enhancement neural network model, and relates to the technical field of natural language processing. According to the method, a pre-training language model BioBERT, a convolutional neural network and a Transform network are combined, and keywords related to biomedical text entities are obtained from an external knowledge base, so that the understanding ability of the model on the biomedical entities and the relationships of the biomedical entities is enhanced. According to the method, entity recognition and relation extraction are synchronously optimized by adopting a combined learning strategy, so that the accuracy of adverse drug reaction detection and the robustness of the model are remarkably improved, and a complex semantic relation can be effectively processed.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and more specifically to a method and system for detecting adverse drug events based on a knowledge-enhanced neural network model. Background Art

[0002] At present, the detection and analysis of adverse drug reactions (ADRs) is a key issue in modern medicine and drug regulation. With the continuous deepening of biomedical research and the rapid growth of medical data, especially text data from electronic health records, drug databases and academic literature, how to efficiently identify and process key medical information in these texts has become a major challenge. The timely discovery and accurate detection of adverse drug reactions are of great significance to drug safety, patient treatment and medical decision-making, and can help reduce drug-related medical risks and ensure patient safety.

[0003] In recent years, the development of deep learning technology has brought new ideas to the detection of adverse drug reactions. Models such as convolutional neural networks (CNN), recurrent neural networks (RNN), and long short-term memory networks (LSTM) can automatically learn complex features in text, capture potential semantic associations, and improve detection accuracy. In addition, pre-trained language models based on self-attention mechanisms (such as BioBERT) have shown excellent performance in processing biomedical texts. These models can capture the complex relationship between drugs and adverse reactions through large-scale pre-training.

[0004] Although the application of deep learning in drug adverse reaction detection has made significant progress, there are still some difficulties. Most existing models focus on the internal feature analysis of text, ignoring the introduction and integration of external knowledge, while the expression of drug adverse reactions is usually highly related to specific medical background and clinical context. Therefore, it is difficult to comprehensively improve the accuracy and robustness of detection by relying solely on text data for detection.

[0005] Therefore, how to effectively combine external knowledge in adverse drug reaction detection to improve the accuracy and robustness of adverse drug reaction detection is an urgent problem that technical personnel in this field need to solve. Summary of the invention

[0006] In view of this, the present invention provides a method and system for adverse drug events detection based on a knowledge-enhanced neural network model, which improves the accuracy and robustness of adverse drug reaction detection by integrating entity description information in an external medical knowledge base and deep learning technology.

[0007] In order to achieve the above object, the present invention adopts the following technical solution:

[0008] A method for detecting adverse drug events based on a knowledge-enhanced neural network model, comprising:

[0009] S1 obtains the biomedical text to be tested and preprocesses it to obtain a part-of-speech tagging tensor, a dependency tensor, and an entity span tensor;

[0010] S2 obtains description information corresponding to the entity in the biomedical text based on an external knowledge base and selects keywords to obtain an entity keyword set;

[0011] S3 processes the biomedical text and the entity keyword set based on a pre-trained language model to obtain a text semantic vector and a keyword embedding vector accordingly;

[0012] S4 is based on the fusion of the part-of-speech tagging tensor, the dependency tensor, the entity span tensor and the text semantic vector, and inputs them into the Transformer network to obtain a global semantic representation of the text;

[0013] S5 inputs the keyword embedding vector into a convolutional neural network to obtain a local feature of the keyword;

[0014] S6 outputs entity relationship joint detection results based on the entity span tensor, the global semantic representation of the text and the local features of the keywords.

[0015] Preferably, the preprocessing in S1 specifically includes:

[0016] Based on the biomedical text, word segmentation and syntactic analysis are performed to extract the part-of-speech tags of the vocabulary and the dependency relationships of the sentences, generate a part-of-speech tag sequence and a dependency relationship sequence, and mark the start and end positions of the entities;

[0017] The part-of-speech tag sequence and the dependency relationship sequence are converted into a part-of-speech tag matrix and the dependency relationship matrix based on word index correspondence;

[0018] Obtain entity span information based on the start and end positions of the entity;

[0019] Based on the part-of-speech tag matrix, the dependency matrix and the entity span information, they are correspondingly converted into the part-of-speech tag tensor, the dependency tensor and the entity span tensor.

[0020] Preferably, the entity keyword set obtained in S2 specifically includes:

[0021] Calculate the TF-IDF score for each word in the description information;

[0022] Sort all words in descending order based on the corresponding TF-IDF scores to obtain a sorting result;

[0023] A preset number of words are selected based on the sorting results as entity keywords to form the entity keyword set.

[0024] Preferably, the TF-IDF score calculation formula is:

[0025]

[0026] Among them, f t,D represents the number of occurrences of word t in the description information D, f t',D represents the total number of occurrences of all words (including t itself) in the description information D, N represents the total number of all relevant documents extracted from the external knowledge base, Z represents the set of documents extracted from the external knowledge base for a specific entity, and |{D'∈Z:t∈D'}| represents the number of documents containing word t.

[0027] Preferably, the text semantic vector and keyword embedding vector are obtained in S3, specifically including:

[0028] The biomedical text is subjected to word embedding processing based on the pre-trained language model to obtain the text semantic vector V:

[0029] V = BioBERT(Tokens);

[0030] Among them, BioBERT represents the pre-trained language model, and Tokens represents the word sequence in the pre-processed biomedical text;

[0031] Based on the pre-trained language model, the entity keyword set is processed by word embedding to obtain the keyword embedding vector K E :

[0032] K E = BioBERT(Keywords);

[0033] Among them, Keywords represents the entity keyword set.

[0034] Preferably, obtaining the global semantic representation of the text in S4 specifically includes:

[0035] Based on the part-of-speech tagging tensor T POS , the dependency tensor T DEP , the entity span tensor T Span Fermented with the text semantic vector V, a comprehensive semantic feature representation F is obtained:

[0036]

[0037] in, Represents feature concatenation operation;

[0038] Based on the comprehensive semantic feature representation F, input into the Transformer network to obtain the global semantic representation H of the text T :

[0039] H T = Transformer(F);

[0040] Among them, Transformer represents the Transformer network.

[0041] Preferably, outputting the entity relationship joint detection result in S6 specifically includes:

[0042] Based on the global semantic representation H of the text T and the local features of the keyword to obtain a joint feature;

[0043] Based on the joint feature and the entity span tensor T Span Perform the maximum pooling operation to obtain the entity features corresponding to each entity;

[0044] Obtaining entity category probability distribution based on the entity features;

[0045] Obtaining a relationship category probability distribution based on any two of the entity features;

[0046] The entity relationship joint detection result is based on the entity category probability distribution and the relationship category probability distribution.

[0047] Preferably, the entity category probability distribution acquisition method is:

[0048] Based on the entity features, the entity features are mapped to the entity category space through a fully connected layer, and the entity category probability distribution is obtained by prediction through a first activation function;

[0049] The method for obtaining the probability distribution of relationship categories is:

[0050] Based on any two of the entity features, concatenation is performed to obtain an entity pair relationship feature;

[0051] Based on the entity pair relationship features being mapped to the entity category space through the fully connected layer, and predicting the relationship category between the two entities through the second activation function, the relationship category probability distribution is obtained.

[0052] Preferably, biomedical texts to be tested are collected from biomedical databases and literature resources, and the texts are parsed using natural language processing tools to output the relationships between drug entities and adverse reaction entities as entity relationship joint detection results.

[0053] A drug adverse event detection system based on a knowledge-enhanced neural network model, comprising: a data processing module, an entity keyword generation module, a pre-trained language model processing module, and a model construction and result output module;

[0054] The data processing module is used to obtain the biomedical text to be detected and perform preprocessing to obtain a part-of-speech tagging tensor, a dependency tensor and an entity span tensor;

[0055] The entity keyword generation module is used to obtain description information corresponding to the entity in the biomedical text based on an external knowledge base and select keywords to obtain an entity keyword set;

[0056] The pre-trained language model processing module is used to process the biomedical text and the entity keyword set based on the pre-trained language model to obtain a text semantic vector and a keyword embedding vector accordingly;

[0057] The model construction and result output module is used to construct a Transformer network and a convolutional neural network, fuse the part-of-speech tagging tensor, the dependency tensor, the entity span tensor and the text semantic vector, and input them into the Transformer network to obtain a global semantic representation of the text; input the keyword embedding vector into the convolutional neural network to obtain local features of the keywords; and output entity relationship joint detection results based on the entity span tensor, the global semantic representation of the text and the local features of the keywords.

[0058] Through the above technical solutions, it can be known that compared with the prior art, the present invention discloses a method and system for detecting adverse drug events based on a knowledge-enhanced neural network model. By combining entity description information in an external medical knowledge base, as well as part-of-speech tags and dependencies in the text, a deep understanding of adverse drug reactions is achieved. Compared with models that do not utilize external knowledge sources, the present invention has stronger knowledge expression capabilities and higher accuracy; the present invention adopts a multi-layer strategy, combining the pre-trained language model BioBERT, Transformer network, and convolutional neural network (CNN). By integrating the global semantic information of the text and the local semantic features of entity keywords, the model can effectively capture complex language patterns and medical background knowledge, greatly improving the accuracy and robustness of adverse drug reaction detection; the present invention can effectively identify the relationship between drug entities and adverse reactions in biomedical texts, and provide reliable technical support for drug safety research and medical decision-making. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0060] Figure 1 A flow chart of a method for detecting adverse drug reactions based on a knowledge-enhanced neural network model provided by the present invention.

[0061] Figure 2 A schematic diagram of the overall framework topology structure of a drug adverse reaction detection method based on a knowledge-enhanced neural network model provided by the present invention.

[0062] Figure 3 A schematic diagram of the structure of the pre-trained language model BioBERT provided by the present invention.

[0063] Figure 4 A topological diagram of the combination of the Transformer network and the convolutional neural network (CNN) provided by the present invention.

[0064] Figure 5 A schematic diagram of the structure of a drug adverse reaction detection system based on a knowledge-enhanced neural network model provided by the present invention. DETAILED DESCRIPTION

[0065] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0066] Example 1

[0067] like Figure 1 As shown, the embodiment of the present invention discloses a method for detecting adverse drug events based on a knowledge-enhanced neural network model, comprising:

[0068] S1 obtains the biomedical text to be tested and preprocesses it to obtain a part-of-speech tagging tensor, a dependency tensor, and an entity span tensor;

[0069] S2 obtains description information corresponding to entities in biomedical texts based on an external knowledge base and selects keywords to obtain an entity keyword set;

[0070] S3 processes biomedical text and entity keyword sets based on a pre-trained language model to obtain text semantic vectors and keyword embedding vectors;

[0071] S4 is based on the fusion of part-of-speech tagging tensor, dependency tensor, entity span tensor and text semantic vector and inputs them into the Transformer network to obtain the global semantic representation of the text;

[0072] S5 inputs the keyword embedding vector into the convolutional neural network to obtain the local features of the keyword;

[0073] S6 outputs entity relationship joint detection results based on entity span tensor, global semantic representation of text and local features of keywords.

[0074] Example 2

[0075] like Figure 2 As shown, the embodiment of the present invention discloses a method for detecting adverse drug events based on a knowledge-enhanced neural network model, comprising:

[0076] S1 obtains the biomedical text to be tested and preprocesses it to obtain the part-of-speech tagging tensor, dependency tensor, and entity span tensor:

[0077] Preferably, the preprocessing in S1 specifically includes:

[0078] Perform word segmentation and syntactic analysis based on biomedical text, extract the part-of-speech tags of words and the dependency relationships of sentences, generate part-of-speech tag sequences and dependency relationship sequences, and mark the start and end positions of entities;

[0079] The part-of-speech tag sequence and the dependency relationship sequence are converted into a part-of-speech tag matrix and a dependency relationship matrix based on the word index of each word in the sentence or text.

[0080] Get entity span information based on the entity's start and end positions;

[0081] Based on the part-of-speech tag matrix, dependency matrix and entity span information, they are converted into part-of-speech tagging tensor, dependency tensor and entity span tensor.

[0082] Preferably, the word index is the position number of each word in the text in the sentence or text (i.e., the word sequence number). The index is used to maintain the order and position information of each word in subsequent processing, so that each word corresponds to its part-of-speech tag, dependency relationship and other features.

[0083] Preferably, this embodiment uses the Stanford CoreNLP tool to perform grammatical analysis on the biomedical text to extract the part-of-speech tags of the vocabulary and the dependency structure of the sentence. The Stanford CoreNLP tool generates a part-of-speech tag sequence POS = {p1, p2, ..., p n} and the dependency sequence DEP = {d1, d2, ..., d n}.

[0084] Preferably, the part-of-speech tag sequence is mapped into a part-of-speech tag matrix based on the word index of each word in the sentence or text (i.e., the word sequence number): pos_matrix∈R n×k , where n represents the number of words and k represents the dimension of the label; the dependency sequence is converted into the dependency matrix dep_matrix∈R n×m , where m represents the number of dependency types.

[0085] Preferably, according to the start and end positions of the marked entity, the span information Span of the entity is obtained, which describes the start and end positions of the entity in the biomedical text, and is expressed as Span = {start E ,end E}, where start E and end E They represent the starting and ending positions of entity E in the biomedical text respectively.

[0086] Preferably, the part-of-speech tagging tensor is represented as: T POS =Tensor(pos_matrix); The dependency tensor is represented as: T DEP =Tensor(dep_matrix); The entity span tensor is represented as: T Span =Tensor(Span).

[0087] S2 obtains the description information corresponding to the entities in the biomedical text based on the external knowledge base and selects keywords to obtain the entity keyword set:

[0088] Preferably, the external knowledge bases include: DrugBank, PubMED and STRING.

[0089] Preferably, for each entity E annotated in the biomedical text, the detailed description information D corresponding to the entity is obtained by querying the external knowledge base. E ; Description information D E Includes the background knowledge of entity E, such as the drug's mechanism of action, side effects, etc., expressed as: D E =Query_KnowledgeBase(E).

[0090] Preferably, the entity keyword set obtained in S2 specifically includes:

[0091] Calculate the TF-IDF score for each word in the description information;

[0092] Sort all words in descending order based on the corresponding TF-IDF scores to get the sorting results;

[0093] Based on the sorting results, a preset number of words are selected as entity keywords to form an entity keyword set.

[0094] Preferably, the TF-IDF score calculation formula is:

[0095]

[0096] Among them, f t,D represents the number of occurrences of word t in the description information D, f t',D represents the total number of occurrences of all words (including t itself) in the description information D, N represents the total number of all relevant documents extracted from the external knowledge base, Z represents the set of documents extracted from the external knowledge base for a specific entity, and |{D'∈Z:t∈D'}| represents the number of documents containing word t.

[0097] Preferably, the words in the description information are scored based on the TF-IDF algorithm, and all the words are sorted in descending order based on the TF-IDF score. The first q words are selected as the most representative entity keywords based on the sorting result to obtain the entity keyword set Keywords: Keywords = {k1.k2,…,k q}, k q Represents the qth keyword extracted from the ranking.

[0098] S3 processes biomedical text and entity keyword sets based on a pre-trained language model, and obtains text semantic vectors and keyword embedding vectors accordingly:

[0099] Preferably, the text semantic vector and keyword embedding vector are obtained in S3, specifically including:

[0100] The structure of the pre-trained language model BioBERT is as follows Figure 3 As shown in the figure, the biomedical text is embedded in words based on the pre-trained language model to obtain the text semantic vector V:

[0101] V = BioBERT(Tokens);

[0102] Among them, BioBERT represents the pre-trained language model, and Tokens represents the word sequence in the pre-processed biomedical text;

[0103] Based on the pre-trained language model, the entity keyword set is processed by word embedding to obtain the keyword embedding vector K E :

[0104] K E = BioBERT(Keywords);

[0105] Among them, Keywords represents the entity keyword set.

[0106] Preferably, the text semantic vector V∈R n×h ,Tokens={t1,t2,…,t n}, where n represents the number of words, h represents the hidden layer representation dimension of the BioBERT model, and t n represents the nth word in the preprocessed biomedical text; the keyword embedding vector K E ∈R q×h , q represents the number of keywords.

[0107] S4 is based on the fusion of part-of-speech tagging tensor, dependency tensor, entity span tensor and text semantic vector and inputs them into the Transformer network to obtain the global semantic representation of the text:

[0108] Preferably, obtaining the global semantic representation of the text in S4 specifically includes:

[0109] Based on the part-of-speech tagging tensor T POS , dependency tensor T DEP , entity span tensor T Span Fuse it with the text semantic vector V to obtain the comprehensive semantic feature representation F:

[0110]

[0111] in, Represents feature concatenation operation;

[0112] Based on the comprehensive semantic feature representation F, it is input into the Transformer network to obtain the global semantic representation H of the text T :

[0113] H T = Transformer(F);

[0114] Among them, Transformer represents the Transformer network.

[0115] Preferably, the comprehensive semantic feature representation F is input into the Transformer network, and the long-distance dependency of the text is captured by the multi-head self-attention mechanism, while retaining the word order information, and the position is annotated by position encoding to generate the global semantic representation H of the text T , H T ∈R n×h .

[0116] S5 inputs the keyword embedding vector into the convolutional neural network to obtain the local features of the keyword:

[0117] Preferably, based on the keyword embedding vector K E The data is input into the convolutional neural network for local feature extraction. The local context information between keywords is captured through one-dimensional convolution operation. Then, the most significant features are extracted through the maximum pooling operation to generate the keyword local feature K CNN :

[0118] K CNN =CNN(K E );

[0119] Among them, CNN stands for convolutional neural network.

[0120] S6 outputs entity relationship joint detection results based on entity span tensor, global semantic representation of text and local features of keywords.

[0121] Preferably, outputting the entity relationship joint detection result in S6 specifically includes:

[0122] Based on the global semantic representation of text H T And keyword local feature K CNN Perform splicing to obtain joint features;

[0123] Based on the joint features and entity span tensor T Span Perform the maximum pooling operation to obtain the entity features corresponding to each entity;

[0124] Obtain entity category probability distribution based on entity features;

[0125] Get the probability distribution of relationship categories based on any two entity features;

[0126] The entity-relationship joint detection result is based on the entity category probability distribution and the relationship category probability distribution.

[0127] Preferably, the combined features represents the splicing operation, H fused ∈R (n+q)×h .

[0128] Preferably, the combined feature H fusedAfter dual processing by the Transformer network and CNN, the fused feature representation can contain global and local semantic information, which plays an important role in accurately predicting the relationship between drug entities and adverse reaction entities.

[0129] Preferably, based on the joint feature H fused With the entity span tensor T Span Perform the maximum pooling operation to obtain the entity features corresponding to each entity:

[0130]

[0131] Among them, EntityFeatures i Represents the entity features corresponding to the i-th entity, and MaxPooling represents the maximum pooling operation, which is used to extract the most significant features from the span of the i-th entity. The entity span tensor representing the i-th entity;

[0132] Preferably, the entity category probability distribution acquisition method is:

[0133] Based on the entity features, it is mapped to the entity category space through the fully connected layer, and the entity category probability distribution is predicted by the first activation function.

[0134] Preferably, in this embodiment, the first activation function uses the Softmax function to extract the entity features EntityFeatures i Processed through the fully connected layer, mapped to the entity category space, and generated the logits (logarithmic probability) of the entity category to represent entity_logits i , refers to the unnormalized score output by the neural network. The logits are normalized using the Softmax function to generate the entity category probability distribution of each entity:

[0135] entity_logits i =Softmax(FC(EntityFeatures i ))

[0136] Among them, entity_logits i represents the entity category probability distribution of the i-th entity. Softmax represents the Softmax function, which is used to convert logits into probability distribution to ensure that the sum is 1, so that the probability of each category can be intuitively represented. FC represents the fully connected layer.

[0137] Preferably, the method for obtaining the probability distribution of relationship categories is:

[0138] Based on the splicing of any two entity features, the entity pair relationship features are obtained;

[0139] Based on the entity-to-entity relationship features, the fully connected layer is used to map them to the entity category space, and the relationship category between the two entities is predicted through the second activation function to obtain the relationship category probability distribution.

[0140] Preferably, an entity pair relationship feature is obtained by concatenating any two entity features:

[0141]

[0142] Among them, EntityFeatures i and EntityFeatures j They represent the entity features of the i-th entity and the j-th entity respectively.

[0143] Preferably, the second activation function of this embodiment adopts the Sigmoid function, and the entity pair relationship feature RelationType is converted through the fully connected layer. ij Map it to the relationship category space, and use the Sigmoid function to predict the relationship category between two entities to obtain the relationship category probability distribution:

[0144] relation_logits ij =Sigmoid(FC(RelationFeatures ij ));

[0145] Among them, relation_logits ij Represents the probability distribution of relationship categories, which is used to represent the probability distribution of the relationship between the i-th entity and the j-th entity. Sigmoid represents the Sigmoid function, which limits the output between 0 and 1 and is used to represent the probability of the existence of a relationship.

[0146] Preferably, the entity category probability distribution of each entity and the relationship category probability distribution between entity pairs are used as entity relationship joint detection results, and the entity relationship joint detection results are expressed as:

[0147] Output=(entity_logits,relation_logits)

[0148] Among them, entity_logits represents the category prediction logits (logarithmic probability) of all entities, that is, the entity category probability distribution of all entities, relation_logits represents the relationship prediction logits (logarithmic probability) of all entity pairs, that is, the relationship category probability distribution of all entity pairs, and Output includes the category prediction of all entities and the relationship prediction results of all entity pairs.

[0149] The output covers the classification probability of entities and the classification probability of relationships between entities, which is a direct product of the entity recognition and relationship classification tasks. With the help of this joint prediction layer, the model can provide accurate classification for entities and relationships in biomedical texts while comprehensively considering various feature information, thereby facilitating the accurate extraction of entities and relationships in adverse drug reaction research.

[0150] Preferably, biomedical texts to be tested are collected from biomedical databases and literature resources, and the texts are parsed using natural language processing tools to output the relationships between drug entities and adverse reaction entities as entity relationship joint detection results.

[0151] Preferably, steps S1-S6 jointly construct a drug adverse reaction detection model, by inputting the text to be detected, and finally outputting the entity relationship joint detection result;

[0152] Obtain the biomedical text to be tested from external biomedical databases (such as PubMed, DrugBank, STRING, etc.) and clinical reports; the biomedical text is used to detect the relationship between drug entities and their adverse reaction-related entities;

[0153] Input the biomedical text into the trained drug adverse reaction detection model for processing. Through the above steps S1-S6, the process from entity recognition to relationship extraction is completed; the model can identify the drug and adverse reaction entities in the text and capture the semantic and logical relationships between these entities;

[0154] Output the prediction results of the drug adverse reaction detection model. The model jointly predicts the drug entities and their adverse reaction entities identified in the biomedical text, and outputs the relationship between the entities, including the association information between drugs and adverse reactions.

[0155] Preferably, the drug adverse reaction detection model includes: a Transformer network and a convolutional neural network CNN, which are used to capture the long-distance dependencies in the text and the local semantic features of entity keywords. Through a customized attention mechanism combined with entity keyword features, the model can more accurately perform joint predictions of drugs and adverse reactions, predict the category of entities through the Softmax function, and use the Sigmoid function to predict the relationship between entities, and output the results of drug adverse event detection.

[0156] ; It can accurately identify drug entities and entities related to their adverse reactions, and output the relationship predictions between these entities as the entity relationship joint detection results in the drug adverse reaction detection task.

[0157] Preferably, in terms of model construction, the present invention uses a multi-layer neural network model, including a pre-trained language model, a Transformer network and a convolutional neural network CNN. First, the model embeds the text through BioBERT, and integrates the biomedical text and the external entity keyword embedding. Then, the Transformer network is used to model the global semantic information of the text, and the convolutional neural network extracts the local features of the entity keywords. Finally, the model predicts the relationship through the span information and joint feature representation of the entity, and generates the relationship prediction results between the drug entity and the adverse reaction entity. In order to improve the accuracy and robustness of the model, an optimization algorithm is used to tune the model, and finally a drug adverse reaction detection model is obtained.

[0158] Preferably, the model is also tested and evaluated: the test results of the model are comprehensively analyzed through evaluation indicators such as accuracy, recall, F1 value, etc. By adjusting the hyperparameters of the pre-trained language model BioBERT, Transformer network and convolutional neural network (CNN), the model performance is optimized and improved to further improve the accuracy and robustness of adverse drug reaction detection.

[0159] Preferably, the AdamW optimization algorithm is used to optimize the model performance, and the specific rules are as follows:

[0160]

[0161] Among them, m t and v t They represent the estimated values ​​of the first-order moment and the second-order moment, β1 and β2 represent the attenuation factors, which are usually set to 0.9 and 0.999 respectively. represents the gradient of the loss function L with respect to the model parameter θ, η represents the learning rate, and ε represents a small constant to prevent division by zero, which is usually set to 1×10 -8 θt represents the model parameters at time t.

[0162] After each training phase, the model's weights are adjusted based on the gradient of the loss function to optimize the model's performance.

[0163] Preferably, when evaluating the performance of the model in this embodiment, accuracy, recall rate, and F value are used as measurement criteria:

[0164] The accuracy is expressed as:

[0165]

[0166] The recall rate is expressed as:

[0167]

[0168] The F1 score corresponding to the F value is:

[0169]

[0170] Among them, TP, FP and FN represent the number of true positives, false positives and false negatives respectively.

[0171] In this embodiment, the Stanford CoreNLP tool is used to extract part-of-speech tags, dependency information, etc. from the biomedical text to be tested, and converted into an index, and then this information is converted into a part-of-speech tag matrix and a dependency matrix. Subsequently, in the word embedding layer, the vocabulary in the text is embedded as a vector representation, and the span information of the entity is extracted and converted into a corresponding vector. These vectors are input to the encoding layer, which combines the Transformer network and the convolutional neural network (CNN) to process the embedding vectors of the text and the entity.

[0172] The encoding layer captures the global context information in the text through the Transformer network, and extracts the local semantic features of entity keywords through the convolutional neural network. The model integrates the word embedding representation of BioBERT and the keyword features extracted from the knowledge base, and combines the part-of-speech tagging matrix and the dependency matrix to generate a comprehensive feature representation. The long-distance dependency relationship in the text is modeled through the Transformer's multi-head self-attention mechanism; at the same time, the convolutional neural network extracts the local features of the entity keywords, and forms a joint feature representation by splicing the global semantic representation of the text with the local features of the keywords.

[0173] The model combines the span information of the entity for joint prediction, uses the Softmax function to predict the entity category, uses the Sigmoid function to predict the relationship between entities, and outputs the relationship prediction results between the drug and adverse reaction entities. The model is trained and tuned through optimization algorithms (such as AdamW) to further improve the accuracy and robustness of the model, and ultimately achieve efficient prediction of drug adverse reaction detection tasks.

[0174] Example 3

[0175] Based on the above embodiments, in a specific embodiment, in order to verify the effectiveness of the model of the present invention, multiple groups of comparative experiments were conducted:

[0176] First, on the public adverse drug reaction (ADE) dataset, the performance of different network models on the adverse drug reaction detection task was experimentally compared.

[0177] The different comparison network models selected in this embodiment include: a BiLSTM-SDP model based on a bidirectional long short-term memory network (BiLSTM) and a shortest dependency path (SDP), a multi-head attention mechanism neural network model, a neural network model based on entity span prediction, a span-based pre-trained language model SpanBERT, an ERSTG model based on a graph neural network, a recurrent neural network GRU and knowledge span encoding knowledge-span, and a Transformer model using the biomedical pre-trained language model BioBERT.

[0178] The present invention adopts a neural network model based on the pre-trained language model BioBERT, and combines the entity keyword information in the external knowledge base, captures the global context information of the text through the Transformer network, and extracts the local features of the entity keyword in combination with the convolutional neural network (CNN). The topological diagram of the combination of the Transformer network and the convolutional neural network (CNN) of the present invention is shown in FIG. Figure 4 shown.

[0179] In terms of model construction, the present invention uses a multi-layer neural network model, including a pre-trained language model, a Transformer network and a convolutional neural network CNN. First, the model embeds the text through BioBERT, and integrates the biomedical text and external entity keyword embedding. Then, the Transformer network is used to model the global semantic information of the text, and the convolutional neural network extracts the local features of the entity keywords. Finally, the model predicts the relationship through the span information and joint feature representation of the entity, and generates the relationship prediction results between the drug entity and the adverse reaction entity. In order to improve the accuracy and robustness of the model, an optimization algorithm is used to tune the model, and finally a drug adverse reaction detection model is obtained.

[0180] The experimental dataset is a public ADE dataset, which mainly includes the annotations of drug entities and adverse reaction entities and their relationships. This dataset is used to train and evaluate the drug adverse reaction detection model.

[0181] In order to ensure a balance between model training efficiency and hardware resource utilization, the training batch size (train_batch_size) in the experiment of this invention is set to 8; to ensure higher accuracy in the evaluation process, the evaluation batch size (eval_batch_size) is set to 1; referring to the common parameters in BERT-related research and verifying its effectiveness in this task through experiments, the learning rate (learning_rate) is set to 4×10 -5 ; To help the model transition smoothly in the early stages of training, the learning rate warmup ratio (warmup_ratio) is set to 10%.

[0182] In terms of model structure design, the present invention sets a multi-head attention mechanism in the Transformer network and selects 4 attention heads so that the model can process information from different semantic spaces in parallel; the hidden layer dimension (hidden_size) of the model is set to 768, and the feedforward network dimension (feedforward_size) is set to 2048 to ensure that the model has sufficient expressive power; in model training, in order to prevent overfitting, the dropout technology is used, and the dropout ratio of the attention layer in the model (attention_dropout) and the dropout ratio of the BERT model (bert_dropout) are both set to 0.1, ensuring that the robustness of the model is improved while maintaining a strong learning ability; in order to optimize the accuracy of relationship prediction, the present invention sets the relationship filtering threshold (relation_filter_threshold) to 0.4, which is obtained through the results of cross-validation and can achieve a good balance between precision and recall.

[0183] The experiment was conducted on an Nvidia RTX 1080Ti GPU with 12GB of video memory, ensuring efficient training and evaluation of large-scale data sets and deep models. The above-mentioned multiple comparison models were constructed and tested on the public adverse drug reaction ADE data set. The comparison experimental results are shown in Table 1:

[0184] Table 1 Comparative experimental results

[0185]

[0186]

[0187] It can be seen from the experimental data in Table 1 that the knowledge-enhanced neural network model proposed in the present invention achieved F1 scores of 93.64% and 88.07% in named entity recognition and relationship extraction tasks, respectively, significantly surpassing other baseline models; the experimental results verify the superior performance of the model of the present invention in the task of adverse drug reaction detection, especially after integrating external knowledge base keywords and semantic information, the model can more accurately capture entities and their relationships in complex biomedical texts, providing stronger recognition and reasoning capabilities for adverse drug reaction detection.

[0188] The experimental results of the impact of different knowledge sources on model performance are shown in Table 2:

[0189] Table 2 Experimental results of different knowledge sources

[0190]

[0191] According to the results in Table 2, different knowledge sources have a significant impact on model performance: when using STRING (protein interaction database), the model has the highest F1 score in the named entity recognition task, reaching 93.71%, and the F1 score in the relationship extraction task is 88.07%. This shows that the protein interaction information provided by the STRING database is particularly useful for capturing the relationship between entities, significantly improving the overall performance of the model. In comparison, DrugBank (drug database) also performs relatively well, with an F1 score of 93.37% for entity recognition and 87.38% for relationship extraction. PubMED (biomedical literature database) performs slightly worse in entity recognition and relationship extraction tasks due to its extensive non-specialized information, achieving F1 scores of 92.93% and 86.57% respectively. When using PubCHEM (chemical substance and biological activity database), the model's performance is similar to DrugBank, with an F1 score of 93.40% for entity recognition and 87.69% for relationship extraction. Wikipedia, as a non-professional knowledge base, performs close to other knowledge sources in entity recognition tasks, but scores slightly lower in relationship extraction tasks, with an F1 score of 87.05%.

[0192] Experimental results show that selecting knowledge sources that are highly relevant to the task can significantly improve the performance of the model, especially the specialized biomedical information provided by STRING (protein interaction database) significantly improves the performance of the model in the task of adverse drug reaction detection. This further verifies the importance of selecting external knowledge sources for biomedical text processing tasks, especially in the task of relation extraction, where the information quality of the knowledge source directly affects the prediction accuracy of the model.

[0193] The experimental results of the impact of different word representation methods on model performance are shown in Table 3:

[0194] Table 3 Experimental results of different word representation methods

[0195]

[0196] From the experimental results in Table 3, it can be seen that different word representation methods have a significant impact on model performance: in the entity recognition task (NER), BioBERT (pre-trained model in the biomedical field) achieved the highest F1 score of 93.71%, and in the relation extraction task (RE), the F1 score was 88.07%. This result shows that BioBERT, as a pre-trained model optimized specifically for the biomedical field, can better capture the professional terms and complex semantics in biomedical texts, especially when dealing with adverse drug reaction detection tasks; in contrast, SciBERT (pre-trained model in the scientific literature field) also performed well in entity recognition and relation extraction tasks, achieving F1 scores of 92.15% and 85.03% respectively, indicating that it has strong generalization ability when dealing with scientific literature corpus; ClinicalBERT (pre-trained model in the clinical field) performed well in relation extraction tasks due to its focus on clinical corpus, with an F1 score of 86.41 %, while the F1 score in the entity recognition task is 92.85%, demonstrating its ability in capturing entities and relations in clinical texts; SpanBERT (span-based pre-training model) emphasizes the capture of entity span information, with F1 scores of 91.50% (NER) and 84.51% (RE), proving its practicality in entity recognition and relationship extraction tasks; in contrast, BERT (general pre-training model) lacks adaptation to the biomedical field, and although it can provide good baseline performance, it performs slightly weaker when processing biomedical texts, with F1 scores of 89.91% in entity recognition tasks and 82.02% in relationship extraction tasks.

[0197] Experimental results verify the key role of domain-specific pre-trained models (such as BioBERT) in improving the performance of biomedical text processing tasks.

[0198] The experimental results of the impact of different keyword selection strategies on model performance are shown in Table 4:

[0199] Table 4 Experimental results of different keyword selection strategies

[0200]

[0201] From the experimental results in Table 4, we can see that different keyword selection strategies have a significant impact on the performance of the model in entity recognition and relation extraction tasks:

[0202] TF-IDF (term frequency-inverse document frequency) has an F1 score of 93.71% in the entity recognition task and an F1 score of 88.07% in the relationship extraction task, which is the best performance. This shows that TF-IDF can effectively extract keywords related to entities and provide the model with the most representative external knowledge information, significantly improving the performance of the model in complex biomedical texts.

[0203] LDA (Latent Dirichlet Allocation) achieved F1 scores of 93.62% (entity recognition) and 88.04% (relation extraction) by identifying latent topics in the text, showing that it can capture domain-specific semantic features in keyword extraction, thereby providing useful background knowledge for the model.

[0204] TextRank is a keyword ranking algorithm based on graph structure, with an F1 score of 93.44% in entity recognition tasks and 87.49% in relationship extraction tasks. Although TextRank can capture keywords in text through graph structure, it lacks professionalism in the biomedical field, so its performance is slightly lower than TF-IDF and LDA.

[0205] Rake (Rapid Automatic Keyword Extraction), an algorithm that extracts keywords by analyzing word frequency and co-occurrence information, has an F1 score of 87.69% in the relation extraction task, slightly higher than TextRank. This shows that Rake has certain advantages in quickly extracting high-frequency keywords, especially when it does not rely on complex topic models.

[0206] Experimental results show that different keyword selection strategies significantly affect the performance of the model in the adverse drug reaction detection task.

[0207] Example 4

[0208] like Figure 5 As shown, the present invention proposes a drug adverse event detection system based on a knowledge-enhanced neural network model, including: a data processing module, an entity keyword generation module, a pre-trained language model processing module and a model construction and result output module;

[0209] A data processing module is used to obtain the biomedical text to be detected and perform preprocessing to obtain a part-of-speech tagging tensor, a dependency tensor, and an entity span tensor;

[0210] An entity keyword generation module is used to obtain description information corresponding to entities in biomedical texts based on an external knowledge base and select keywords to obtain an entity keyword set;

[0211] A pre-trained language model processing module is used to process biomedical text and entity keyword sets based on the pre-trained language model to obtain text semantic vectors and keyword embedding vectors;

[0212] The model building and result output module is used to build the Transformer network and convolutional neural network, fuse the part-of-speech tagging tensor, dependency tensor, entity span tensor and text semantic vector, and input them into the Transformer network to obtain the global semantic representation of the text; input the keyword embedding vector into the convolutional neural network to obtain the local features of the keywords; based on the entity span tensor, the global semantic representation of the text and the local features of the keywords, output the entity relationship joint detection results.

[0213] Preferably, it also includes a detection module;

[0214] The detection module is used to input the biomedical text to be detected into the trained drug adverse event detection model. The model combines the text and keyword information to detect drug entities and adverse reaction entities, and outputs the results of drug adverse event detection.

[0215] Example 5

[0216] Based on the same inventive concept, the present invention also provides a computer device, including a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;

[0217] Memory, used to store computer programs;

[0218] The processor, when used to execute the program stored in the memory, can implement a drug adverse event detection method based on a knowledge-enhanced neural network model as in Example 1 or 2.

[0219] The electronic device may include: a processor, a communications interface, a memory, and a communications bus, wherein the processor, the communications interface, and the memory communicate with each other via the communications bus. The processor may call logic instructions in the memory to execute a drug adverse event detection method based on a knowledge-enhanced neural network model in Embodiment 1 or 2.

[0220] In addition, the logic instructions in the above-mentioned memory can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0221] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0222] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for detecting adverse drug events based on a knowledge-enhanced neural network model, characterized in that: include: S1 obtains the biomedical text to be tested and preprocesses it to obtain a part-of-speech tagging tensor, a dependency tensor, and an entity span tensor; S2 obtains description information corresponding to the entity in the biomedical text based on an external knowledge base and selects keywords to obtain an entity keyword set; S3 processes the biomedical text and the entity keyword set based on a pre-trained language model to obtain a text semantic vector and a keyword embedding vector accordingly; S4 is based on the fusion of the part-of-speech tagging tensor, the dependency tensor, the entity span tensor and the text semantic vector, and inputs them into the Transformer network to obtain a global semantic representation of the text; S5 inputs the keyword embedding vector into a convolutional neural network to obtain a local feature of the keyword; S6 outputs entity relationship joint detection results based on the entity span tensor, the global semantic representation of the text and the local features of the keywords.

2. A method for detecting adverse drug events based on a knowledge-enhanced neural network model according to claim 1, characterized in that: The preprocessing in S1 specifically includes: Based on the biomedical text, word segmentation and syntactic analysis are performed to extract the part-of-speech tags of the vocabulary and the dependency relationships of the sentences, generate a part-of-speech tag sequence and a dependency relationship sequence, and mark the start and end positions of the entities; The part-of-speech tag sequence and the dependency relationship sequence are converted into a part-of-speech tag matrix and the dependency relationship matrix based on word index correspondence; Obtain entity span information based on the start and end positions of the entity; Based on the part-of-speech tag matrix, the dependency matrix and the entity span information, they are correspondingly converted into the part-of-speech tag tensor, the dependency tensor and the entity span tensor.

3. The method for detecting adverse drug events based on a knowledge-enhanced neural network model according to claim 1, characterized in that: The entity keyword set obtained in S2 includes: Calculate the TF-IDF score for each word in the description information; Sort all words in descending order based on the corresponding TF-IDF scores to obtain a sorting result; A preset number of words are selected based on the sorting results as entity keywords to form the entity keyword set.

4. The method for detecting adverse drug events based on a knowledge-enhanced neural network model according to claim 3, characterized in that: The TF-IDF score calculation formula is: Among them, f t,D represents the number of occurrences of word t in the description information D, f t',D represents the total number of occurrences of all words (including t itself) in the description information D, N represents the total number of all relevant documents extracted from the external knowledge base, Z represents the set of documents extracted from the external knowledge base for a specific entity, and |{D'∈Z:t∈D'}| represents the number of documents containing word t.

5. The method for detecting adverse drug events based on a knowledge-enhanced neural network model according to claim 3, characterized in that: The text semantic vector and keyword embedding vector are obtained in S3, including: The biomedical text is subjected to word embedding processing based on the pre-trained language model to obtain the text semantic vector V: V = BioBERT(Tokens); Among them, BioBERT represents the pre-trained language model, and Tokens represents the word sequence in the pre-processed biomedical text; Based on the pre-trained language model, the entity keyword set is processed by word embedding to obtain the keyword embedding vector K E : K E =BioBERT(Keywords); Among them, Keywords represents the entity keyword set.

6. A method for detecting adverse drug events based on a knowledge-enhanced neural network model according to claim 5, characterized in that: The global semantic representation of the text is obtained in S4, including: Based on the part-of-speech tagging tensor T POS , the dependency tensor T DEP , the entity span tensor T Span Fermented with the text semantic vector V, a comprehensive semantic feature representation F is obtained: in, Represents feature concatenation operation; Based on the comprehensive semantic feature representation F, input into the Transformer network to obtain the text global semantic representation H T : H T =Transformer(F); Among them, Transformer represents the Transformer network.

7. The method for detecting adverse drug events based on a knowledge-enhanced neural network model according to claim 6, characterized in that: S6 outputs the entity relationship joint detection results, including: Based on the global semantic representation H of the text T and the local features of the keyword to obtain a joint feature; Based on the joint feature and the entity span tensor T Span Perform the maximum pooling operation to obtain the entity features corresponding to each entity; Obtaining entity category probability distribution based on the entity features; Obtaining a relationship category probability distribution based on any two of the entity features; The entity relationship joint detection result is based on the entity category probability distribution and the relationship category probability distribution.

8. The method for detecting adverse drug events based on a knowledge-enhanced neural network model according to claim 7, characterized in that: The entity category probability distribution acquisition method is: Based on the entity features, the entity features are mapped to the entity category space through a fully connected layer, and the entity category probability distribution is obtained by prediction through a first activation function; The method for obtaining the probability distribution of relationship categories is: Based on any two of the entity features, concatenation is performed to obtain an entity pair relationship feature; Based on the entity pair relationship features being mapped to the entity category space through the fully connected layer, and predicting the relationship category between the two entities through the second activation function, the relationship category probability distribution is obtained.

9. The method for detecting adverse drug events based on a knowledge-enhanced neural network model according to claim 1, characterized in that: The biomedical texts to be tested are collected from biomedical databases and literature resources, and the texts are parsed using natural language processing tools. The relationships between drug entities and adverse reaction entities are output as the entity relationship joint detection results.

10. A drug adverse event detection system based on a knowledge-enhanced neural network model, using a drug adverse event detection method based on a knowledge-enhanced neural network model as described in any one of claims 1 to 9, characterized in that: include: Data processing module, entity keyword generation module, pre-trained language model processing module and model construction and result output module; The data processing module is used to obtain the biomedical text to be detected and perform preprocessing to obtain a part-of-speech tagging tensor, a dependency tensor and an entity span tensor; The entity keyword generation module is used to obtain description information corresponding to the entity in the biomedical text based on an external knowledge base and select keywords to obtain an entity keyword set; The pre-trained language model processing module is used to process the biomedical text and the entity keyword set based on the pre-trained language model to obtain a text semantic vector and a keyword embedding vector accordingly; The model construction and result output module is used to construct a Transformer network and a convolutional neural network, fuse the part-of-speech tagging tensor, the dependency tensor, the entity span tensor and the text semantic vector, and input them into the Transformer network to obtain a global semantic representation of the text; input the keyword embedding vector into the convolutional neural network to obtain local features of the keywords; and output entity relationship joint detection results based on the entity span tensor, the global semantic representation of the text and the local features of the keywords.

Images