Medical text entity-relation extraction method and system based on grammar analysis

Abstract

The present invention relates to the technical field of medical data processing. Disclosed are a medical text entity-relation extraction method and system based on grammar analysis. According to the present invention, considering the complexity of medical relations, in order to prevent an incorrect result of entity extraction from affecting relation extraction, the relation extraction is specifically arranged before the entity extraction. In the entity extraction process, cross attention is used to integrate relations and encoding features, so as to increase the interaction between the relations and entity information. After extracting the relations and entities, by means of grammar analysis, semantic information is introduced, and part-of-speech and grammar information are fully utilized, thereby improving the accuracy of entity-relation extraction. Using an end-to-end model and constructing a sentence-level task facilitate representation and learning of sentence semantics.

Description

A method and system for extracting entity relations from medical text based on grammatical analysis

Technical field

[0001] The present invention belongs to the technical field of medical data processing, and more specifically, relates to a method and system for extracting entity relationships from medical text based on grammatical analysis. [Background Technology]

[0002] Medical data holds immense value, especially in the context of relationships within medical text. Relationship extraction within medical text refers to the process of extracting specific factual information from medical text, storing it as structured data in a database for user query or further analysis. For example, a biomedical scientist seeking new treatment options for a disease from a vast amount of biomedical literature could potentially discover valuable therapeutic clues or methods by using drug-disease relationship extraction (relationship extraction refers to extracting information about interactions between proteins, genes, or drugs from text) within medical text.

[0003] Existing relationship extraction technologies are mostly designed for traditional text, with few being applicable to medical text. Existing medical data extraction methods all suffer from the following flaws and shortcomings: insufficient utilization of semantic information and insufficient interaction between relationship and entity information, resulting in poor recognition. Furthermore, the BIO sequence tagging method cannot resolve the triple overlap problem in medical text.

[0004] [Summary of the invention]

[0005] In view of the defects of the prior art, the purpose of the present invention is to provide a medical text entity relationship extraction method and system based on grammatical analysis, aiming to solve the problems of insufficient utilization of semantic information and insufficient interaction between relationship and entity information in existing extraction methods.

[0006] To achieve the above objectives, in a first aspect, the present invention provides a method for extracting entity relationships from medical text based on grammatical analysis, comprising:

[0007] Input the medical text to be extracted into the trained end-to-end model to obtain a set of entity relationship triples as the extraction result;

[0008] in,

[0009] The end-to-end model includes: an encoder, a medical relationship extraction module, a cross-attention mechanism module, a medical entity extraction module, and a grammatical analysis module;

[0010] The encoder is used to perform vector encoding on the medical text to obtain a set of sentence-level feature vectors;

[0011] The medical relationship extraction module is used to input each feature vector in the sentence-level feature vector set into the first linear neural network to obtain a sentence-level relationship vector set;

[0012] The cross-attention mechanism module is used to use each relationship vector in the sentence-level relationship vector set as a Cross-Attention query, use the corresponding feature vector as the key and value of Cross-Attention, and output a sentence-level fusion vector set;

[0013] The medical entity extraction module is used to input each fusion vector in the sentence-level fusion vector set into a second linear neural network to obtain a sentence-level entity vector set;

[0014] The grammatical analysis module is used to first perform a dot product of each feature vector in the sentence-level feature vector set with the corresponding sentence-level annotation vector, and then pass the dot product result vector through the activation function Relu to obtain a sentence-level non-negative vector with a dimension of (relationship type * sentence length * sentence length), and the annotation includes a noun annotation and a dependency relationship annotation; in each relationship type, the sentence-level non-negative vector and the corresponding sentence-level entity vector are entity matched to obtain an entity relationship triple (medical entity 1, medical relationship, medical entity 2).

[0015] Preferably, in the medical entity extraction module, the entities in the sentence are annotated using a pointer method to obtain a sentence-level entity vector with a dimension of (2*sentence length). The first row is used to annotate the head of each entity in the sentence, and the second row is used to annotate the tail of each entity in the sentence.

[0016] It should be noted that the present invention preferably uses the above-mentioned pointer annotation method to extract entities. Unlike sequence annotation, it only annotates the head and tail of the entity, but not the middle, which can solve the problem of entity overlap.

[0017] Preferably, the encoder adopts SciBERT.

[0018] Preferably, SciBERT is fine-tuned using the ADE ChemProt dataset and predefined hyperparameters.

[0019] Preferably, the annotation is as follows:

[0020] The annotation vector element corresponding to each noun in the sentence is marked as x1. If two nouns have a dependency relationship, the corresponding annotation vector element is marked as x2 to obtain the sentence-level annotation vector.

[0021] To achieve the above-mentioned objectives, in a second aspect, the present invention provides a medical text entity relationship extraction system based on grammatical analysis, comprising: a processor and a memory; the memory is used to store computer-executable instructions; the processor is used to execute the computer-executable instructions so that the method described in the first aspect is executed.

[0022] To achieve the above objectives, in a third aspect, the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program runs on a processor, the processor executes the method described in the first aspect.

[0023] In general, the above technical solutions conceived by the present invention have the following beneficial effects compared with the prior art:

[0024] The present invention proposes a method and system for extracting entity relationships from medical texts based on grammatical analysis. Taking into account the complexity of medical relationships, in order to prevent the erroneous results of entity extraction from affecting relationship extraction, relationship extraction is arranged before entity extraction. In the entity extraction process, cross attention is used to fuse relationships and encoding features to increase the interaction between relationship and entity information. After extracting relationships and entities, semantic information is introduced with the help of grammatical analysis, and part of speech and grammatical information are fully utilized to improve the accuracy of entity relationship extraction. The use of an end-to-end model and the construction of sentence-level tasks are conducive to the representation and learning of sentence semantics.

Brief Description of the Drawings

[0025] FIG1 is a schematic diagram of an end-to-end model for extracting entity relationships from medical texts provided by the present invention. [Specific implementation method]

[0026] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0027] The present invention provides a method for extracting entity relationships from medical texts based on grammatical analysis, comprising: inputting the medical text to be extracted into a trained end-to-end model to obtain a set of entity relationship triples as an extraction result.

[0028] As shown in Figure 1, the end-to-end model includes: an encoder, a medical relationship extraction module, a cross-attention mechanism module, a medical entity extraction module and a grammatical analysis module.

[0029] encoder

[0030] The encoder is used to perform vector encoding on the medical text to obtain a set of sentence-level feature vectors.

[0031] Preferably, the encoder adopts SciBERT.

[0032] In terms of network structure, SciBERT is fully bidirectional, has deep layers, and a large enough function space to represent. It also proposes self-attention to overcome the problem of long-distance dependency. The pre-training tasks are well designed, and sentence-level tasks are added, which is conducive to the representation and learning of sentence semantics. The training corpus is large and the training is sufficient.

[0033] Preferably, SciBERT is fine-tuned using the ADE ChemProt dataset and predefined hyperparameters.

[0034] Medical text is a paragraph-based text with a character count of approximately 200-500. It can be in Chinese or a foreign language, such as ancient Chinese medical texts. It is stored in JSON format. When the key is text, the value contains the text itself. When the key is a triple list, the value contains the text (medical entity 1, medical relationship, medical entity 2).

[0035] Before encoding, the text to be processed may be preprocessed, including but not limited to: deleting or interpolating missing data, format conversion, etc.

[0036] Medical relationship extraction module

[0037] The medical relationship extraction module is used to input each feature vector in the sentence-level feature vector set into the first linear neural network to obtain a sentence-level relationship vector set.

[0038] The results of SciBERT are input into the first linear neural network and finally classified using the Softmax function. When the result is greater than the threshold, it indicates that there is a relationship.

[0039] Cross-Attention Mechanism

[0040] The cross-attention mechanism module is used to use each relationship vector in the sentence-level relationship vector set as a Cross-Attention query, use the corresponding feature vector as the key and value of Cross-Attention, and output a sentence-level fusion vector set.

[0041] This invention uses Cross Attention. Cross Attention is typically used as a decoder module, alongside Self Attention as an encoder. Its input query comes from the output of the encoder (Self Attention), while the key and value come from the initial input. Here, the key and value are the results of BERT, and the query is the result of relation extraction, thus enhancing the interaction between relation and entity information.

[0042] Medical entity extraction

[0043] The medical entity extraction module is used to input each fusion vector in the sentence-level fusion vector set into the second linear neural network to obtain a sentence-level entity vector set.

[0044] Preferably, in the medical entity extraction module, the entities in the sentence are annotated using a pointer method to obtain a sentence-level entity vector with a dimension of (2*sentence length). The first row is used to annotate the head of each entity in the sentence, and the second row is used to annotate the tail of each entity in the sentence.

[0045] Syntax analysis module

[0046] The grammatical analysis module is used to first perform a dot product of each feature vector in the sentence-level feature vector set with the corresponding sentence-level annotation vector, and then pass the dot product result vector through the activation function Relu to obtain a sentence-level non-negative vector with a dimension of (relationship type * sentence length * sentence length), and the annotation includes a noun annotation and a dependency relationship annotation; in each relationship type, the sentence-level non-negative vector and the corresponding sentence-level entity vector are entity matched to obtain an entity relationship triple (medical entity 1, medical relationship, medical entity 2).

[0047] The present invention performs part-of-speech tagging and dependency parsing on sentences. Part-of-speech tagging involves assigning appropriate part-of-speech labels to each word in a sentence. Parts of speech are categorized as verbs, nouns, adjectives, and so on. Part-of-speech tagging is a preprocessing step for many NLP tasks. For example, knowing the part of speech of each word in a sentence makes syntactic analysis much easier.

[0048] Preferably, the annotation is specifically as follows: the annotation vector element corresponding to each noun in the sentence is marked as x1, and if two nouns have a dependency relationship, the corresponding annotation vector element is marked as x2 to obtain a sentence-level annotation vector.

[0049] In one embodiment, an NLP tool is used to preprocess the sentence to obtain part-of-speech information and dependency syntactic information in the sentence, obtain a part-of-speech matrix, and compare it with the true value to obtain a Loss function.

[0050] Table 1 compares the performance of different extraction models on the adverse drug event dataset. Table 2 compares the performance of different extraction models on the chemprot dataset. As can be seen from the comparison, SciBERT, adopted by the present invention, achieves higher accuracy, recall, and F1 scores than other models.

[0051] Table 1

[0052] EVALUATION ON ADVERSE DRUG EVENTS DATASET.

[0053] Table 2

[0054] EVALUATION OF ALGORITHMS ON CHEMPROT DATASET.

[0055] It will be easily understood by those skilled in the art that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for extracting entity relations from medical text based on grammatical analysis, characterized in that: include: Input the medical text to be extracted into the trained end-to-end model to obtain a set of entity relationship triples as the extraction result; in, The end-to-end model includes: an encoder, a medical relationship extraction module, a cross attention mechanism module, a medical entity extraction module and a grammar analysis module; The encoder is used to perform vector encoding on the medical text to obtain a set of sentence-level feature vectors; The medical relationship extraction module is used to input each feature vector in the sentence-level feature vector set into the first linear neural network to obtain a sentence-level relationship vector set; The cross-attention mechanism module is used to use each relationship vector in the sentence-level relationship vector set as a Cross-Attention query, use the corresponding feature vector as the key and value of Cross-Attention, and output a sentence-level fusion vector set; The medical entity extraction module is used to input each fusion vector in the sentence-level fusion vector set into the second linear neural network to obtain a sentence-level entity vector set; The grammatical analysis module is used to first perform a dot product of each feature vector in the sentence-level feature vector set with the corresponding sentence-level annotation vector, and then pass the dot product result vector through the activation function Relu to obtain a sentence-level non-negative vector with a dimension of (relationship type*sentence length*sentence length), wherein the annotation includes a noun annotation and a dependency relationship annotation; in each relationship type, the sentence-level non-negative vector and the corresponding sentence-level entity vector are entity matched to obtain an entity relationship triple (medical entity 1, medical relationship, medical entity 2).

2. The method according to claim 1, characterized in that In the medical entity extraction module, the entities in the sentence are annotated using pointers to obtain a sentence-level entity vector with a dimension of (2*sentence length). The first row is used to annotate the head of each entity in the sentence, and the second row is used to annotate the tail of each entity in the sentence.

3. The method according to claim 1, characterized in that The encoder adopts SciBERT.

4. The method according to claim 3, characterized in that SciBERT was fine-tuned with the ADE ChemProt dataset and predefined hyperparameters.

5. The method according to claim 1, characterized in that The markings are as follows: The annotation vector element corresponding to each noun in the sentence is marked as x1. If two nouns have a dependency relationship, the corresponding annotation vector element is marked as x2 to obtain a sentence-level annotation vector.

6. A medical text entity relationship extraction system based on grammatical analysis, characterized in that: include: Processor and memory; The memory is used to store computer-executable instructions; The processor is used to execute the computer-executable instructions so that the method according to any one of claims 1 to 5 is executed.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program runs on a processor, the processor is caused to execute the method according to any one of claims 1 to 5.

Images