Document-level biomedical relationship extraction method based on prompt optimization model

Abstract

The invention discloses a document-level biomedical relationship extraction method based on a prompt optimization model, on one hand, a selection area constraint module is designed, redundant information can be effectively eliminated through a novel selection area constraint method, and complex interaction between entities is simplified, so that the extraction process focuses on a target entity, and the relationship extraction quality is improved; on the other hand, a knowledge optimization prompt template construction module is designed, entity type knowledge can be used for optimizing a prompt template, the prompt template is made to learn sufficient fact knowledge of specific entity pairs, and therefore the semantic richness of the prompt template and the overall performance of document-level biomedical relation extraction are improved. In a word, according to the method, text relation information can be concentrated, redundant negative information is eliminated, relation fact knowledge contained by an entity to the entity is fully mined, and accurate extraction of the document-level biomedical relation under the condition of complex texts is achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of biomedical natural language processing, and in particular to a document-level biomedical relationship extraction method based on a prompt optimization model. Background Art

[0002] In recent years, with the continuous deepening of biomedical research, the number of biomedical literature has grown exponentially. These documents contain rich biomedical knowledge and are of great significance for promoting research and application in the medical field. However, it is very difficult for researchers to read such a large number of documents. Therefore, it has become a vital task to automatically extract structured knowledge from biomedical literature. Among them, the biomedical relationship extraction task plays a very important role in the field of biomedical natural language processing.

[0003] The goal of biomedical relationship extraction is to determine the relationships between biomedical entity pairs from unstructured biomedical texts. Currently, biomedical relationship extraction techniques mainly focus on sentence-level or fragment-level relationship extraction. Although these techniques perform well in certain specific tasks, they are often unable to handle complex relationships across sentences or even paragraphs. Document-level biomedical relationship extraction aims to extract intra-sentence entity pair relationships and cross-sentence entity pair relationships from the perspective of the entire document, which helps to improve the efficiency of information acquisition and processing. However, compared with sentence-level relationship extraction, the number of entities involved in the document is larger, and the processing process needs to comprehensively consider contextual information and document structure to extract the complex logical relationships of entity pairs, which makes the document-level biomedical relationship extraction task more challenging and complex.

[0004] In the past, document-level biomedical relationship extraction methods mainly include sequential structure-based methods and graph structure-based methods. Sequential structure-based methods use deep neural networks (such as CNN, RNN, LSTM, GRU) and attention mechanisms to capture long-distance document information, thereby achieving document-level biomedical relationship extraction; graph structure-based methods construct document graphs by finding the connections between keywords in documents to obtain the global structural information of documents, thereby performing document-level biomedical relationship extraction. In recent years, many Transformer-based pre-trained language models have achieved advanced performance in capturing long-distance text context information and have been widely used in the study of document-level biomedical relationship extraction; prompt learning-based methods have become a new paradigm for many natural language processing tasks, which formalizes the classification task as a masked language modeling problem.

[0005] At present, in the task of document-level biomedical relationship extraction, the mainstream methods are based on Transformer and reasoning technology. However, such methods often rely too much on complex downstream network architectures and contextual representations provided by pre-trained language models, which affects the quality of reasoning. In contrast, prompt learning-based methods are more efficient in the training and reasoning process, require fewer parameters and computing resources, and are more flexible and interpretable. However, most of the existing prompt learning-based methods ignore the integration of factual knowledge contained in the entity pairs themselves, and also often ignore the negative impact of some redundant noise in biomedical literature on the model extraction effect. Summary of the invention

[0006] The present invention aims to solve the above-mentioned technical problems existing in the prior art and provides a document-level biomedical relationship extraction method based on a prompt optimization model.

[0007] The technical solution of the present invention is: a document-level biomedical relationship extraction method based on a prompt optimization model, which is composed of a selection constraint module, a knowledge optimization prompt template construction module and a prompt relationship prediction module, and is performed in the following steps:

[0008] Step 1. Obtain a relation instance text in the document to be extracted through the selection constraint module, process the text with the selection constraint condition, and obtain the document sequence after the selection constraint. The specific steps are as follows:

[0009] Step 1.1 Obtain a relation instance text in the document to be extracted and determine the sentence position set p where all mentions of the head entity are located h The set of sentence positions p where all mentions of the tail entity are located t , the head entity and the tail entity constitute the target entity pair that needs to be predicted in the document, that is, each relationship instance;

[0010] Step 1.2: According to formula (1), obtain the set p of all sentences that have associated information with the head entity and the tail entity;

[0011] p=p h ∪p t (1)

[0012] Step 1.3: Set p is used as the selection constraint condition. Only the sentences in the document that exist in set p are retained through the judgment mechanism, thereby obtaining the document sequence after the selection constraint;

[0013] Step 2. Construct a knowledge optimization prompt template through the knowledge optimization prompt template construction module. The specific steps are as follows:

[0014] Step 2.1 Use ENTITY / TYPE MARKER to index the head and tail entity positions of the document sequence after the selection constraint, that is, insert special markers around the target entity in the document and add markers at the beginning of the text to obtain the text sequence after adding markers, denoted as x;

[0015] Step 2.2 defines the prompt template as shown in (2):

[0016] T(·)=“·The relations between ET .h and ET ·t is[MASK].” (2)

[0017] Among them, ET ·h and E.T. ·t Represents the type of head entity and tail entity respectively;

[0018] Map x to T(·) to obtain the knowledge optimization prompt template shown in (3);

[0019] x prompt =T(x)=“x The relations between ET xh and ET xt is[MASK].” (3)

[0020] The [MASK] is a mask mark;

[0021] Step 3. Complete the relationship instance classification through the prompt relationship prediction module. The specific steps are as follows:

[0022] Step 3.1 Construct the knowledge optimization prompt template x prompt Input the pre-trained language model T5 based on the Transformer architecture and calculate the hidden vector representation h of the mask mark [MASK] position [MASK] ;

[0023] Step 3.2 generates the probability distribution of each possible label word at the mask [MASK] position, that is, performs relationship prediction through linear transformation and softmax function, thereby completing the relationship instance classification, that is, obtaining the relationship type of the document relationship instance;

[0024] The linear transformation is shown in (4):

[0025] z=h [MASK] ·W+b (4)

[0026] Where W is the weight matrix in the output layer, and b is the bias vector in the output layer;

[0027] The softmax function, as shown in formula (5), converts the unnormalized score vector z into a probability distribution, thereby generating a probability distribution P(y|x) of each possible label word at the mask position;

[0028] P(y|x)=P([MASK]=v|x prompt )=softmax(z) (5)

[0029] in, is the set of all label words.

[0030] Compared with the prior art, the present invention designs a selection constraint module on the one hand, which can effectively eliminate redundant information and simplify the complex interactions between entities through a novel selection constraint method, so that the extraction process focuses on the target entity and improves the quality of relationship extraction; on the other hand, it designs a knowledge optimization prompt template construction module, which can use entity type knowledge to optimize the prompt template, so that the prompt template can learn sufficient factual knowledge of specific entity pairs, thereby improving the semantic richness of the prompt template and the overall performance of document-level biomedical relationship extraction. In short, the present invention can concentrate text relationship information, eliminate redundant negative information, fully explore the relationship factual knowledge contained in the entity pairs themselves, and realize the accurate extraction of document-level biomedical relationships in complex text conditions. Experimental results show that the present invention exhibits excellent competitive performance on the two datasets of CDR and GDA, and effectively improves the overall accuracy of document-level biomedical relationship extraction. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 It is a schematic diagram of the overall architecture of an embodiment of the present invention. DETAILED DESCRIPTION

[0032] A document-level biomedical relationship extraction method based on a prompt optimization model of the present invention is as follows Figure 1 As shown, it is composed of a selection constraint module, a knowledge optimization prompt template construction module and a prompt relationship prediction module, and is performed in the following steps:

[0033] Step 1. Obtain a relation instance text in the document to be extracted through the selection constraint module, process the text with the selection constraint condition, and obtain the document sequence after the selection constraint. The specific steps are as follows:

[0034] Step 1.1 Obtain a relation instance text in the document to be extracted and determine the sentence position set p where all mentions of the head entity are located h The set of sentence positions p where all mentions of the tail entity are located t , the head entity and the tail entity constitute the target entity pair to be predicted in the document, that is, each relationship instance; the mention is the multiple occurrences of each entity in the document;

[0035] Step 1.2: According to formula (1), obtain the set p of all sentences that have associated information with the head entity and the tail entity;

[0036] p=p h ∪p t (1)

[0037] Step 1.3: Set p is used as the selection constraint condition. Only the sentences in the document that exist in set p are retained through the judgment mechanism, thereby obtaining the document sequence after the selection constraint;

[0038] Step 2. Construct a knowledge optimization prompt template through the knowledge optimization prompt template construction module. The specific steps are as follows:

[0039] Step 2.1 Use ENTITY / TYPE MARKER to index the head and tail entity positions of the document sequence after the selection constraint, that is, insert special markers [E] and [ / E] around the target entity in the document and add the marker [CLS] at the beginning of the text to obtain the text sequence after adding the marker, denoted as x, such as [CLS]...[E2]vocal fold palsy[ / E2]...[E1]disulfiram[ / E1]..., [E1][ / E1] marks the head entity, and [E2][ / E2] marks the tail entity;

[0040] Step 2.2 defines the prompt template as shown in (2):

[0041] T(·)=“·The relations between ET ·h and ET ·t is[MASK].” (2)

[0042] Among them, ET ·h and E.T. ·t Represents the type of head entity and tail entity respectively;

[0043] Map x to T(·) to obtain the knowledge optimization prompt template shown in (3);

[0044] x prompt =T(x)=“x The relations between ET xh and ET xt is[MASK].” (3)

[0045] The [MASK] is a mask tag; in addition to the original tag, the prompt template also includes at least one mask tag [MASK], which is used to guide the pre-trained language model to predict the label relationship at the mask position. The constructed optimized template introduces additional entity type knowledge in the natural language text prompt to improve the performance;

[0046] Step 3. Complete the relationship classification by prompting the relationship prediction module. The specific steps are as follows:

[0047] Step 3.1 Construct the knowledge optimization prompt template x prompt Input the T5 (Text-to-Text Transfer Transformer) model, a pre-trained language model based on the Transformer architecture, and calculate the hidden vector representation h of the mask mark [MASK] position. [MASK] ;

[0048] The T5 is a pre-trained language model based on the Transformer architecture (Text-to-Text Transfer Transformer) proposed by Google;

[0049] Step 3.2 generates the probability distribution of each possible label word at the mask [MASK] position, that is, performs relationship prediction through linear transformation and softmax function, thereby completing relationship classification and obtaining the relationship type of the document relationship instance;

[0050] The linear transformation is shown in (4):

[0051] z=h [MASK] ·W+b (4)

[0052] Where W is the weight matrix in the output layer, and b is the bias vector in the output layer;

[0053] The softmax function, as shown in formula (5), converts the unnormalized score vector z into a probability distribution, thereby generating a probability distribution P(y|x) of each possible label word at the mask position;

[0054] P(y|x)=P([MASK]=v|x prompt )=softmax(z) (5)

[0055] in, is the set of all label words.

[0056] In order to verify the effectiveness of the present invention, experiments were conducted on two widely used document-level biomedical relationship extraction datasets, CDR and GDA, based on the method provided by the present invention. The Biocreative V Chemical DiseaseRelation benchmark (CDR) is a human-annotated relationship extraction dataset in the biomedical field consisting of PubMed abstracts. It contains the relationship between chemical and disease concepts: chemical-induced disease (CID). The Gene Disease Associations (GDA) is a large-scale relationship extraction dataset in the biomedical field, and its task is to predict the interaction between gene and disease concepts. The statistical data of the two datasets are shown in Table 1.

[0057] Table 1 Statistics of CDR and GDA datasets

[0058]

[0059] The evaluation criteria used in the experiment are P, R and F1 value, where P is the precision, R is the recall, and F1 value is a comprehensive evaluation criterion for evaluating general classification problems. In addition, the Intra-F1 and Inter-F1 metrics are also used to evaluate the performance of the model on intra-sentence relations and inter-sentence relations.

[0060] The performance comparison results between the method of the present invention and the existing methods are shown in Tables 2 and 3. The results show that the method of the present invention is superior to the existing methods in all aspects. On the CDR dataset, compared with the DocuNet model with the best effect in the Transformer-based model, the method of the present invention improves the overall F1 value by 1.30%; compared with the CPT-RI model with the best effect in the prompt-based model, the method of the present invention improves the overall F1 value by 6.00%. In addition, compared with the CGM2IR model in the Transformer-based model, the method of the present invention improves the Intra-F1 and Inter-F1 by 0.87% and 14.76%, respectively.

[0061] Table 2 Experimental results of the present invention on the CDR dataset.

[0062]

[0063] Note: The “-” in this table refers to data not given in the original paper.

[0064] The overall performance of the method of the present invention on the GDA dataset is excellent, with P, R, and F1 values ​​reaching 84.76%, 87.20%, and 85.80%, respectively, which is better than most existing models and reaches an advanced level. It is worth noting that the scale of the GDA dataset is nearly 50 times that of the CDR, so its sensitivity to the model is low, resulting in insignificant improvements in all evaluation indicators. Despite this, compared with the DocuNet model, which is the best Transformer-based model, the method of the present invention still improves the overall F1 value by 0.50%. In addition, compared with the CPT-RI model, which is the best prompt-based model, the method of the present invention improves the overall F1 value by 0.20%. This significant performance improvement is attributed to the effective use of the constituency constraint strategy and knowledge optimization prompt method by the method of the present invention. The constituency constraint strategy can eliminate the influence of redundant noise, allowing the model to focus on key information related to the target entity, and also simplifies the complex interaction process between entities. At the same time, injecting knowledge into the prompt template can enable the prompt template to learn rich factual knowledge of specific entity pairs, thereby improving the quality of relational reasoning. In addition, the hint prediction module can make full use of the powerful context understanding and generation capabilities of the pre-trained language model and can more accurately capture the relational information in the text.

[0065] Table 3 Experimental results of the present invention on the GDA dataset

[0066]

[0067] All test results verify the effectiveness and advancement of the method in the document-level biomedical relationship extraction task.

Claims

1. A document-level biomedical relationship extraction method based on a prompt optimization model, which is technically characterized by comprising a selection constraint module, a knowledge optimization prompt template construction module, and a prompt relationship prediction module, and is performed in the following steps: Step 1. Obtain a relation instance text in the document to be extracted through the selection constraint module, process the text with the selection constraint condition, and obtain the document sequence after the selection constraint. The specific steps are as follows: Step 1.1 Obtain a relation instance text in the document to be extracted and determine the sentence position set p where all mentions of the head entity are located h The set of sentence positions p where all mentions of the tail entity are located t , the head entity and the tail entity constitute the target entity pair that needs to be predicted in the document, that is, each relationship instance; Step 1.2: According to formula (1), obtain the set p of all sentences that have associated information with the head entity and the tail entity; p=p h ∪p t (1) Step 1.3: Set p is used as the selection constraint condition. Only the sentences in the document that exist in set p are retained through the judgment mechanism, thereby obtaining the document sequence after the selection constraint; Step 2. Construct a knowledge optimization prompt template through the knowledge optimization prompt template construction module. The specific steps are as follows: Step 2.1 Use ENTITY / TYPE MARKER to index the head and tail entity positions of the document sequence after the selection constraint, that is, insert special markers around the target entity in the document and add markers at the beginning of the text to obtain the text sequence after adding markers, denoted as x; Step 2.2 defines the prompt template as shown in (2): T(·)="·The relations between ET .h and ET .t is[MASK].” (2) in, ET .h and E.T. ·t Represents the type of head entity and tail entity respectively; Map x to T(·) to obtain the knowledge optimization prompt template shown in (3); x prompt =T(x)="x The relations between ET xh and ET xt is[MASK].” (3) The [MASK] is a mask mark; Step 3. Complete the relationship instance classification through the prompt relationship prediction module. The specific steps are as follows: Step 3.1 Construct the knowledge optimization prompt template x prompt Input the pre-trained language model T5 based on the Transformer architecture and calculate the hidden vector representation h of the mask mark [MASK] position [MASK] ; Step 3.2 generates the probability distribution of each possible label word at the mask [MASK] position, that is, performs relationship prediction through linear transformation and softmax function, thereby completing the relationship instance classification, that is, obtaining the relationship type of the document relationship instance; The linear transformation is shown in (4): z=h [MASK] ·W+b (4) Where W is the weight matrix in the output layer, and b is the bias vector in the output layer; The softmax function, as shown in formula (5), converts the unnormalized score vector z into a probability distribution, thereby generating a probability distribution P(y|x) of each possible label word at the mask position; P(y|x)=P([MASK]=v|x prompt )=softmax(z) (5) in, is the set of all label words.

Images