A method, apparatus and storage medium for entity-oriented biomedical relation extraction

Abstract

This invention discloses an entity-oriented biomedical relation extraction method, apparatus, and storage medium. The method includes the following steps: Step 1: Extracting global semantic information of relation instances and extracting entity-oriented semantic information of relation instances; Step 2: Calculating the difference vectors corresponding to each entity in an entity pair using the global semantic information and the entity-oriented semantic information; Step 3: Combining the global semantic information of relation instances and the difference vectors corresponding to each entity in an entity pair to complete relation classification. This addresses the problem that existing technologies cannot achieve correct classification when entity pairs share the same context.

Description

Technical Field

[0001] This invention relates to an entity-oriented biomedical relation extraction method, apparatus, and storage medium, belonging to the fields of biomedical natural language processing and deep learning technology. Background Technology

[0002] In recent years, Biomedical Natural Language Processing (BioNLP) has shown broad application prospects in the automatic extraction of professional knowledge from biomedical literature. Biomedical Relation Extraction (BioRE) utilizes contextual information in biomedical text to extract semantic relationships between entity pairs. This is a fundamental and challenging topic in biomedical NLP and has received significant attention. In numerous applications in biomedical research, such as protein-protein interaction (PPI) analysis for building protein complex discovery networks and revealing disease mechanisms, BioRE is a key aspect of constructing biomedical knowledge and a core component of knowledge graphs. Furthermore, it has important applications in fields such as assisted diagnosis, meta-analysis, and new drug research.

[0003] BioRE in text is the process of identifying whether two entities in a given sentence interact, and if so, classifying the interaction type. If a sentence contains more than two entities, then all possible entity pairs in the sentence should be computed separately. To date, various methods have been researching how to obtain global semantic information from input instances with predefined entity labels. Deep learning-based methods have become the mainstream approach for BioRE due to their ability to handle raw input, automatically extract features, and perform end-to-end learning. The application of neural networks in these models has enabled them to achieve significant performance on various BioRE tasks. In particular, CNN- and RNN-based methods are increasingly being used. In recent years, with BioBERT learning domain-related knowledge through pre-training on a large-scale biomedical corpus, the good performance of pre-trained language models (PLMs) on many BioNLP tasks has attracted attention. Although BioBERT and its variants / extensions have achieved great success on BioRE, most of these models are developed based on information from a single input instance; in other words, these methods cannot effectively handle the case where a sentence contains multiple entity pairs, i.e., the proximity semantic problem.

[0004] Here are two examples to illustrate the issue of near-synonyms, such as Figure 2As shown. On the one hand, the proximity semantic problem arises from entity overlap. For sentence S1, there are 8 entities, so 28 pairs of relation instances are considered for interaction. However, only 2 instances are annotated as positive relation interactions. Furthermore, there is a serious near-synonymous semantic problem due to the use of overlapping entities. For example, there is an interaction between entity “[IFN-gamma SC1]3” and entity “[IFN-gamma receptor alpha chain]6”. However, due to entity overlap, there is no interaction between “[IFN-gamma SC1]3” and entities “[IFN-gamma]4” or “[IFN-gamma receptor]5”. On the other hand, near-synonymous semantic problems can also arise from different meanings of entities determined by context. For the second sentence, it contains 6 entities and 15 relation instances, of which only 1 instance is annotated as a positive relation interaction. There are three identical entities (“[EPO]2”, “[EPO]4”, and “[EPO]5”). There is an interaction relationship between entities “[erythropoietin(EPO)receptor]3” and “[EPO]4”, but no interaction relationship between entities “[erythropoietin(EPO)receptor]3” and “[EPO]5”. Although entities “[EPO]4” and “[EPO]5” are close to each other, their interactions with other entities are quite different.

[0005] In biological relation extraction, a common approach is to replace entities in a sentence with special tags and then feed them as input to pre-trained language models (PLMs). This method helps extract global semantic information from the input, thus achieving optimal performance in relation classification. However, a major limitation of existing research is that it ignores the issue of synonymous semantics. In the field of natural language processing, Wu et al. utilized PLMs and integrated target entity information to handle relation extraction tasks. Wu et al.'s approach considered only one input instance, inputting the text along with entities 1 and 2 into the pre-trained language model, adding information about entities 1 and 2 to the global semantics. Although they achieved success in some cases, because the pre-trained language model itself can learn the interaction information between entities 1, 2, and the text, the model's improvement is relatively limited in the following situations:

[0006] First, the developed model may become confusing in sentences containing multiple entity pairs. This is because when these entity pairs share the same context, such as... Figure 3As shown, the entity "EPO" is mentioned three times in sentence S2. However, it's important to note that entity "[EPO1]2" is a component of entity "[erythropoietin(EPO)receptor]3", entity "[EPO]4" interacts with entity "[erythropoietin(EPO)receptor]3", and entity "[EPO3]5" is part of entity "[EPOmimetic petide]6" and parallels "[EPO]4". This leads to a near-synonymous semantic problem, resulting in highly similar extracted information with limited discriminative information, making correct classification difficult.

[0007] Secondly, in biological relation extraction, entities may contain varying amounts of relevant information, and not all of this information is necessary to represent the relationships between them. Treating all information equally without proper distinction may result in semantically unsatisfactory representations. Summary of the Invention

[0008] The technical problem to be solved by the present invention is to provide a biomedical relationship extraction method, device and storage medium based on entity orientation, so as to overcome the shortcomings of the prior art.

[0009] The technical solution of this invention is as follows:

[0010] Firstly, a biomedical relation extraction method based on entity orientation is provided, the method comprising the following steps:

[0011] Step 1: Extract the global semantic information of the relation instance and extract the entity-oriented semantic information of the relation instance;

[0012] Step 2: Calculate the difference vectors corresponding to each entity in the entity pair using global semantic information and entity-oriented semantic information;

[0013] Step 3: Combine the global semantic information of the relation instance with the difference vectors corresponding to each entity in the entity pair to complete the relation classification.

[0014] Specifically, the method for extracting global semantic information of relation instances from blocks is as follows: special markers are inserted into the sentences of relation instances to identify entities, then the information is fed into a pre-trained language model for semantic encoding, and finally the CLS vector of the pre-trained language model is used as global semantic information.

[0015] Specifically, the method for extracting entity-oriented semantic information of relation instances is as follows: entity identification, masking, and clue insertion operations are performed on different entities in the relation instance, and then the data is input into a pre-trained language model. Convolution and pooling operations are used to further extract entity features to represent entity-oriented semantic information.

[0016] Specifically, the method for calculating the difference vector is as follows:

[0017]

[0018]

[0019] Among them, O g For global semantic information, It is semantic information oriented towards entity e1. This refers to semantic information oriented towards entity e2. ⊕ represents vector concatenation, and - represents vector subtraction. It is the difference vector corresponding to entity e1. It is the difference vector corresponding to entity e2.

[0020] Specifically, the method for classifying relations by combining the global semantic information of relation instances and the difference vectors corresponding to each entity in an entity pair is as follows:

[0021] S01. Concatenate the global semantic information and the difference vectors corresponding to each entity in the entity pair.

[0022]

[0023] S02. Use MLP networks to classify relation types:

[0024] y` = Softmax(MLP(MLP(E) e )+MLP(O g )));

[0025] Among them, E e y' represents the concatenated features, and y' represents the classification result.

[0026] The second aspect: provides an entity-oriented biomedical relation extraction device, the device comprising: a processor and a memory, wherein the memory stores computer program instructions suitable for execution by the processor, the computer program instructions being executed by the processor causing the processor to perform the entity-oriented biomedical relation extraction method as described in any one of the first aspects.

[0027] Third aspect: A storage medium is provided, including computer program instructions stored on the storage medium, wherein the computer program instructions are executed by a processor to perform the entity-oriented biomedical relation extraction method as described in any one of the first aspects.

[0028] The beneficial effects of this invention are as follows: Compared with the prior art, this invention proposes to first grasp the overall information through global semantics, and then learn entity-oriented semantic information for each entity, thus increasing the difference information between entity pairs. Secondly, it proposes an entity difference vector to learn the different representations between entity pairs. This vector captures the semantics of the target entity and enhances the distinguishability between entity pairs. Furthermore, it combines global semantic information and entity-oriented semantic information for relationship classification, ensuring sufficient distinguishability between entity pairs when they share the same context, thus achieving correct classification. Attached Figure Description

[0029] Figure 1 This is a flowchart of the present invention;

[0030] Figure 2 This is an example diagram of biomedical relationship extraction in the background technology;

[0031] Figure 3 Here is another example diagram of biomedical relationship extraction in the background technology;

[0032] Figure 4 This is a schematic diagram of the present invention;

[0033] Figure 5 This is a technical roadmap of the extraction model of the present invention;

[0034] Figure 6 A schematic block diagram of an entity-oriented biomedical relationship extraction device provided for at least one embodiment of this disclosure;

[0035] Figure 7 A schematic block diagram of a storage medium provided for at least one embodiment of this disclosure. Detailed Implementation

[0036] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings.

[0037] Example 1: As shown in the attached document Figure 1 , 4 As shown in Figure 5, an entity-oriented biomedical relation extraction method is provided, the method comprising the following steps:

[0038] Step 1: Extract global semantic information and entity-oriented semantic information of relation instances. Specifically, the method for extracting global semantic information of relation instances is as follows: Special markers are inserted into the sentence of the relation instance to identify entities, then the data is fed into a pre-trained language model for semantic encoding, and finally, the CLS vector of the pre-trained language model is used as the global semantic information. The method for extracting entity-oriented semantic information of relation instances is as follows: Entity identification, masking, and cue insertion operations are performed on different entities in the relation instance, then the data is input into a pre-trained language model, and convolution and pooling operations are used to further extract entity features to represent entity-oriented semantic information.

[0039] To understand this step, given a sentence X = {x1, x2, ..., x...} s ,…,x t ,…,x u ,…,x v ,…,x n}, where {x s ,…,x t} represents entity e1, {x u ,…,x v} represents entity e2, and n is the sentence length. To explicitly capture the positional information of entities in the current entity pair, four predefined special markers are inserted at the beginning and end positions of entities e1 and e2.

[0040] Taking the sentence "The sequences that confer on FGF-7 its specific binding to KGFR have not been identified." as an example, the sentence contains two entities, "FGF-7" and "KGFR". After inserting special identifiers on both sides of the entities, the relation instance is: "X in : The sequences that confer on[B_PROT_1]FGF-7[E_PROT_1]its specific binding to[B_PROT_2]KGFR[E_PROT_2]have not been identified.". for and It can be represented as " [B_PROT_1]The first entity is FGF-7:Thesequences that confer on[MASK]its specific binding to KGFR have not been identified.[E_PROT_1]"," [B_PROT_2]The second entity is KGFR:The sequences that confer on FGF-7its specific binding to[MASK]have not been identified.[E_PROT_2]".

[0041] Therefore, the input instance used to extract global semantic information of the current entity can be represented as follows:

[0042] X in ={x1,x2,…,l 11 ,x s ,…,x t ,l 12 ,…,l 21 ,x u ,…,x v ,l 22 ,…,x n}

[0043] Among them, l 11 l 12 These represent predefined special markers inserted at the beginning and end of the first entity e1, respectively. 21 l 22 The tags represent the beginning and end of the second entity e2. By inserting predefined special tags next to the entity boundaries, the deep neural network can focus on the current entity pair. Each predefined special tag is encoded with the same representation and can be regarded as an "anchor" of the current entity pair. To extract global semantic information, a PLM (e.g., BERT) is used as the sentence encoder. This process can be formalized as:

[0044]

[0045] [CLS] and [SEP] are special markers in pre-trained language models. It is considered to cover all semantic information of the current input and is commonly used for classification tasks. L represents the layer number in the BERT model. The [CLS] vector of the topmost layer in the BERT model is used as a representation of the global semantic information of the input instance because research has found that morphological and syntactic information at lower levels is represented at lower levels, while higher-level and more semantic text features are learned at higher levels, denoted as O. g .

[0046] Secondly, to extract sentence representations from the target entities in the current input instance, different input instances are used for different entities. For entity e1, the entity in the sentence is first replaced with [MASK]. Second, an entity cue is inserted at the beginning of the sentence. Finally, two predefined special tags l are inserted at both ends of the sentence. 11 and l 12 Therefore, the input instance of entity e1 can be obtained as follows:

[0047]

[0048] in, The entity cues are represented by s, t and u, v, which represent the start and end positions of entities e1 and e2 in the sentence, respectively. Replacing entities in the sentence with [MASK] allows the model to spontaneously learn the positional information of entities within the sentence. Entity cues are crucial for increasing entity-sentence interaction and ensuring the model doesn't lose original entity information. The use of special tags allows the model to better understand information about different entities. Similarly, for entity e2, the same input instance can be obtained.

[0049]

[0050] for and Using another BERT model for encoding, it can be represented as:

[0051]

[0052]

[0053] in and Representing input instances respectively and The length. [CLS] and [SEP] are two special markers in BERT. L represents the number of BERT layers. This represents the hidden state (dimensional H) of the last layer (L-th layer). To obtain more information about the target entity, all information from the last layer of the BERT model is used, not just the vector representation corresponding to [CLS]. A CNN is also used to process local structural information. For the input matrix... Convolution operation includes a filter f k The application window generates a new feature. The convolution operation is shown below:

[0054]

[0055]

[0056] δ is the ReLU activation function, and b is a learnable parameter. Next, max pooling is used to further maximize the learned features, and an MLP network is used to map them to a fixed dimension.

[0057]

[0058]

[0059]

[0060] MLP(X)=δ(WX+b)

[0061] For entity e2, the same operation is used to obtain the entity's directional semantic information.

[0062] Step 2: Calculate the difference vectors for each entity in the entity pair using global semantic information and entity-oriented semantic information. These difference vectors represent the semantics that capture the target entity, enhancing the distinguishability between entity pairs and better adapting to biomedical relation extraction tasks. The calculation method for the difference vectors is as follows:

[0063]

[0064]

[0065] Among them, O g For global semantic information, It is semantic information oriented towards entity e1. This refers to semantic information oriented towards entity e2. ⊕ represents vector concatenation, and - represents vector subtraction. It is the difference vector corresponding to entity e1. It is the difference vector corresponding to entity e2.

[0066] Step 3: Combine the global semantic information of the relation instance with the difference vectors corresponding to each entity in the entity pair to complete the relation classification. The relation classification method is as follows:

[0067] S01. Concatenate the global semantic information and the difference vectors corresponding to each entity in the entity pair.

[0068]

[0069] S02. Use MLP networks to classify relation types:

[0070] y` = Softmax(MLP(MLP(E) e )+MLP(O g )));

[0071] Among them, E e y' represents the concatenated features, and y' represents the classification result.

[0072] Example 2:

[0073] At least one embodiment of this disclosure also provides an entity-oriented biomedical relationship extraction device. Figure 6 This is a schematic block diagram of an entity-oriented biomedical relation extraction apparatus provided in at least one embodiment of the present disclosure. The entity-oriented biomedical relation extraction apparatus includes a processor and a memory, wherein the memory stores computer program instructions suitable for execution by the processor, the computer program instructions, when executed by the processor, causing the processor to perform the entity-oriented biomedical relation extraction method as described in any one of Embodiment 1.

[0074] For example, the processor may be a central processing unit (CPU), a graphics processing unit (GPU), a tensor processor (TPU), or other processing units with data processing and / or instruction execution capabilities. For instance, the processor can be implemented as a general-purpose processor, or as a microcontroller, microprocessor, digital signal processor, dedicated image processing chip, or field-programmable logic array, etc. For example, the memory may include at least one type of volatile memory and non-volatile memory, such as read-only memory (ROM), hard disk, flash memory, etc. Accordingly, the memory can be implemented as one or more computer program products, which may include various forms of computer-readable storage media on which one or more computer program instructions may be stored. The processor can execute the program instructions to perform any of the entity-oriented biomedical relation extraction methods provided in at least one embodiment of this disclosure. The memory can also store various other applications and various data, such as various data used and / or generated by the applications.

[0075] Example 3:

[0076] At least one embodiment of this disclosure also provides a storage medium (e.g., a non-transitory storage medium). Figure 7 This is a schematic block diagram of a storage medium provided in at least one embodiment of this disclosure. Figure 7 As shown, the storage medium includes computer program instructions stored on the storage medium. When executed by a processor, the computer program instructions perform the entity-oriented biomedical relation extraction method provided in at least one embodiment of this disclosure.

[0077] For example, storage media can take many forms, including tangible storage media, carrier media, or physical transmission media. Stable storage media can include optical discs or magnetic disks, and other storage systems used in computers or similar devices that enable the system components described in the figure. Unstable storage media can include dynamic memory, such as the main memory of a computer platform. Tangible transmission media can include coaxial cables, copper cables, and optical fibers, such as the lines that form a bus within a computer system. Carrier transmission media can transmit electrical signals, electromagnetic signals, acoustic signals, or optical signals. These signals can be generated by radio frequency or infrared data communication methods. Typical storage media (e.g., computer-readable media) include hard disks, floppy disks, magnetic tapes, and any other magnetic media; CD-ROMs, DVDs, DVD-ROMs, and any other optical media; punched cards and any other physical storage media containing a punch pattern; RAM, PROMs, EPROMs, FLASH-EPROMs, and any other memory chips or magnetic tapes; carrier waves for transmitting data or instructions, cables or connection devices for transmitting carrier waves, and any other data that can be read using computer program instructions (e.g., program code) and / or by a computer.

[0078] Computer program instructions (e.g., program code) for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including but not limited to object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0079] In some examples, the functionality described in at least one embodiment of this disclosure may also be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), and so on.

[0080] All aspects not detailed herein are well-known to those skilled in the art. Finally, it should be noted that the above embodiments are merely illustrative of the technical solutions of this invention and not intended to limit it. Although the invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of this invention without departing from the spirit and scope of the invention, and all such modifications and substitutions should be covered within the scope of the claims of this invention.

Claims

1. An entity-oriented biomedical relation extraction method, characterized in that: The method comprises the following steps: Step one: extracting global semantic information of the relation instance, and extracting entity-oriented semantic information of the relation instance; Step two: calculating the difference vector corresponding to each entity in the entity pair through the global semantic information and the entity-oriented semantic information; Step three: combining the global semantic information of the relation instance and the difference vector corresponding to each entity in the entity pair to complete relation classification; The method for extracting the global semantic information of the relation instance is: inserting special markers to identify entities in the sentence of the relation instance, then inputting into a pre-trained language model for semantic encoding, and finally using the CLS vector of the pre-trained language model as the global semantic information; The method for extracting the entity-oriented semantic information of the relation instance is: performing entity identification, MASK and clue insertion operations on different entities in the relation instance, then inputting into a pre-trained language model, and using convolution and pooling operations to further extract entity features for representing the entity-oriented semantic information; The method for calculating the difference vector is: ， wherein, is global semantic information, is entity oriented semantic information, is entity oriented semantic information, is a vector concatenation operation, is a vector subtraction operation, is an entity corresponding difference vector, is an entity corresponding difference vector; The method for combining the global semantic information of the relation instance and the difference vector corresponding to each entity in the entity pair to complete relation classification is: S01, splicing the global semantic information and the difference vector corresponding to each entity in the entity pair, ； S02, using an MLP network to classify the relation type: ； wherein, denotes a stitching feature, denotes a classification result.

2. An entity-oriented biomedical relation extraction apparatus, characterized by comprising: The device comprises a processor and a memory, wherein the memory stores computer program instructions suitable for the processor, and the computer program instructions are run by the processor to make the processor execute the entity-oriented biomedical relation extraction method of claim 1.

3. A storage medium comprising computer program instructions stored on the storage medium, wherein, The computer program instructions are run by the processor to execute the entity-oriented biomedical relation extraction method of claim 1.

Images