A hierarchical convolution-based biomedical relation extraction method and device and storage medium
By dividing sentences into five overlapping channels and combining token-level and channel-level convolutions, the problems of data imbalance and semantic overlap in long sentences in biomedical literature are solved, improving the accuracy and classification ability of biomedical relation extraction.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- GUIZHOU UNIV
- Filing Date
- 2023-04-25
- Publication Date
- 2026-05-22
AI Technical Summary
Existing technologies struggle to effectively handle semantic relationships between biomedical entities in long sentences within biomedical literature, particularly in the face of data imbalance and semantic overlap issues.
A hierarchical convolution method is adopted to divide the sentence into five overlapping channels. The text of each channel is encoded using the pre-trained language model PubMedBERT. Local and global semantic features are extracted by combining token-level and channel-level convolution. The complete semantic features of relation instances are obtained by max pooling and concatenation, and finally relation type classification is performed.
It improves the accuracy and classification ability of biomedical relation extraction, solves the problems of data imbalance and semantic overlap, and achieves state-of-the-art performance on multiple datasets.
Smart Images
Figure CN116414989B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to a method, apparatus, and storage medium for extracting biomedical relationships using hierarchical convolution, belonging to the fields of biomedical natural language processing and deep learning technology. Background Technology
[0002] Biomedical Relationship Extraction (BioRE) aims to extract semantic relationships between biomedical entities. Biomedical entity relationships can indicate protein-protein interactions (PPIs), drug-drug interactions (DDIs), and chemical-protein interactions (CPIs), which are highly valuable in uncovering potential reactions between biochemical entities. Therefore, BioRE tasks play a crucial role in practical applications such as dialogue systems, online treatment systems, and information recommendation, and are also widely used to support various biomedical research areas, such as protein complex identification, disease mechanisms, adverse drug reaction discovery, and drug repurposing. In the rapidly growing biomedical literature, it is impossible to manually discover these interactions from massive amounts of data. Therefore, the automatic extraction of predefined relationships between biomedical entities has attracted considerable attention in biomedical information processing.
[0003] Automatically extracting entity relations from sentences is a challenging task, primarily due to three problems. First, because relation types are asymmetric, every entity pair in the sentence needs to be evaluated. This leads to a severe data imbalance problem, as most entity pairs are negative examples. Second, the semantic expressions of all entity pairs within the same sentence overlap. Since they share the same contextual features within the sentence, it is difficult to distinguish them based solely on a simple representation of the sentence structure. Finally, long and complex sentences are widely used in biomedical literature, and a long sentence typically contains many biomedical entities.
[0004] In recent years, most BioRE research has employed deep learning models, such as convolutional neural networks, recurrent neural networks, attention mechanisms, and graph neural networks. Neural networks have the advantage of learning high-order abstract features and encoding semantic features in the raw input (e.g., the advantage of pre-trained language models). However, these models are typically implemented directly on the raw input, making them weaker at capturing contextual features and semantic dependencies related to entity order. In the general domain, several strategies have been proposed to address these issues, such as feature combination, multichannel, position embedding, entity indicators, and neural feature engineering. Despite significant success in the general domain, extracting relationships from biomedical literature is more challenging because it widely uses long sentences to express semantic relationships between biomedical entities. These long sentences often contain many biomedical entities, leading to more severe data imbalance and semantic overlap problems.
[0005] In related research, several strategies have been proposed to encode contextual features and semantic dependencies associated with named entities. Location embedding and entity indication are two traditional methods for addressing this issue. They enable the model to recognize the location of entities. However, they fail to encode the contextual features of the sentence. Feature combination and neural feature engineering are methods that use combined features to encode semantic dependencies. However, these methods heavily rely on prior knowledge. Multichannel methods divide the sentence into five channels, which has the advantage of capturing structural information about relational instances. However, since the abstract representation of each channel is learned independently, it is difficult to encode semantic dependencies between channels. Summary of the Invention
[0006] The technical problem to be solved by the present invention is to provide a method, apparatus and storage medium for extracting biomedical relationships using hierarchical convolution, so as to overcome the shortcomings of the prior art.
[0007] The technical solution of this invention is as follows:
[0008] Firstly, a hierarchical convolutional biomedical relation extraction method is provided, comprising the following steps: Step 1: Based on the structural information of the relation instance, the relation instance is divided into five overlapping channels with the entity position as the boundary; Step 2: The pre-trained language model PubMedBERT is used to encode the relation instance and the text of the five channels to obtain word vector representations; Step 3: Based on the channel positions divided in Step 1 and the semantic features obtained in Step 2, token-level convolution is used to extract local semantic features of the relation instance, and channel-level convolution is used to obtain global semantic dependency features between channels, and max pooling is performed on the local semantic features and global semantic dependency features; Step 4: Based on the channel positions divided in Step 1, the complementary parts of the local semantic features and global semantic features extracted in Step 3 are concatenated to obtain the complete semantics of the relation instance, and the complete semantics of the obtained relation instance is used to classify the relation type.
[0009] Furthermore, it also includes: Step 5, using a real biomedical relationship extraction dataset to verify validity.
[0010] Furthermore, step one also includes: inserting special markers on both sides of the entity of the relation instance, wherein the special markers are l 11 l 12 and l 21 l 22 , l 11 and l 12 In PPI tasks, it represents "B_PROT"; in DDI tasks, it represents "B_DRUG"; and in CPI tasks, it represents "B_CHEM". 21 and l 22In PPI tasks, it represents "E_PROT", in DDI tasks, it represents "E_DRUG", and in CPI tasks, it represents "E_CHEM".
[0011] Furthermore, in step two, clue text is added before the text of each channel to supplement it, forming a semantically complete input. Then, the pre-trained language model PubMedBERT is used to encode the supplemented text to obtain the word vector representation of each channel. Furthermore, word vectors at different positions are selected based on different channels to obtain the semantic features of each channel.
[0012] Specifically, in step three,
[0013] The method for extracting local semantic features from relation instances using token-level convolution is as follows:
[0014] Convolution is performed on the word vectors corresponding to each complementary channel to obtain
[0015] right Perform convolutional max pooling operations to obtain the local semantic features V of the complementary channel of the i-th channel. i η ;
[0016] The method for obtaining global semantic dependency features between channels using channel-level convolution is as follows:
[0017] Convolution is performed on the word vectors corresponding to each channel to obtain
[0018] right Perform convolutional max pooling operations to obtain the global semantic dependency features of the i-th channel.
[0019] Specifically, the method for concatenating the complementary parts of the local semantic features and global semantic dependency features extracted in step three to obtain the complete semantics of the relation instance is as follows:
[0020]
[0021] Here, V represents the complete semantics of a relation instance.
[0022] Furthermore, before classifying the relation type based on the complete semantics of the obtained relation instances, convolution and pooling operations are applied to the complete semantics of the obtained relation instances. The method for applying convolution and pooling operations to the complete semantics of the obtained relation instances is as follows:
[0023]
[0024] Among them, W i and bi These are learnable parameters. This represents the complete semantics of the relation instance after convolutional pooling.
[0025] Specifically, the method for classifying relation types based on the complete semantics of a relation instance is as follows:
[0026]
[0027]
[0028] Among them, Linear j (X)=W j X+b j Linear j (X) represents a linear layer, where j is 1 or 2. Indicates the type of relationship predicted by the model.
[0029] In a second aspect, a biomedical relation extraction apparatus based on hierarchical convolution is provided, characterized in that: the apparatus includes: a processor and a memory, wherein the memory stores computer program instructions suitable for execution by the processor, and the computer program instructions, when executed by the processor, cause the processor to perform the biomedical relation extraction method based on hierarchical convolution as described in any one of the first aspects.
[0030] Thirdly, a storage medium is provided, including computer program instructions stored on the storage medium, wherein the computer program instructions are executed by a processor to perform a biomedical relation extraction method of hierarchical convolution as described in any one of the first aspects.
[0031] The beneficial effects of this invention are as follows: Compared with the prior art, this invention includes two convolutional layers, implemented in a hierarchical manner. In this model, a sentence is divided into five channels using two named entities in a relation instance. Then, a traditional token-level convolutional operation is used to learn local contextual features from each channel. In particular, a channel-level convolutional operation is designed to learn the semantic dependencies between different channels. This gives the method the following advantages:
[0032] 1) This invention divides a sentence into five channels by two entities, enabling the model to recognize entity boundaries and learn the structural features between biomedical entities in long sentences;
[0033] 2) This invention employs token-level and channel-level convolutions to effectively learn contextual features and semantic dependencies related to entities, enhancing the distinguishability between entity pairs and helping to solve the label imbalance problem;
[0034] 3) This invention fully utilizes the structural features in relational instances, which can provide a more powerful nonlinear function approximator for BioRE, thereby enhancing the network's ability to classify relational instances and alleviating the data imbalance problem;
[0035] 4) This invention divides relation instances into multiple channels, and the semantics of each channel are complete and different, thus obtaining more potential knowledge, enabling this invention to make full use of the potential knowledge of the pre-trained language model;
[0036] 5) This invention was evaluated on seven datasets to support the extraction of PPI, DDI and CPI, and achieved best performance on all datasets. Attached Figure Description
[0037] Figure 1 A flowchart illustrating the implementation of at least one embodiment of this disclosure;
[0038] Figure 2 A diagram of a hierarchical convolutional biomedical relation extraction model provided for at least one embodiment of this disclosure;
[0039] Figure 3 A schematic block diagram of a deep text clustering apparatus based on a multi-core method that fuses multi-scale semantic representations, provided for at least one embodiment of this disclosure;
[0040] Figure 4 A schematic block diagram of a storage medium provided for at least one embodiment of this disclosure. Detailed Implementation
[0041] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings.
[0042] Example 1:
[0043] As attached Figures 1-2 As shown, a method for extracting biomedical relationships using hierarchical convolution is described, the method comprising the following steps:
[0044] Step 1: Based on the structural information of the relation instance, divide the relation instance into five overlapping channels with the entity location as the boundary. Step 2: Use the pre-trained language model PubMedBERT to encode the text of the relation instance and the five channels to obtain word vector representations. Step 3: Based on the channel positions obtained in Step 1 and the semantic features obtained in Step 2, use token-level convolution to extract local semantic features from the relation instance, and use channel-level convolution to obtain global semantic dependency features between channels. Perform max pooling on the local and global semantic dependency features. Step 4: Based on the channel positions obtained in Step 1, concatenate the complementary parts of the local and global semantic features extracted in Step 3 to obtain the complete semantics of the relation instance. Classify the relation type based on the obtained complete semantics.
[0045] In step one, based on the structural features of the relation instance, the relation instance is divided into five overlapping channels, allowing the same word to learn semantic representations in different channels. Simultaneously, to better identify the positional structural information of entities, in addition to dividing the relation instance into five channels, special markers are inserted on both sides of the entity within the relation instance. The specific operation is formalized as follows:
[0046] Original sentence S: S={x1,x2,…,x s ,…,x t ,…,x u ,…,x v ,…,x n}, where {x s ,…,x t} represents entity 1, {x u ,…,x v Let} represent entity 2, and n be the sentence length. Each input sentence is divided into five channels using two named entities. This invention primarily studies the semantic dependencies of long sentences in biomedical literature. Therefore, instead of five independent channels, each sentence is divided into five overlapping channels, allowing each channel to incorporate more sentence information. The channel segmentation is as follows:
[0047] C1 = {x1,…,x} n}
[0048] C2={x s ,…,x n}
[0049] C3={x t+1 ,…,x n}
[0050] C4={x u ,…,x n}
[0051] C5={x v+1 ,…,x n}
[0052] Because different channels contain different sentence context features, it is possible to learn semantic dependencies at different granularities. Compared to learning abstract features from the entire sentence, learning abstract features from these channels helps to perceive entity boundaries and encode local context features related to two named entities.
[0053] At the same time, special markers are inserted on both sides of the entity to mark its location and highlight its information. Among them, l 11 l 12 and l 21 l 22 These are two sets of special markers, each representing a different text. In PPI tasks, they represent “B_PROT” and “E_PROT”, in DDI they represent “B_DRUG” and “E_DRUG”, and in CPI they represent “B_CHEM” and “E_CHEM”.
[0054] To understand this process, consider this example: Given the input sentence "The retinoblastoma protein associates with the Protein phosphatase type 1 Catalytic subunit." with an entity pair "retinoblastoma protein" and "Protein phosphatase type 1", the resulting five-channel segmentation is as follows:
[0055]
[0056] For relational instances, special markers are inserted around the entity to indicate the entity's boundaries. The specific insertion format is as follows:
[0057]
[0058]
[0059] In step two, to ensure the integrity of semantic information for each channel, the text for each channel is supplemented to form a semantically complete input. Then, the pre-trained language model PubMedBERT is used to map each word into a distributed representation. Furthermore, word vectors at different positions are selected based on different channels to obtain the semantic features of each channel. As mentioned above, although segmenting relation instances into five channels based on entity position helps learn more semantic dependencies, it results in incomplete semantic information for each channel. Therefore, for all channels, the text C in each channel... i Add clue text P before i This ensures the semantic information of each channel is complete. Next, the pre-trained language model PubMedBERT is used to encode all channels:
[0060]
[0061] Therefore, the encoding of all channels can be represented as At the same time, for relation instances with inserted special identifiers Another BERT is used for encoding as well:
[0062]
[0063] Among them, each h i Corresponding to each x i An abstract representation. After encoding, based on the structural characteristics of the relation instance, and using the entity location as the boundary, [the following is defined]. It is divided into complementary channels for each channel in step one, that is:
[0064]
[0065]
[0066]
[0067]
[0068]
[0069] In step two, to ensure the semantic integrity of each channel, the text of each channel is supplemented to form a semantically complete input. To understand this process, an example is given: for instance, the text clue used is "This is the ith channel, there are (is)2(1) entities(entity), the text is:". Next, the text of all channels and the relation instance text are input into the pre-trained language model PubMedBERT to obtain the encoded representation of each channel, and the representation of the relation instance output is segmented according to the non-position of the channels.
[0070] In step three, the method of extracting local semantic features from relation instances using token-level convolution is as follows:
[0071] Convolution is performed on the word vectors corresponding to each complementary channel to obtain
[0072] right Perform convolutional max pooling operations to obtain the local semantic features V of the complementary channel of the i-th channel. i η ;
[0073] in, W represents the word vector corresponding to each complementary channel. c b and f are learnable parameters. c It is a ReLU activation function;
[0074] The method for obtaining global semantic dependency features between channels using channel-level convolution is as follows:
[0075] Convolution is performed on the word vectors corresponding to each channel to obtain
[0076] right Perform convolutional max pooling operations to obtain the global semantic dependency features of the i-th channel.
[0077] in, W represents the word vector corresponding to each complementary channel. c b and f are learnable parameters. c It is a ReLU activation function.
[0078] Performing the same operation on each channel yields the features for each channel. Therefore, after token-level convolution and pooling, and channel-level convolution and pooling, the following abstract representation can be obtained:
[0079]
[0080]
[0081] In step four, based on the channel position obtained in step three, complementary parts of the local and global semantic features extracted in step three are concatenated to obtain the complete semantics of the relation instance for relation type classification. The specific operation method is as follows:
[0082]
[0083] To further enhance the effectiveness of the features, convolution and pooling operations are applied again based on the features from all channels. Specifically, before classifying the relation type from the complete semantics of the obtained relation instances, convolution and pooling operations are applied to the complete semantics of the obtained relation instances. The method for applying convolution and pooling operations to the complete semantics of the obtained relation instances is as follows:
[0084]
[0085] Among them, W i and b i These are learnable parameters. This represents the complete semantics of the relation instance after convolutional pooling.
[0086] Finally, the relation type is classified based on the obtained semantic features. The method for classifying the relation type based on the complete semantics of the obtained relation instances is as follows:
[0087]
[0088]
[0089] Among them, Linear j (X)=W j X+b j Linear j (X) represents a linear layer, where j is 1 or 2. Indicates the type of relationship predicted by the model.
[0090] Finally, step five is performed to validate the effectiveness using a real biomedical relationship extraction dataset.
[0091] Experiments were conducted using the AImed, BioInfer, IEPA, HPRD50, LLL, DDI, and ChemProt datasets to verify the effectiveness of this invention on PPI, DDI, and CPI tasks. First, the predicted relation types were obtained through steps one through four. Then, performance was evaluated based on the standard relation types to verify the effectiveness of the method. The evaluation criteria used were precision (P), recall (R), and F1 score, where F = 2*P*R / (P+R).
[0092] Example 2:
[0093] At least one embodiment of this disclosure also provides a biomedical relation extraction apparatus based on hierarchical convolution. Figure 3 This is a schematic block diagram of a hierarchical convolutional biomedical relation extraction apparatus provided in at least one embodiment of the present disclosure. The hierarchical convolutional biomedical relation extraction apparatus includes a processor and a memory, wherein the memory stores computer program instructions suitable for execution by the processor, the computer program instructions being executed by the processor causing the processor to perform the hierarchical convolutional biomedical relation extraction method as described in any one of Embodiment 1.
[0094] For example, the processor may be a central processing unit (CPU), a graphics processing unit (GPU), a tensor processor (TPU), or other processing units with data processing and / or instruction execution capabilities. For instance, the processor can be implemented as a general-purpose processor, or as a microcontroller, microprocessor, digital signal processor, dedicated image processing chip, or field-programmable logic array, etc. For example, the memory may include at least one type of volatile memory and non-volatile memory, such as read-only memory (ROM), hard disk, flash memory, etc. Accordingly, the memory can be implemented as one or more computer program products, which may include various forms of computer-readable storage media on which one or more computer program instructions may be stored. The processor can execute the program instructions to perform any of the hierarchical convolution biomedical relation extraction methods provided in at least one embodiment of this disclosure. The memory can also store various other applications and various data, such as various data used and / or generated by the applications.
[0095] Example 3:
[0096] At least one embodiment of this disclosure also provides a storage medium (e.g., a non-transitory storage medium). Figure 4 This is a schematic block diagram of a storage medium provided in at least one embodiment of this disclosure. Figure 4 As shown, the storage medium includes computer program instructions stored on the storage medium. When executed by a processor, the computer program instructions perform the hierarchical convolution biomedical relation extraction method provided in at least one embodiment of this disclosure.
[0097] For example, storage media can take many forms, including tangible storage media, carrier media, or physical transmission media. Stable storage media can include optical discs or magnetic disks, and other storage systems used in computers or similar devices that enable the system components described in the figure. Unstable storage media can include dynamic memory, such as the main memory of a computer platform. Tangible transmission media can include coaxial cables, copper cables, and optical fibers, such as the lines that form a bus within a computer system. Carrier transmission media can transmit electrical signals, electromagnetic signals, acoustic signals, or optical signals. These signals can be generated by radio frequency or infrared data communication methods. Typical storage media (e.g., computer-readable media) include hard disks, floppy disks, magnetic tapes, and any other magnetic media; CD-ROMs, DVDs, DVD-ROMs, and any other optical media; punched cards and any other physical storage media containing a punch pattern; RAM, PROMs, EPROMs, FLASH-EPROMs, and any other memory chips or magnetic tapes; carrier waves for transmitting data or instructions, cables or connection devices for transmitting carrier waves, and any other data that can be read using computer program instructions (e.g., program code) and / or by a computer.
[0098] Computer program instructions (e.g., program code) for performing the operations of this disclosure can be written in one or more programming languages or a combination thereof, including but not limited to object-oriented programming languages such as Java, Smalltalk, and C++, as well as conventional procedural programming languages such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).
[0099] In some examples, the functionality described in at least one embodiment of this disclosure may also be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), and so on.
[0100] All aspects not detailed herein are well-known to those skilled in the art. Finally, it should be noted that the above embodiments are merely illustrative of the technical solutions of this invention and not intended to limit it. Although the invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of this invention without departing from the spirit and scope of the invention, and all such modifications and substitutions should be covered within the scope of the claims of this invention.
Claims
1. A method for extracting biomedical relationships using hierarchical convolution, characterized in that, The method includes the following steps: Step 1: Based on the structural information of the relation instance, divide the relation instance into five overlapping channels with the entity position as the boundary; Step 2: Encode the text of the relation instance and the five channels using the pre-trained language model PubMedBERT to obtain word vector representations; Step 3: Based on the channel positions divided in Step 1 and the semantic features obtained in Step 2, extract local semantic features of the relation instance using token-level convolution, and obtain global semantic dependency features between channels using channel-level convolution, and perform max pooling operation on the local semantic features and global semantic dependency features; Step 4: Based on the channel positions divided in Step 1, concatenate the complementary parts of the local semantic features and global semantic features extracted in Step 3 to obtain the complete semantics of the relation instance, and classify the relation type based on the obtained complete semantics of the relation instance; The key feature is that, in step two, clue text is added before the text of each channel to supplement it, forming a semantically complete input. Then, the pre-trained language model PubMedBERT is used to encode the supplemented text to obtain the word vector representation of each channel. Furthermore, word vectors at different positions are selected based on different channels to obtain the semantic features of each channel.
2. The method for extracting biomedical relationships using hierarchical convolution according to claim 1, characterized in that, It also includes: Step 5, using real biomedical relationship extraction datasets for validity verification.
3. The method for extracting biomedical relationships using hierarchical convolution according to claim 1, characterized in that, Step one further includes: inserting special markers on both sides of the entity of the relation instance, the special markers being... , and , , and In PPI tasks, it represents "B_PROT"; in DDI tasks, it represents "B_DRUG"; and in CPI tasks, it represents "B_CHEM". and In PPI tasks, it represents "E_PROT", in DDI tasks, it represents "E_DRUG", and in CPI tasks, it represents "E_CHEM".
4. The method for extracting biomedical relationships using hierarchical convolution according to claim 3, characterized in that, In step three The method for extracting local semantic features from relation instances using token-level convolution is as follows: Convolution is performed on the word vectors corresponding to each complementary channel to obtain ; right Perform convolutional max pooling operations to obtain the local semantic features of the complementary channel of the i-th channel. ; The method for obtaining global semantic dependency features between channels using channel-level convolution is as follows: Convolution is performed on the word vectors corresponding to each channel to obtain ; right Perform convolutional max pooling operations to obtain the global semantic dependency features of the i-th channel. .
5. The method for extracting biomedical relationships using hierarchical convolution according to claim 4, characterized in that, The method for concatenating the complementary parts of the local semantic features and global semantic dependency features extracted in step three to obtain the complete semantics of the relation instance is as follows: ， Here, V represents the complete semantics of a relation instance.
6. The method for extracting biomedical relationships using hierarchical convolution according to claim 5, characterized in that, Before classifying the relation type based on the complete semantics of the obtained relation instances, convolution and pooling operations are applied to the complete semantics of the obtained relation instances. The method for applying convolution and pooling operations to the complete semantics of the obtained relation instances is as follows: ， in, and These are learnable parameters. This represents the complete semantics of the relation instance after convolutional pooling.
7. The biomedical relation extraction method based on hierarchical convolution according to claim 6, characterized in that, The method for classifying relation types based on the complete semantics of a relation instance is as follows: ， ， in, , This represents a linear layer, where j is 1 or 2. Indicates the type of relationship predicted by the model.
8. A biomedical relation extraction device based on hierarchical convolution, characterized in that: The apparatus includes a processor and a memory, wherein the memory stores computer program instructions suitable for execution by the processor, the computer program instructions being executed by the processor causing the processor to perform the hierarchical convolution biomedical relation extraction method as described in any one of claims 1-7.
9. A storage medium comprising computer program instructions stored on the storage medium, wherein, The computer program instructions are executed by the processor to perform the hierarchical convolution biomedical relation extraction method as described in any one of claims 1-7.