Document-level biomedical relationship extraction method

Abstract

The invention is applicable to the technical field of biomedical natural language processing, and provides a document-level biomedical relationship extraction method, which comprises the following steps of: performing dual-channel modeling by fusing semantic distance and sentence distance, distributing differentiated weights for entity mentions, and dynamically aggregating mention features to generate enhanced entity representation; the problem that static entity representation neglects the mentioned specificity and distance dependence is solved; a heterogeneous graph containing entities and mentions is constructed through a multi-relation graph attention reasoning network, an independent feature channel is designed for each relation, global semantic dependency is captured in combination with dynamic relation weights, and prediction deviation is corrected by fusing local interaction features and global semantic information through an initial reasoning-enhanced verification dual-stage mechanism in dual-stage reasoning. The two reasoning mechanisms are different in emphasis and complement each other, so that the specificity and robustness of relation reasoning are effectively improved, and the comprehensive performance of document-level biomedical relation extraction is improved.

Description

Technical Field

[0001] This invention belongs to the field of biomedical natural language processing technology, and particularly relates to a document-level biomedical relation extraction method. Background Technology

[0002] With the rapid development of biomedical research, the number of documents in databases such as PubMed is growing exponentially. Statistics show that PubMed adds over 1 million new documents annually. These unstructured texts contain a wealth of crucial knowledge, such as the relationship between chemical substances and disease treatment or induction, and the mechanisms linking genes to diseases. Document-level biomedical relation extraction, as a core task of biomedical information extraction, aims to infer the relationships between entity pairs from complete documents. Compared to sentence-level extraction, it faces three major challenges: first, ambiguity due to multiple mentions of the same entity, where the semantic function and relational contribution differ across mentions; second, long-distance dependencies across sentences, where most key relations require the context of multiple sentences for inference; and third, relation specificity and global consistency, where the semantic features of different relation types differ significantly, and the inference results must conform to the overall logic of the document. The goal of biomedical relation extraction is to determine the relationships between biomedical entity pairs from unstructured biomedical texts. Most previous methods focused on predicting relationships between entities at the sentence level, i.e., sentence-level relation extraction. However, from a realistic perspective, relational facts in biomedical texts are mostly expressed jointly through multiple sentences. Therefore, many recent studies have shifted their attention to biomedical document-level relation extraction, which utilizes the entire input document to extract intra-sentence entity pair relations and cross-sentence entity pair relations. These studies can be categorized into sequence-based, graph-based, and Transformer-based models.

[0003] Sequence-based methods typically rely on BiLSTM, attention mechanisms, and other techniques to process text sequences. While they can capture local semantics, they cannot effectively model global entity relationships and are weak in handling cross-sentence relationships. To overcome this limitation and improve model performance on complex tasks, researchers have begun to explore graph-based methods. These methods capture relationships between entities by constructing document graphs, but they often employ static graph structures and uniform attention mechanisms, neglecting the dynamic influence of relationship types and easily leading to the weakening of feature differences between different relationships. With the development of pre-trained models, Transformer-based methods, with their significant advantages in capturing long-range dependencies, have become the cutting-edge approach in document-level biomedical relationship extraction.

[0004] However, as research progresses, document-level biomedical relation extraction based on Transformer faces several challenges. First, entity representation suffers from staticity. Existing methods often employ simple operations like average pooling and logSumExp pooling to aggregate entity mention features, assuming all mentions contribute equally to the relation. This neglects mention specificity and fails to consider distance dependencies, leading to the loss of crucial semantic information. Second, relation specificity mining is insufficient. Models do not design independent feature propagation channels for different relation types, causing relation specificity information to gradually decay during feature propagation, making it difficult to accurately distinguish similar relations. Finally, inference robustness is limited. Most methods adopt a "single-inference" model, relying solely on local features of entity pairs for relation prediction, lacking a global consistency verification mechanism. In scenarios with noise interference and cross-sentence relation extraction, systematic prediction biases easily occur, making it difficult to capture complex cross-sentence semantic dependencies. Summary of the Invention

[0005] The purpose of this invention is to provide a document-level biomedical relation extraction method, which aims to solve the problems mentioned in the background art.

[0006] The present invention is implemented as follows: a document-level biomedical relation extraction method, specifically including the following steps: Step 1: Preprocess and encode the input document to obtain a word embedding representation containing contextual semantics; Step 2: Fuse the semantic distance and sentence distance of entity mentions, and dynamically aggregate mention features to generate enhanced entity representations; Step 3: Construct a heterogeneous graph containing entities and mentions, and use a dynamic relationship-aware graph attention mechanism to iteratively update node features to capture global semantic dependencies and relationship-specific information between entities; Step 4: Generate and optimize relationship predictions in two stages: "initial reasoning - enhanced verification", integrate local interaction features and global semantic information, and correct the initial prediction bias; Step 5: Optimize the model using a joint loss function of "initial loss + secondary loss" and output the final relationship classification result of entity pairs.

[0007] This invention provides a document-level biomedical relation extraction method. This method employs dual-channel modeling of semantic distance and sentence distance to assign differentiated weights to different entity mentions. This effectively captures mention specificity and distance dependence while resolving the key information loss problem caused by traditional static pooling. Furthermore, this method strengthens the propagation of relation-specific information by constructing independent feature propagation channels for each relation type and combining them with context-aware dynamic relation weights. It innovatively adopts a progressive architecture of "initial inference-enhanced verification": the first stage generates initial relation predictions based on entity-level enhanced representations, local structural features, and contextual information, capturing direct association clues between entity pairs; the second stage extracts global semantic features of the document through mean pooling and combines structural association information between entity pairs and other entities to systematically correct the initial prediction results, effectively compensating for the bias of single local feature inference and improving the accuracy and logical consistency of relation predictions. Experimental results on two widely used biomedical datasets, CDR and GDA, demonstrate that this method possesses excellent overall performance, especially in cross-sentence relation extraction tasks, effectively improving the overall performance of document-level biomedical relation extraction. Attached Figure Description

[0008] Figure 1 This is an overall architecture diagram of a document-level biomedical relation extraction method provided in an embodiment of the present invention; Figure 2 This is an architecture diagram of a multi-relation graph attention reasoning network in a document-level biomedical relation extraction method provided in an embodiment of the present invention. Detailed Implementation

[0009] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0010] The specific implementation of the present invention will be described in detail below with reference to specific embodiments.

[0011] like Figure 1 and Figure 2 As shown, this is an embodiment of the present invention providing a document-level biomedical relation extraction method, implemented based on the pre-trained language model BioBERT. This model integrates a document encoding module, a dynamic distance-aware entity interaction enhancement module (DDA-EIE), a multi-relation graph attention inference network module (MR-GARN), a two-stage inference enhancement module (DSIE), and a classifier module, specifically including the following steps: Step 1: The input document is preprocessed and encoded using the document encoding module to obtain a word embedding representation containing contextual semantics; specifically as follows: Step 1.1: Input biomedical documents, insert special markers "*" at the beginning and end of each entity mention to mark the mention boundaries, insert marker "[CLS]" at the beginning of each sentence and marker "[SEP]" at the end to distinguish the semantic boundaries of different sentences; Step 1.2: Encode the processed document using the pre-trained language model BioBERT to obtain word embedding representations. ,in, , It is the total length after inserting the marker. For embedded dimensions, Represent the real number field; denote each entity The One mention The embedding of the special marker "*" at the beginning position serves as a representation of the reference. .

[0012] Step 2: Using DDA-EIE, the semantic distance and sentence distance of entity mentions are fused, and mention features are dynamically aggregated to generate enhanced entity representations, solving the problem that static entity representations ignore mention specificity and distance dependency; specifically as follows: Step 2.1: Analyze entity pairs in the document , among which entities Include One mention ,entity Include One mention Calculate each pair of mentions Comprehensive distance : (1); in, As a balancing coefficient, its core function is to adjust semantic distance. Distance from sentence Contribution weight in comprehensive distance calculation; Representing entities The mention index is used to traverse entities. All references included; Represents entities The mention index is used to traverse entities. All references included; Sentence distance is used to characterize the spatial relationship between two mentions in a document, as shown in formula (2): (2); in, and Entities The mentions and entities The The sentence number in which the reference appears; This indicates the total number of sentences contained in the document; The value range is [0,1], and the smaller the value, the closer the spatial distance between the two mentions; Semantic distance, used to quantify the similarity between two mentions in their contextual semantics, is calculated using the cosine similarity of the initial representations of the mentions: (3); In the formula, The L2 norm of a vector; For entity mention The initial semantic embedding vector; For entity mention Transpose of the initial semantic embedding vector; The range of values ​​is The smaller the value, the stronger the semantic connection between the two mentions; Step 2.2: Calculate each pair of mentions correlation strength To measure the importance of semantic interaction between mentions: (4); in, A learnable query and key transformation matrix used to map mention embeddings to the same semantic space; A scaling factor is adapted to the embedding dimension of the attention score to mitigate the attention score bias caused by the curse of dimensionality.

[0013] Step 2.3: Introduce the distance offset term With distance weighting To further optimize the association strength, the guiding role of distance information in mention interactions can be enhanced. (5); in, It is a learnable distance-aware coefficient used to capture the association between reference embedding and distance; It is a global bias term used to adjust the overall strength of the distance bias. (6); In the formula, This is a learnable distance weight parameter, ranging from [0.5, 1.0], used to control the degree to which distance inhibits or promotes association strength; when At that time, the overall distance The smaller, The larger the value, the stronger the promoting effect on the correlation strength.

[0014] Step 2.4: Fuse the association strength, distance bias term, and distance weighting term to obtain the original attention score, and normalize it using the Softmax function to generate the attention weight matrix. : (7); (8); in, The original attention score is ; Indicates mention right The level of attention is such that the sum of all elements in the matrix is ​​1.

[0015] Step 2.5: Perform weighted aggregation of mention features based on the attention weight matrix to obtain entities. Regarding entities Weighted representation By using weighted summation, entity representations are made more focused on mentions that significantly contribute to relation prediction: (9); Step 2.6: Calculate entity pairs Contextual features Contextual features are represented by document word embeddings. Mean pooling yields: (10); in, This indicates the mean pooling operation, the result of which depends on The structure and specified dimension dim are used in the formula to compare along the 0th dimension. Calculate the average to obtain the aggregated result. ; Step 2.7: Weighted representation With context features splicing, generating entities through nonlinear transformation about Enhanced representation The formula is as follows: (11); in, This represents a vector concatenation operation. It is a learnable transformation matrix; To enhance the representation dimension, For contextual feature dimension, For bias terms, This is the activation function, used to introduce nonlinear feature transformations.

[0016] Step 2.8: For the entity It is then concatenated with the enhanced representations of all other entities in the document to obtain the entity. The final enhancement representation Stack the final enhanced representations of all entities to form an entity representation matrix. ,in The total number of entities in the document. Maximum number of mentions; Step 3: Construct a heterogeneous graph containing entities and mentions using MR-GARN. Iteratively update node features using a dynamic relationship-aware graph attention mechanism to capture global semantic dependencies and relationship-specific information between entities; details are as follows: Step 3.1: Construct a document heterogeneity graph; The node types in the graph include entity nodes and mention nodes. Entity nodes are characterized by the entity augmentation representation obtained in step 2, and mention nodes are characterized by the initial mention representation obtained in step 1. The edge types in the graph are based on a predefined set of relations. Determine (NR stands for "no relation") for any two nodes and If the corresponding entity or the mentioned entity has a relationship The construction type is Directed edges, original edge characteristics By splicing nodes and The characteristics and element-wise difference between the two are obtained, that is The final edge feature adjacency matrix is ​​formed. ,in This represents the total number of nodes in the graph. The number of predefined valid relation types; Step 3.2: Merge the entity augmented representation with the document graph node representation to obtain the initial augmented node representation. ; First, the dimensions of the entity node representation and the mention node representation are unified. A linear transformation is used to map the mention node representation to the same dimension as the entity node representation, resulting in a node representation matrix with unified dimensions. And introduce a gating mechanism to fuse entity representation matrices Node representation matrix : (12); (13); (14); in, and Entity representation matrices Node representation matrix The result of flattening; For learnable gated transformation matrix, This is a gated bias term; The Sigmoid activation function has an output value range of [0,1] and is used to control the fusion ratio of entity representation and node representation; This is element-wise multiplication; For dimensional reshaping operations, flattening Restored to the initial enhanced 3D node representation ; For gating fusion mechanism; It is a one-dimensional vector obtained by gating and fusing entity and node representations and then "flattening" it. Step 3.3: Stack L layers of Dynamic Relationship Graph Attention Network (DynaRelGAT) and iteratively update node features; For each layer of DynaRelGAT, first check the edge feature adjacency matrix. Nonlinear transformations are performed on the edge features to enhance them. Expressive ability: (15); in, For nodes and Interrelationship The original edge features; The learnable edge feature transformation matrix. The original edge feature dimension, The dimension of the transformed edge features; For edge feature bias terms; Step 3.4: Calculate the relation-specific attention coefficient ; For each relation type Independently calculate the attention coefficients between nodes to capture the semantic differences between different relationships: (16); in, and These are the feature representations of the nodes in the current layer; For relationship A dedicated attention parameter matrix, The nodes represent the feature dimensions of the flattened matrix. This is the scaling factor for dimension normalization; Step 3.5: Normalize the attention coefficients to obtain the attention weights; (17); In the formula, For nodes The set of neighboring nodes; This is an activation function used to alleviate the vanishing gradient problem; The attention weights between node u and its neighboring nodes under a specific relationship type r; for Let u be the set of its neighboring nodes. A specific neighbor node in the data; Step 3.6: Use K attention heads to aggregate neighbor features in parallel to capture semantic associations from multiple perspectives: (18); in, The relationship of the k-th attention head Dedicated parameter matrix; For the k-th attention head, node Based on relationships The results of neighbor feature aggregation; Step 3.7: Introduce context-aware dynamic relationship weights to adaptively adjust the contribution of different relationships to node feature updates;

[0017] (19); (20); in, For nodes Neighbor node characteristics The result of mean pooling; For a multilayer perceptron, the output relationship is... The initial weights; Let r be the initial weight of relation r; To normalize the initial weights of all relations; Let r' be the initial weight; This represents the feature representation of a node in the current layer. For the concatenation operation of multi-attention head results; Formula (19) is normalized by Softmax to ensure ; Step 3.8: Integrate multi-attention heads and dynamic relationship weights, and update the nodes. Feature representation Furthermore, residual connections and layer normalization are introduced to improve model training stability and feature propagation efficiency. (twenty one); (twenty two); in, The output transformation matrix is ​​used to map the concatenated features back to the original dimensions; For the first Nodes output by DynaRelGAT layer feature; For the first Layer nodes feature; This is a layer normalization operation used to standardize the feature distribution; The nodes represent the feature dimensions of the flattened matrix. Step 3.9: Repeat steps 3.3 to 3.8 to stack L layers of DynaRelGAT to obtain the final augmented node representation. ;extract The portion corresponding to the entity node forms an entity-level augmented representation. This is used for subsequent reasoning.

[0018] Step 4: Using DSIE, relationship predictions are generated and optimized in two stages: "initial inference" and "enhanced verification." Local interaction features and global semantic information are integrated to correct initial prediction biases and improve inference robustness; details are as follows: Step 4.1: Initial Relationship Score and Predicted Probability; Step 4.1.1: Extract entity structural features: entity structural features Taken from Corresponding entity in The representation of, that is For entities The Each mentioned node Its structural features Taken from augmented node representation The representation of the corresponding mentioned node, i.e. ,in To mention nodes Index in the diagram; Step 4.1.2: Calculate the contextual features of entities and mentions: First, construct the attention matrix for entities and mentions, and then... Its attention matrix The attention matrix is ​​obtained by averaging all its mentions, as shown in the following formula: (twenty three); in, For entities The number of mentions; To mention nodes Attention matrix.

[0019] Step 4.1.3: Weighted Aggregation Word Embedding Representation Based on Attention Matrix This yields the contextual features of the entity and the mention: (twenty four); (25); In the formula: To avoid small constants that divide by zero; For entities Contextual features; To mention nodes Contextual features; Step 4.1.4: For entity pairs It concatenates entity-level augmented representations, structural features, contextual features, and feature interaction terms to fuse multi-dimensional features and generate joint features for entity pairs: (26); in, Used to capture characteristic interaction information of entity pairs; Used to obtain joint features , The feature dimensions after concatenation; Step 4.1.5: Generate initial relation scores and predicted probabilities using a multilayer perceptron; (27); (28); in, For initial inference, a multilayer perceptron, For entity pairs Belongs to Relationship The initial score; Initial predicted probabilities used to map the scores to the [0,1] interval ; Step 4.2: Secondary relational reasoning; By integrating global semantic features and structural association features, the initial prediction bias is corrected to generate the final relationship prediction; Step 4.2.1: Calculate the structural association features of entity pairs; utilize the edge feature adjacency matrix. With entity-level augmented representation Capture the topological relationships between entity pairs and other entities in the document: (29); in, In the edge feature adjacency matrix, the nodes With entity pair The edge features between corresponding nodes For entity pairs Structural association features, A peace agreement for Einstein; Step 4.2.2: Calculate document-level global semantic features; Entity-level augmentation representation Perform mean pooling to obtain global semantic features. : (30); in, It is used to capture the overall semantic tendency of a document; Mean pooling is an operation that performs mean pooling along a specified dimension. Its core function is to adjust the input feature matrix. The average value is calculated along a specific dimension, thereby aggregating high-dimensional entity-level features into low-dimensional document-level global semantic features; Step 4.2.3: Fuse multi-source features to generate joint features for secondary inference. And a secondary inference score and a final prediction probability are generated through a multilayer perceptron; (31); (32); (33); in, For a multilayer perceptron with secondary reasoning, For entity pairs Belongs to Relationship The second score; This is the final predicted probability, used for relation classification.

[0020] Step 5: Using the classifier module, optimize the model using a joint loss function of "initial loss + secondary loss" to output the final relationship classification result for entity pairs; details are as follows: Step 5.1: For the initial loss Cross-entropy loss is used to constrain the initial prediction probability. Real Relationship Tags Consistency: (34); in, For real relationship tags, if entity pairs Belongs to Relationship ,but ,otherwise ; It represents the effective relation number plus the irrelevance coefficient.

[0021] Step 5.2: For secondary losses Marginal loss is used to enhance the logical consistency between the secondary inference score and the initial prediction, and to suppress prediction bias caused by noise interference. (35); in, For entity pairs Not a relationship The initial score, Let be the marginal constraint function, when the quadratic score Lower than the initial score difference At that time, losses occurred; Step 5.3: Weighted fusion of the initial loss and the secondary loss to obtain the joint loss. The contribution weights used to adjust the quadratic loss: (36); in, This is the loss balance coefficient.

[0022] To verify the effectiveness of this invention, systematic experiments were conducted on two widely used document-level biomedical relation extraction datasets: CDR (Biocreative V Chemical Disease Relation benchmark) and GDA (Gene Disease Associations). The CDR dataset, released by the BioCreative-V community, contains 1500 PubMed abstracts and binary relations between chemical and disease concepts, constructed using manual annotation. This dataset is evenly divided into training, validation, and test sets, each containing 500 documents. Its main task is to identify semantic relationships between chemical substances and disease entities in the text. The GDA dataset contains 30192 PubMed abstracts, focusing on the complex connections between gene and disease concepts on a larger scale. It was constructed using a remote supervision method on 30,192 abstracts from MEDLINE. The test set contains 1000 samples, and the remaining data is divided into training and validation sets at an 80%:20% ratio, focusing on mining the associations between genes and disease entities. Detailed statistical information for the two datasets is shown in Table 1.

[0023] Table 1. Statistical data for CDR and GDA datasets

[0024] The experiment used the F1 score, an evaluation metric conforming to official standards, to reasonably evaluate the performance of the proposed model on the Bio-DocRE task. Furthermore, inspired by previous research, the model's Intra-F1 and Inter-F1 scores are also reported, assessing the extraction effectiveness of intra-sentence and inter-sentence relation instances, respectively.

[0025] The performance comparison results of our method with existing methods are shown in Tables 2 and 3. Our method achieves the best performance, with an Overall-F1 score of 88.9%, Intra-F1 of 90.7%, and Inter-F1 of 84.3%, surpassing all baseline models and fully demonstrating the effectiveness of the adaptive spatial awareness and hierarchical reasoning mechanism. Our method shows significant performance improvement on the CDR dataset: compared to the state-of-the-art baseline model Topic-BiGRU-U-Net, our method improves the overall F1 score by 1.8%, especially in cross-sentence relation extraction, where our method reaches 84.3%, far exceeding Topic-BiGRU-U-Net's 81.7%, indicating its advantage in capturing long-distance semantic dependencies.

[0026] Table 2. Experimental results of this method on the CDR dataset.

[0027] Note: "-" in this table refers to data not given in the original paper, and underlined results refer to the highest results in the existing models.

[0028] Our method still achieves state-of-the-art performance on the GDA dataset, with an Overall-F1 score of 85.1%, an Intra-F1 score of 88.9%, and an Inter-F1 score of 69.2%. Compared to the baseline model, its overall F1 score exceeds the state-of-the-art baseline model REG-DRE by 0.3%, and its Inter-F1 score exceeds the FILR model by 0.3%. Compared to the CDR dataset, our model shows a smaller performance improvement on the GDA dataset. We speculate that this is related to the fact that the GDA dataset is much larger than the CDR dataset, and the number of inter-sentence relations only accounts for 16% of the total relations shown in Table 1. The large-scale data reduces the model's sensitivity to structural optimization. However, our method still maintains its leading position, further validating its stability.

[0029] Table 3. Experimental results of this method on the GDA dataset.

[0030] Note: "-" in this table refers to data not given in the original paper, and underlined results refer to the highest results in the existing models.

[0031] In summary, this method effectively captures the ambiguity, global relational dependencies, and logical consistency of entity multi-mentions through a collaborative design of dynamic distance-aware entity interaction enhancement, multi-relation graph attention reasoning, and two-stage verification, achieving superior performance in document-level biomedical relation extraction tasks.

[0032] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A document-level biomedical relation extraction method, characterized in that, Specifically, the following steps are included: Step 1: Preprocess and encode the input document to obtain a word embedding representation containing contextual semantics; Step 2: Fuse the semantic distance and sentence distance of entity mentions, and dynamically aggregate mention features to generate enhanced entity representations; Step 3: Construct a heterogeneous graph containing entities and mentions, and use a dynamic relationship-aware graph attention mechanism to iteratively update node features to capture global semantic dependencies and relationship-specific information between entities; Step 4: Generate and optimize relationship predictions in two stages: "initial reasoning - enhanced verification", integrate local interaction features and global semantic information, and correct the initial prediction bias; Step 5: Optimize the model using the joint loss function of "initial loss + secondary loss" and output the final relationship classification result of entity pairs.

2. The document-level biomedical relation extraction method according to claim 1, characterized in that, The specific steps of step 1 are as follows: Step 1.1: Input biomedical documents, insert special markers "*" at the beginning and end of each entity mention to mark the mention boundary, insert marker "[CLS]" at the beginning of each sentence and marker "[SEP]" at the end of each sentence to distinguish the semantic boundaries of different sentences; Step 1.2: Encode the processed document using the pre-trained language model BioBERT to obtain word embedding representations. ,in, , It is the total length after inserting the marker. For embedded dimensions, Represent the real number field; denote each entity The One mention The embedding of the special marker "*" at the beginning position serves as a reference to this mention. .

3. The document-level biomedical relation extraction method according to claim 2, characterized in that, The specific steps of step 2 are as follows: Step 2.1: Analyze entity pairs in the document , among which entities Include One mention ,entity Include One mention Calculate each pair of mentions Comprehensive distance : (1)； in, This is a balancing coefficient used to adjust semantic distance. Distance from sentence Contribution weight in comprehensive distance calculation; Representing entities The mention index is used to traverse entities. All references included; Representing entities The mention index is used to traverse entities. All references included; Sentence distance is used to characterize the spatial relationship between two mentions in a document, as shown in formula (2): (2)； in, and Entities The mentions and entities The The sentence number in which the reference appears; This indicates the total number of sentences contained in the document; The value range is [0,1], and the smaller the value, the closer the spatial distance between the two mentions; Semantic distance, used to quantify the similarity between two mentions in their contextual semantics, is calculated using the cosine similarity of the initial representations of the mentions: (3)； In the formula, The L2 norm of a vector; For entity mention The initial semantic embedding vector; For entity mention Transpose of the initial semantic embedding vector; The range of values ​​is The smaller the value, the stronger the semantic connection between the two mentions; Step 2.2: Calculate each pair of mentions correlation strength To measure the importance of semantic interaction between mentions; (4)； in, A learnable query and key transformation matrix used to map mention embeddings to the same semantic space; A scaling factor is adapted to the embedding dimension of the attention score to mitigate the attention score bias caused by the curse of dimensionality. Step 2.3: Introduce the distance offset term With distance weighting Optimize association strength and enhance the guiding role of distance information in mention interactions; (5)； in, It is a learnable distance-aware coefficient used to capture the association between reference embedding and distance; It is a global bias term used to adjust the overall strength of the distance bias. (6)； In the formula, This is a learnable distance weight parameter, ranging from [0.5, 1.0], used to control the degree to which distance inhibits or promotes association strength; when At that time, the overall distance The smaller, The larger the value, the stronger the promoting effect on the correlation strength; Step 2.4: Fuse the association strength, distance bias term, and distance weighting term to obtain the original attention score, and normalize it using the Softmax function to generate the attention weight matrix. ; (7)； (8)； in, The original attention score; Indicates mention right The degree of attention is such that the sum of all elements in the matrix is ​​1; Step 2.5: Perform weighted aggregation of mention features based on the attention weight matrix to obtain entities. Regarding entities Weighted representation ; (9)； Step 2.6: Calculate entity pairs Contextual features Contextual features are represented by document word embeddings. The result is obtained by mean pooling. (10)； in, This indicates the mean pooling operation, the result of which depends on The structure and specified dimension dim are used in the formula to compare along the 0th dimension. Calculate the average to obtain the aggregated result. ; Step 2.7: Weighted representation With context features splicing, generating entities through nonlinear transformations about Enhanced representation The formula is as follows: (11)； in, This represents a vector concatenation operation. It is a learnable transformation matrix; To enhance the representation dimension, For contextual feature dimension, For bias terms, This is an activation function used to introduce nonlinear feature transformations; Step 2.8: For the entity It is then concatenated with the enhanced representations of all other entities in the document to obtain the entity. The final enhancement representation Stack the final enhanced representations of all entities to form an entity representation matrix. ,in The total number of entities in the document. The maximum number of mentions.

4. The document-level biomedical relation extraction method according to claim 3, characterized in that, The specific steps of step 3 are as follows: Step 3.1: Construct a document heterogeneity graph; The node types in the graph include entity nodes and mention nodes. Entity nodes are characterized by the entity augmentation representation obtained in step 2, and mention nodes are characterized by the mention initial representation obtained in step 1. The edge types in the graph are based on a predefined set of relations. It is certain that for any two nodes and If the corresponding entity or the mentioned entity has a relationship The construction type is Directed edges, original edge characteristics By splicing nodes and The characteristics and element-wise difference between the two are obtained, that is The final edge feature adjacency matrix is ​​formed. ,in This represents the total number of nodes in the graph. The number of predefined valid relation types; Step 3.2: Merge the entity augmented representation with the document graph node representation to obtain the initial augmented node representation. ; First, the dimensions of the entity node representation and the mention node representation are unified. A linear transformation is used to map the mention node representation to the same dimension as the entity node representation, resulting in a node representation matrix with unified dimensions. And introduce a gating mechanism to fuse entity representation matrices Node representation matrix : (12)； (13)； (14)； in, and Entity representation matrices Node representation matrix The result of flattening; For learnable gated transformation matrix, This is a gated bias term; The Sigmoid activation function has an output value range of [0,1] and is used to control the fusion ratio of entity representation and node representation; This is element-wise multiplication; For dimensional reshaping operations, flattening Restored to the initial 3D augmented node representation ; For gating fusion mechanism; It is a one-dimensional vector obtained by gating and fusing entity and node representations and then "flattening" them. Step 3.3: Stack L layers of dynamic relationship graph attention network and iteratively update node features; For each layer of DynaRelGAT, first check the edge feature adjacency matrix. Nonlinear transformations are performed on the edge features to enhance them. Expressive ability: (15)； in, For nodes and Interrelationship The original edge features; The learnable edge feature transformation matrix. The original edge feature dimension, The dimension of the transformed edge features; For edge feature bias terms; Step 3.4: Calculate the relation-specific attention coefficient ; For each relation type Independently calculate the attention coefficients between nodes to capture the semantic differences between different relationships: (16)； in, and These are the feature representations of the nodes in the current layer; For relationship A dedicated attention parameter matrix, The nodes represent the feature dimensions of the flattened matrix. This is the scaling factor for dimension normalization; Step 3.5: Normalize the attention coefficients to obtain the attention weights; (17)； In the formula, For nodes The set of neighboring nodes; This is an activation function used to alleviate the vanishing gradient problem; The attention weights between node u and its neighboring nodes under a specific relationship type r; Let u be the set of its neighboring nodes. A specific neighbor node in the data; Step 3.6: Use K attention heads to aggregate neighbor features in parallel to capture semantic associations from multiple perspectives; (18)； in, The relationship of the k-th attention head Dedicated parameter matrix; For the k-th attention head, node Based on relationships The results of neighbor feature aggregation; Step 3.7: Introduce context-aware dynamic relationship weights to adaptively adjust the contribution of different relationships to node feature updates; (19)； (20)； in, For nodes Neighbor node characteristics The result of mean pooling; For a multilayer perceptron, the output relationship is... The initial weights; Let r be the initial weight of relation r; To normalize the initial weights of all relations; Let r' be the initial weight; This represents the feature representation of a node in the current layer. For the concatenation operation of multi-attention head results; Formula (19) is normalized by Softmax to ensure ; Step 3.8: Integrate multi-attention heads and dynamic relationship weights, and update the nodes. Feature representation Furthermore, residual connections and layer normalization are introduced to improve model training stability and feature propagation efficiency. (21)； (22)； in, The output transformation matrix is ​​used to map the concatenated features back to the original dimensions; For the first Nodes output by DynaRelGAT layer feature; For the first Layer nodes feature; This is a layer normalization operation used to standardize the feature distribution; The nodes represent the feature dimensions of the flattened matrix. Step 3.9: Repeat steps 3.3 to 3.8 to stack L layers of DynaRelGAT to obtain the final augmented node representation. ;extract The portion corresponding to the entity node forms an entity-level augmented representation. This is used for subsequent reasoning.

5. The document-level biomedical relation extraction method according to claim 4, characterized in that, The specific steps of step 4 are as follows: Step 4.1: Obtain the initial relationship score and predicted probability; Extract entity structural features, calculate the contextual features of entities and mentions, obtain the contextual features of entities and mentions based on the attention matrix weighted aggregation, generate joint features of entity pairs by fusing multi-dimensional features, and finally generate initial relation scores and prediction probabilities through a multilayer perceptron. Step 4.2: Secondary relational reasoning; By integrating global semantic features and structural association features, the initial prediction bias is corrected, and the final relationship prediction is generated.

6. The document-level biomedical relation extraction method according to claim 5, characterized in that, The specific steps of step 4.1 are as follows: Step 4.1.1: Extract entity structural features. Taken from Corresponding entity in The representation of, that is For entities The Each mentioned node Its structural features Taken from augmented node representation The representation of the corresponding mentioned node, i.e. ,in To mention nodes Index in the diagram; Step 4.1.2: Calculate the contextual features of entities and mentions. First, construct the attention matrix for entities and mentions. Its attention matrix The attention matrix is ​​obtained by averaging all its mentions, as shown in the following formula: (23)； in, For entities The number of mentions; To mention nodes Attention matrix; Step 4.1.3: Weighted Aggregation Word Embedding Representation Based on Attention Matrix This yields the contextual features of the entity and the mention: (24)； (25)； In the formula: To avoid small constants that divide by zero; For entities Contextual features; To mention nodes Contextual features; Step 4.1.4: For entity pairs It concatenates entity-level augmented representations, structural features, contextual features, and feature interaction terms, and fuses multi-dimensional features to generate joint features for entity pairs: (26)； in, Used to capture characteristic interaction information of entity pairs; Used to obtain joint features , The feature dimensions after concatenation; Step 4.1.5: Generate initial relation scores and predicted probabilities using a multilayer perceptron; (27)； (28)； in, For initial inference, a multilayer perceptron, For entity pairs Belongs to Relationship The initial score; Initial predicted probabilities used to map the scores to the [0,1] interval .

7. The document-level biomedical relation extraction method according to claim 6, characterized in that, The specific steps of step 4.2 are as follows: Step 4.2.1: Calculate the structural association features of entity pairs; Using edge feature adjacency matrix With entity-level augmented representation Capture the topological relationships between entity pairs and other entities in the document: (29)； in, In the edge feature adjacency matrix, the nodes With entity pair The edge features between corresponding nodes For entity pairs Structural association features, A peace agreement for Einstein; Step 4.2.2: Calculate document-level global semantic features; Entity-level augmentation representation Perform mean pooling to obtain global semantic features. : (30)； in, It is used to capture the overall semantic tendency of a document; Mean pooling is performed along a specified dimension on the input feature matrix. The average value is calculated along a specific dimension, and the high-dimensional entity-level features are aggregated into low-dimensional document-level global semantic features. Step 4.2.3: Fuse multi-source features to generate joint features for secondary inference. And a secondary inference score and a final prediction probability are generated through a multilayer perceptron; (31)； (32)； (33)； in, For a multilayer perceptron with secondary reasoning, For entity pairs Belongs to Relationship The second score; This is the final predicted probability, used for relation classification.

8. The document-level biomedical relation extraction method according to claim 7, characterized in that, The specific steps of step 5 are as follows: Step 5.1: For the initial loss Cross-entropy loss is used to constrain the initial prediction probability. Real Relationship Tags Consistency: (34)； in, For real relationship tags, if entity pairs Belongs to Relationship ,but ,otherwise ; This represents the effective relation count plus the irrelevance coefficient. Step 5.2: For secondary losses Marginal loss is used to enhance the logical consistency between the secondary inference score and the initial prediction, and to suppress prediction bias caused by noise interference. (35)； in, For entity pairs Not a relationship The initial score, Let be the marginal constraint function, when the quadratic score Lower than the initial score difference At that time, losses occurred; Step 5.3: Weighted fusion of the initial loss and the secondary loss to obtain the joint loss. The contribution weights used to adjust the quadratic loss: (36)； in, This is the loss balance coefficient.

Images