A document-level relation extraction method guided by entity-guided axial attention and dual-threshold calibration

By employing an entity-guided axial attention mechanism and adaptive dual-focus loss, a global multi-threshold calibration method is used to explicitly model entity pair interactions, solving the problems of information propagation noise and sample imbalance in document-level relation extraction, and improving the accuracy and consistency of extraction.

CN123068637APending Publication Date: 2026-11-10NORTHEASTERN UNIV CHINA
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611593044.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-10-15
Publication Date
2026-11-10

AI Technical Summary

Technical Problem

Existing document-level relation extraction methods lack explicit modeling of structural dependencies between entity pairs, and adaptive threshold learning lacks global calibration, leading to noise ingress during information propagation and imbalance between positive and negative samples, which affects extraction accuracy and consistency.

Method used

An entity-guided axial attention mechanism is used to explicitly model entity pair interactions. Combined with adaptive dual-focus loss and global multi-threshold calibration, the accuracy and stability of the model are improved by selective information propagation guided by entities and differential processing of positive and negative samples.

Benefits of technology

It effectively filters out irrelevant entity pairs, alleviates the imbalance between positive and negative samples, and improves the accuracy of document-level relation extraction and the consistency of prediction results.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123068637A_ABST
    Figure CN123068637A_ABST
Patent Text Reader

Abstract

This invention discloses a document-level relation extraction method based on entity-guided axial attention and dual-threshold calibration, belonging to the field of document-level relation extraction technology. This invention combines entity-guided axial attention, adaptive dual-focus loss, and global multi-threshold calibration into the document-level relation extraction task. Entity-guided axial attention explicitly models the interaction relationships between entity pairs on an entity pair grid, and utilizes the semantic information of head and tail entities to guide information propagation in the row and column directions, reducing noise from irrelevant entity pairs. Adaptive dual-focus loss simultaneously reweights positive and negative relations, alleviating the imbalance problem caused by a high proportion of unrelated entity pairs. Global multi-threshold calibration learns corresponding global thresholds for different relation subspaces, improving the consistency of overall prediction results while preserving the adaptive judgment ability of entity pairs.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of document-level relation extraction technology, specifically to a document-level relation extraction method using entity-guided axial attention and dual-threshold calibration. Background Technology

[0002] Relation extraction is a fundamental task in natural language processing (NLP), aiming to automatically identify semantic relationships between entities from unstructured text. It plays a crucial supporting role in applications such as knowledge graph construction, information retrieval, and intelligent question answering. With deepening needs, the task scenario has expanded from single sentences to multi-sentence documents, i.e., document-level relation extraction. Unlike sentence-level relation extraction, document-level relation extraction requires comprehensive analysis of the entire multi-sentence text to determine entity pair relationships, facing greater challenges: target entity pairs are often distributed across sentences, the same entity appears in multiple references within the document, and relation inference often requires multi-hop logical reasoning using intermediate entities. Most current mainstream document-level relation extraction methods employ a pre-trained language model based on the Transformer architecture to encode the input document, obtaining a word-level contextual representation; then, they aggregate information about the same entity from multiple locations throughout the text to form an entity-level semantic representation; subsequently, they fuse head entity, tail entity, and relevant contextual features to construct entity pair representations. Finally, a classifier performs multi-label classification on the entity pair to predict one or more semantic relationships corresponding to it.

[0003] To enhance the interaction modeling capabilities between entity pairs, some existing methods construct entity pairs within a document into a two-dimensional grid structure and introduce an axial attention mechanism to achieve information transmission along the grid's row and column dimensions. This mechanism allows entity pairs with the same head or tail entity to complete information interaction, improving the model's two-hop reasoning ability to some extent. However, these methods employ a fixed aggregation strategy during the information propagation phase, lacking guidance from head and tail entities. They cannot selectively constrain the information propagation path based on the semantic features of the head and tail entities, easily introducing entity pair information unrelated to the target relationship, resulting in noise propagation and affecting the accuracy of relationship extraction.

[0004] Furthermore, document-level relation extraction is essentially a multi-label classification task: the same entity pair may correspond to one or more semantic relations, or it may not have any relation at all. Real-world datasets often contain a high proportion of unrelated entity pairs, leading to a severe imbalance between positive and negative samples, making the model prone to predicting entity pairs as unrelated. To address this issue, existing methods generally introduce adaptive threshold strategies for relation discrimination: for each candidate entity pair, the model simultaneously outputs scores for each relation category and a threshold category score. If a relation category score is higher than the threshold category score, the relation is considered valid; otherwise, it is invalid. When all relation category scores are lower than the threshold category score, the candidate entity pair is marked as unrelated. However, existing adaptive threshold methods primarily optimize the ability to distinguish between positive relations and the threshold, making it difficult to effectively suppress the negative sample interference caused by a large number of unrelated entity pairs. More importantly, the thresholds for different entity pairs are learned independently, lacking a globally unified calibration mechanism, resulting in inconsistent prediction standards for different entity pairs, leading to problems such as missed detections, false positives, and conflicting prediction results.

[0005] In summary, existing document-level relation extraction methods have the following main shortcomings: First, most methods lack sufficient explicit modeling of the structural dependencies between entity pairs, and interaction methods based on entity pair grids still lack selective information propagation guided by entity semantics, easily introducing irrelevant information. Second, existing threshold learning methods are insufficient in handling class imbalance, and adaptive thresholds lack an effective global calibration mechanism, making it difficult to balance entity pair differences and prediction standard consistency. Therefore, a document-level relation extraction method is needed that can explicitly model entity pair interactions, achieve selective information propagation guided by entity semantics, and improve threshold learning and global calibration under imbalanced positive and negative samples. Summary of the Invention

[0006] In view of at least one of the above technical problems, this application provides a document-level relation extraction method with entity-guided axial attention and dual-threshold calibration, which aims to combine entity-guided axial attention, adaptive dual-focus loss and global multi-threshold calibration into the document-level relation extraction task to improve the accuracy and stability of document-level relation extraction.

[0007] The technical solution of this invention is:

[0008] A document-level relation extraction method using entity-guided axial attention and dual-threshold calibration, the method comprising:

[0009] Obtain the original document, as well as the document's sentence set, entity set, and entity mention set, and preprocess the document to obtain the lexical sequence and lexical subsequence;

[0010] Based on lexical subsequences, document-level lexical hidden representations and document-level attention weights are obtained;

[0011] Encode each lexical subsequence to obtain fragment-level lexical hidden representations and fragment-level attention weights. Merge each fragment-level lexical hidden representation and fragment-level attention weight to obtain document-level lexical hidden representations and document-level attention weights.

[0012] Entity representations and entity attention representations for each entity are constructed based on document-level lexical hiding representations and document-level attention weights; based on the entity attention representations of the head and tail entities, normalized lexical importance weights for entity pairs are constructed, and then contextual representations of entity pairs are constructed.

[0013] Based on the entity representations and context representations of the head and tail entities of the entity pair, construct the initial features of each entity pair in the document, and initialize the entity pair mesh using the initial features of each entity pair.

[0014] The final entity pair mesh is obtained by updating the entity pair mesh based on the entity-guided axial attention mechanism.

[0015] A relation classifier is trained using adaptive dual-focus loss.

[0016] During the inference phase of the relation classifier, the relation category of each entity pair in the final entity pair grid is predicted, and a global multi-threshold calibration is performed on the relation category prediction results to obtain the calibrated relation prediction results.

[0017] Optionally, according to the document-level relation extraction method based on entity-guided axial attention and dual-threshold calibration, the preprocessing of the document to obtain a lexical sequence and lexical subsequences includes:

[0018] Input the original document into the pre-trained word segmenter to obtain a sequence of words;

[0019] A sliding window is used to segment the word sequence, resulting in word subsequences for each segment.

[0020] Optionally, according to the document-level relation extraction method based on entity-guided axial attention and dual-threshold calibration, the step of obtaining document-level lexical hidden representations and document-level attention weights based on lexical subsequences includes:

[0021] Each word subsequence is input into a pre-trained language model for encoding to obtain the fragment-level word hidden representation and fragment-level attention weights corresponding to each word subsequence.

[0022] The fragment-level lexical hidden representations and fragment-level attention weights corresponding to each lexical subsequence are aligned and merged according to the corresponding positions of each lexical in the document to obtain document-level lexical hidden representations and document-level attention weights.

[0023] Optionally, according to the document-level relation extraction method with entity-guided axial attention and dual-threshold calibration, the method for constructing entity representations and entity attention representations for each entity based on document-level lexical hiding representations and document-level attention weights is as follows:

[0024] Known Entities Entity mention set By summing the logarithms and the exponents Various entities mentioned Corresponding lexical hidden representation Aggregate to obtain entities entity representation Specifically, it is expressed as: ; through averaging Various entities mentioned Corresponding attention representation Aggregate to obtain entities Entity attention representation Specifically, it is expressed as: .

[0025] Optionally, according to the document-level relation extraction method based on entity-guided axial attention and dual-threshold calibration, the step of constructing normalized lexical importance weights for entity pairs based on the entity attention representations of the head and tail entities, and then constructing the contextual representation of the entity pairs, includes:

[0026] For entity pairs Head entity Entity attention representation Tail-end entity Entity attention representation Element-wise multiplication is performed to obtain the word-level joint attention vector; the average of all word-level joint attention vectors is then performed, and the averaged vector is then... Normalization yields entity pairs Normalized lexical importance weights Specifically, it is expressed as: ,in Indicates the first [item] in the input document Each lexical unit has a corresponding lexical importance weight. Represents the word sequence of the input document;

[0027] Based on the normalized lexical importance weights, the hidden representations of each lexical are weighted and pooled to obtain entity pairs. Context representation If each lexical element comes from the entire sentence of the document, then This represents the global context; if each lexical comes from a entity containing the head entity... or tail entity The sentence mentioned by the entity, then Context-aware representation of sentences.

[0028] Optionally, according to the document-level relation extraction method with entity-guided axial attention and dual-threshold calibration, the step of constructing initial entity pair features for each entity pair in the document based on the entity representations and context representations of the head and tail entities of the entity pair, and initializing the entity pair mesh using the initial entity pair features of each entity pair, includes:

[0029] For entity pairs Represent the head entity Entity representation of context The feature dimension is concatenated to obtain and to A linear transformation is performed, followed by a nonlinear transformation of the result using the hyperbolic tangent activation function to obtain the head-side features. ; Represent the tail entity Entity representation of context The feature dimension is concatenated to obtain and to A linear transformation is performed, followed by a nonlinear transformation of the result using the hyperbolic tangent activation function to obtain the tail-side features. ;

[0030] Head side features and tail side features Simultaneously input into the grouped bilinear function Interactive modeling is performed, and the modeling results are evaluated using a projection matrix. Perform mapping to obtain entity pairs The initial features of the entity pairs;

[0031] The initial features of each entity pair in the document are indexed according to the header entity index. End-of-Pack Entity Index This corresponds to the corresponding cells in the entity pair mesh, thus constructing the initial entity pair mesh. .

[0032] Optionally, according to the document-level relation extraction method based on entity-guided axial attention and dual-threshold calibration, the step of updating the entity pair mesh based on the entity-guided axial attention mechanism to obtain the final entity pair mesh includes:

[0033] Document Included The entity representations of each entity are combined to form an entity representation matrix. ;

[0034] From the initial entity pair mesh Initially, based on the entity representation matrix Stacked in series, level by level The entity guides the axial attention block, for the first entity... Layer entity to mesh Overall update to ,in, Indicates the first Layer entities guide the input entity pairs of the axial attention block to the mesh; Indicates after the first The layer entity guides the updated entity pair mesh output by the axial attention block; The layer index representing the entity-guided axial attention block; It is an empirical value representing the total number of layers of the entity-guided axial attention block; Indicates to and Apply a solid to guide the axial attention block;

[0035] In the In the layer entity guide axial attention block, first based on the head entity entity representation Construct line direction entity guide mark ,in This represents the learnable projection matrix used to generate row-direction entity guide markers, and then through... The input entity pair mesh of this attention block The Entity pair feature sequence of a row Perform line direction aggregation guided by the head entity to obtain the updated line direction entity guide marker. Entity pair feature sequences updated by row direction ;

[0036] Reuse Generate line direction scaling parameters and offset parameters For each entity pair feature after the row direction is updated Continue with feature modulation:

[0037] ;

[0038] ;

[0039] in, This represents the learnable projection matrix used to generate the line direction modulation parameters;

[0040] Similarly, perform column-direction aggregation and feature modulation guided by tail entities;

[0041] Regarding the first The input entity of the entity guiding the axial attention block is a mesh. Each entity in the feature Constructing a multilayer perceptron to predict two-dimensional gate vectors ,use Two components and Update the results for each row direction. Column direction update results Weighting is performed to obtain the entity pair feature increments that fuse the row and column direction updates. This leads to the entity pair feature increment mesh. ;

[0042] Incremental mesh of entity pairs features Entity to Mesh By summing the residuals, we obtain the intermediate solid pair mesh. After performing layer normalization, the result of layer normalization is input into the position feedforward network, and then the residuals are summed to obtain the first... Layer entity to mesh ;

[0043] Repeat the above steps in sequence. Layer entities guide axial attention updates, thereby obtaining updated entity pairs in the mesh. ;

[0044] For each mesh cell in the entity pair mesh, assume it is located in the entity pair mesh. Location, based on its entity's initial features Calculate the scalar gating coefficient ,in This represents the learnable parameters used to generate scalar gating coefficients, and utilizes... For entity pairs of initial features and the process Entity pair features updated at each layer Interpolation and fusion are performed to obtain entity pairs The final entity pair representation ;

[0045] The final entity pair representations of all entity pairs are combined together to form the final entity pair mesh after entity-guided axial attention update, represented as: .

[0046] Optionally, according to the document-level relation extraction method with entity-guided axial attention and dual-threshold calibration, predicting the relation category of each entity pair in the final entity pair grid during the relation classifier inference stage includes:

[0047] Predefined set of relation categories And introduce a threshold category. The final entity will be meshed. The input is fed into a relation classifier, which outputs a predicted score for each predefined relation category and a predicted score for a threshold category for each entity pair. When predicting relations, each relation category... Predicted score and threshold category When compared with the predicted scores, The predicted score is higher than When predicting the score, the relationship category is used as the prediction result for the relationship category of the current entity pair.

[0048] Optionally, according to the document-level relation extraction method based on entity-guided axial attention and dual-threshold calibration, the method of training the relation classifier using adaptive dual-focus loss is as follows:

[0049] Based on the predicted scores of each relation category and the predicted score of the threshold category Calculate the positive class probability respectively. and negative class threshold probability :

[0050] right Re-weighting the focus yields the set of positive class relations. Calculated positive class loss ;in Represents entity pairs The corresponding set of positive relationships, that is, the set of relationship categories that the entity actually has; This represents the weighting coefficient corresponding to the positive class loss; This represents a parameter used to control the degree of focus for positive classes;

[0051] right Re-weighting the focus yields the set of negative class relations. The calculated negative class loss ;in Represents entity pairs The corresponding set of negative relation classes, that is, all relation classes other than positive relation classes among all predefined relation classes; This represents the weighting coefficients corresponding to the negative class loss; This represents a parameter used to control the degree of focus on negative classes;

[0052] The objective function for training a relation classifier is to minimize .

[0053] Optionally, according to the document-level relation extraction method with entity-guided axial attention and dual-threshold calibration, during the relation classifier inference stage, a global multi-threshold calibration is performed on the relation category prediction results to obtain calibrated relation prediction results, including:

[0054] For any relation category Calculation in the original document Predicted score Relative to threshold category Predicted score relation boundary score And in pseudo-documents Predicted score Relative to threshold category Predicted score relation boundary score If relation class r appears in both the original document and the pseudo-document, then let relation class r be... Final relation boundary score If the relation class r only appears in the original document, then for Impose punishment, make If the relation class r only appears in the pseudo-document, then for Impose punishment, make ;

[0055] The frequency of each relation category in the training data is counted, and all relation categories are sorted according to their frequency. Then, based on the preset relation subspace partitioning rules, the sorted relation categories are divided into subspaces. In each sub-label space, each relation category corresponds to a sub-label space. The sub-label space is represented as ,in, Indicates the index of the sub-label space; Indicates the first Individual tag space;

[0056] For the Calculate the global threshold in the sub-label space. For the first Individual tag space Collect predicted relation categories on the development set. Candidate relations are sorted from highest to lowest according to their final relation boundary scores. F1 scores are then calculated for each candidate relation at different thresholds. The final relation boundary score of the candidate relation that maximizes the F1 score in the open set is taken as the final relation boundary score. global threshold ;

[0057] During the reasoning phase, for simultaneously satisfying and For the relation category r, r is directly accepted as the relation prediction result for the corresponding entity pair; for the remaining candidate relation categories... , and its final relation boundary score With its own Global threshold of sub-label space Compare, if satisfied If the relation class r is true, then the relation class r is accepted; otherwise, r is not accepted.

[0058] The beneficial effects of adopting the above technical solution are as follows:

[0059] (1) Based on the construction of entity pair mesh, this invention applies an entity-guided axial attention mechanism to explicitly model the interaction relationship between entity pairs on the entity pair mesh. This mechanism uses the semantic information of the head entity and the tail entity to guide the information propagation in the row and column dimensions of the mesh, realizes the selective information transmission of related entity pairs, enables the model to filter out noise brought by irrelevant entity pairs, and effectively utilizes cross-sentence information and intermediate entity information to perform two-hop reasoning.

[0060] (2) The present invention designs an adaptive dual-focus loss function to perform differentiated reweighting on positive and negative relationships, which effectively alleviates the problem of imbalance between positive and negative samples caused by the excessive proportion of unrelated entity pairs.

[0061] (3) This invention proposes a global multi-threshold calibration mechanism, which learns a dedicated global threshold for each relation subspace, so that the model can take into account both the adaptive judgment ability of entity pairs and the consistency of global prediction. While retaining the adaptive judgment ability of entity pairs, it improves the consistency of prediction results between different entity pairs, thereby improving the accuracy and stability of document-level relation extraction. Attached Figure Description

[0062] Figure 1 This is a flowchart of the document-level relation extraction method with entity-guided axial attention and dual-threshold calibration in this embodiment;

[0063] Figure 2 This is a schematic diagram of the adaptive dual-focus loss mechanism in this embodiment;

[0064] Figure 3 This is a schematic diagram of the global multi-threshold calibration mechanism in this embodiment. Detailed Implementation

[0065] The specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples. The following examples are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0066] The core idea of ​​this invention is to combine entity-guided axial attention, adaptive dual-focus loss, and global multi-threshold calibration for document-level relation extraction. First, entity pairs are constructed into a two-dimensional entity pair feature grid. Head and tail entities are used to guide information aggregation in the row and column directions, respectively, to explicitly model the interaction relationships between entity pairs. Second, adaptive dual-focus loss applies focus weights to the loss terms corresponding to the positive and negative classes, respectively, making the model pay more attention to the difficult-to-distinguish positive and negative class relationships, thus alleviating the problem of positive-negative sample imbalance. Finally, prediction information from original documents and pseudo-documents is fused, and corresponding global thresholds are determined for different relation subspaces. While preserving the adaptive discrimination capability of entity pairs, the prediction results are globally calibrated, thereby improving the accuracy and consistency of document-level relation extraction.

[0067] This embodiment's entity-guided axial attention and dual-threshold calibration document-level relation extraction method uses two pre-trained language models (PLMs) – BERT-base and RoBERTa-large – as encoders. These are commonly used models in the document-level relation extraction field, differing in scale: BERT-base contains 12 encoding layers with a hidden dimension of 768 and 12 attention heads; RoBERTa-large contains 24 encoding layers with a hidden dimension of 1024 and 16 attention heads. Using two types of encoders aims to verify the applicability and stability of the proposed method on pre-trained language models of different scales. Furthermore, the experimental datasets used in this embodiment are Re-DocRED and DWIE. These two datasets differ in document size, number of relation categories, and annotation characteristics, allowing for a comprehensive evaluation of the generalization ability of the proposed method under different data conditions. DWIE contains 799 documents and 65 relation categories, with 602 documents in the training set, 98 in the development set, and 99 in the test set. Re-DocRED contains 4053 documents and 96 relation categories, with 3053 documents in the training set, 500 in the development set, and 500 in the test set. Each dataset includes: a predefined set of relation categories, the original document, the sentence segmentation results corresponding to the document, the entity segmentation information corresponding to the document, the entity mentions corresponding to the entity, entity pairs consisting of head and tail entities, and the correspondence between entity pairs.

[0068] Referring to the above core ideas and Figure 1 The entity-guided axial attention and dual-threshold calibration document-level relation extraction method of this embodiment specifically includes the following steps:

[0069] S1. Obtain the original document and its sentence set, entity set, and entity mention set, and preprocess the document to obtain the lexical sequence and lexical subsequence of the document;

[0070] S1.1, Transfer the original document The input to the pre-trained language model is processed by the word segmenter to obtain... Corresponding lexical sequence :

[0071] ;

[0072] in, Document lexical sequence; Indicates the index of a lexical unit in the lexical sequence; Represents the first word in the lexical sequence Each word element; Document The total number of lexical units contained therein.

[0073] Document The corresponding set of sentences is represented as:

[0074] ;

[0075] in, express A collection of sentences; This represents the index of the sentence within the sentence set; Represents the first sentence in the set of sentences. One sentence; express The total number of sentences contained therein.

[0076] Document The corresponding entity set is represented as:

[0077] ;

[0078] in, Indicates the document to be processed. A collection of entities; This represents the index of the entity within the entity collection; Represents the first in the entity set One entity; Document The total number of entities included.

[0079] Document-level relation extraction is modeled as a multi-label classification task, the goal of which is to extract relational information from head entities. Tail-end entity The entity pairs constituted Predict the corresponding relationship, where, and These represent the indices of the head and tail entities in the entity set, respectively. For each entity pair... The model starts from a predefined set of relation categories. It can predict one or more relationships, or predict NA labels that indicate no relationship exists between entity pairs.

[0080] Entity Collection Each entity in Each corresponds to a set of entity mentions. :

[0081] ;

[0082] in, Representing entities The set of entity mentions; This indicates the index of the entity mention within the entity mention set; Representing entities The One entity was mentioned; Representing entities The number of times the corresponding entity is mentioned.

[0083] S1.2, Lexical Sequence The segments are divided to obtain the word subsequences corresponding to each segment;

[0084] To handle long texts, this implementation uses a sliding window to process word sequences. Segment; The document is divided into subsequences of lexical units using a sliding window, such that each subsequence corresponds to a specific text segment of the document, and the end of the text segment corresponding to the preceding subsequence overlaps with the beginning of the text segment corresponding to the following subsequence. Specifically, the sliding window is set to a fixed lexical length. Set the length of overlapping words between adjacent text segments to 1. , and thus Divided into A sequence of lexical subsequences that overlap in content :

[0085] ;

[0086] in, Index representing a subsequence of lexical terms; Indicates that the document is The word sequence partitioning obtained by the first The nth word subsequence, corresponding to the nth word... A text fragment; This represents the total number of text segments obtained from the division; This indicates a fixed sliding window lexical length. Indicates the length of overlapping terms between adjacent text segments. and These are all empirical parameters, and their values ​​will vary depending on the document. The publicly available empirical configuration is: Re-DocRED. and They are 1536 and 128 respectively, DWIE's and They are 3000 and 128 respectively; Indicates according to the length of the sliding window lexicon Length of overlapping terms between adjacent text segments , for word sequence Functions for performing segmentation operations.

[0087] S2, divide each word subsequence The fragment-level lexical hidden representations and fragment-level attention weights are encoded by inputting them into a pre-trained language model to obtain the corresponding fragment-level lexical hidden representations and fragment-level attention weights. The fragment-level lexical hidden representations and fragment-level attention weights of each lexical subsequence are then merged to obtain the document. Document-level lexical hiding representation and document-level attention weights;

[0088] S2.1, the result obtained in S1 Each subsequence of lexical units is input into a pre-trained language model for encoding. The lexical hidden representations and attention weights output from the last three layers of the pre-trained language model are then averaged to obtain the fragment-level lexical hidden representations and fragment-level attention weights for each text segment.

[0089] For the The word subsequence corresponding to each text segment The encoding result is represented as:

[0090] ;

[0091] in This means that a pre-trained language model is used to encode the subsequence of lexical units corresponding to a certain text segment, and the lexical hidden representation and attention weights output by the last three layers of the model are averaged. Indicates the first Fragment-level lexical hidden representations corresponding to each lexical subsequence; Indicates the first Fragment-level attention weights corresponding to each word subsequence.

[0092] S2.2 The encoding results of each text segment are sorted according to each word in the document. Align and merge the corresponding positions in the document to obtain the document-level lexical hiding representation. Document-level attention weights ;

[0093] For fragment-level lexical hiding representation, the lexical hiding representation in each text fragment is mapped to the corresponding lexical representation in the document. The position of the word in the text; for a word that appears in multiple overlapping text segments at the same time, the word's corresponding word hidden representation in different text segments is accumulated, and then averaged according to the number of times the word is covered by multiple text segments, so as to obtain the document-level word hidden representation corresponding to the word.

[0094] For fragment-level attention weights, the attention weights corresponding to each text fragment are mapped according to the global index of the term in document D. The attention weights mapped to the same global position are summed and aggregated, and the aggregated attention weight matrix is ​​normalized row by row (i.e. for each query term) to obtain document-level attention weights.

[0095] The result is shown in the following formula:

[0096] ;

[0097] = ;

[0098] in, This indicates that the encoding results of each text segment are arranged in the document according to their corresponding tokens. Perform alignment and merging operations on the positions within; Indicates the first [item] in the input document The lexical hidden representation corresponding to each lexical unit contains Each feature dimension; based on each terminology in the document The order of arrangement in the middle will The hidden representations of each term are stacked vertically in the row direction to obtain the document. Document-level lexical hiding representation ;document attention weights .

[0099] Document-level lexical hiding representation Used to characterize documents The contextual semantic information of each word is used to construct entity representations and entity-to-context representations; document-level attention weights. Used to characterize documents The attention relationships between each word are used to construct entity attention representations and calculate the importance weights of entity pairs for their corresponding words.

[0100] S3. Construct entity representations for each entity based on document-level lexical hiding representation; construct entity attention representations for each entity based on document-level attention weights; construct normalized lexical importance weights for entity pairs based on the entity attention representations of the head entity and the tail entity; construct global context representations and sentence-aware context representations for entity pairs based on the normalized lexical importance weights for entity pairs.

[0101] This implementation method is based on the document. Document-level lexical hiding representation Based on the lexical hiding representations corresponding to the entity mentions of each entity, construct the entity representation of the corresponding entity; based on the document Document-level attention weights Based on the attention weights corresponding to the entity mentions of each entity, construct the entity attention representation for that entity; based on the head entity... Entity attention representation Tail-end entity Entity attention representation Construct entity pairs Normalized lexical importance weights According to entity pair Normalized lexical importance weights We perform weighted pooling on the hidden representations of each word within the sentence to construct two contextual representations for the entity pair—the global contextual representation. and sentence-aware context representation ;

[0102] S3.1, For entities and its entity mention set By summing the logarithms and the exponents The entity references in the text are aggregated using the corresponding lexical hidden representations to obtain the entity. entity representation ;

[0103] ;

[0104] in, Indicates entity mention The corresponding lexical hidden representation; Representing entities The number of times the corresponding entity is mentioned.

[0105] S3.2, For entities and its entity mention set The attention weights corresponding to each entity are averaged to achieve aggregation, resulting in the entity... Entity attention representation :

[0106] ;

[0107] in, This represents the number of attention heads in the multi-head attention mechanism of the pre-trained language model; Indicates entity mention The corresponding attention representation, Representing entities The number of times the corresponding entity is mentioned.

[0108] S3.3, For entities with a head. Tail-end entity The entity pairs constituted According to the head entity Entity attention representation Tail-end entity Entity attention representation Calculate entity pairs Normalized lexical importance weights ;

[0109] Specifically, considering each attention head, for the head entity Entity attention representation Tail-end entity Entity attention representation Element-wise multiplication yields the corresponding word-level joint attention vector; then, the word-level joint attention vectors corresponding to each attention head are averaged, and the averaged vector is then processed. Normalization, thus obtaining entity pairs Corresponding normalized lexical importance weights :

[0110] ;

[0111] in, Indicates the first [item] in the input document The lexical importance weights corresponding to each lexical are scalars used to represent the importance of each lexical in the input document for entity pairs. The greater the weight of a word element, the more important it is in constructing the contextual representation of the entity pair.

[0112] S3.4, Based on the normalized lexical importance weights We construct global context representations of entity pairs and sentence-aware context representations.

[0113] Global context is used to construct initial features for entity pairs corresponding to the original document, while sentence-aware context is used to construct initial features for entity pairs corresponding to the pseudo-document. For a document containing multiple sentences, the original document corresponds to all sentences in the document, while the pseudo-document corresponds to the sentences containing the head entities. or tail entity All sentences mentioned by the entity The original document is based on the document. The complete collection of sentences Constructing a contextual representation; pseudo-documents are based on entities including headers. or tail entity The set of sentences mentioned by the entity Construct a context representation.

[0114] (1) Construct a global context representation of entity pairs;

[0115] For entity pairs In the document The entire sentence range Internally, based on the normalized lexical importance weights Weighted pooling is performed on the hidden representations of each lexical unit to obtain entity pairs. global context representation As shown in the formula below:

[0116] ;

[0117] in, Indicates the input document The total number of sentences contained. Represents the first sentence in the set of sentences. One sentence. Represents the first word in the lexical sequence Each word element, Indicates the first Hidden representation of each word element Represents entity pairs The corresponding number Normalized importance weights of each word element.

[0118] (2) Construct sentence-aware context representations of entity pairs;

[0119] In addition to global context representation, entity pair context can also be constructed using a sentence-aware approach. Specifically, this involves only representing the context from the header entity. or tail entity Extract entity pairs and contextual information from sentences in which entities are mentioned.

[0120] set up Indicates that it contains a header entity. or tail entity The set of sentences mentioned by the entity, and:

[0121] ;

[0122] In the sentence collection Within the scope, the hidden representations of each lexical unit are weighted and pooled according to the normalized lexical unit importance weights to obtain entity pairs. Sentence-aware context representation As shown in the formula below:

[0123] ;

[0124] in, Indicates that it contains a header entity. or tail entity The set of sentences mentioned by the entity; Sentence It belongs to the set of sentences mentioned above.

[0125] S4. Based on the head and tail entity representations and the entity pair context representation, construct initial entity pair features for each entity pair in the document, and initialize the entity pair mesh using the initial entity pair features of each entity pair.

[0126] S4.1 Based on the entity representation and entity pair context representation obtained in S3, construct the initial entity pair features for each entity pair in the document.

[0127] For the head entity Tail-end entity The entity pairs constituted Let its corresponding entity be represented in context as ,like Used to construct the initial features of the entity pairs corresponding to the original document. Retrieve global context representation ,Right now ,like The initial features used to construct the entity pairs corresponding to the pseudo-documents are then... Sentence-aware context representation ,Right now ,Right now:

[0128] ;

[0129] For entity pairs First, the head entity is represented. Entity representation of context The feature dimension is concatenated to obtain and to The projection matrix can be learned from the model. Perform a linear transformation to obtain And then Nonlinear transformation using the hyperbolic tangent activation function is used to obtain head-side features. :

[0130] ;

[0131] The learnable projection matrix of the model is the learnable parameter matrix of the model. Its parameter values ​​are determined during the model training process by calculating the gradient through backpropagation based on the training loss and using optimization algorithms for iterative updates.

[0132] Similarly, representing the tail entity Entity representation of context The feature dimension is concatenated to obtain and to The projection matrix can be learned from the model. Perform a linear transformation to obtain And then Tail-side features are obtained by performing a nonlinear transformation using the hyperbolic tangent activation function. :

[0133] ;

[0134] In obtaining entity pairs Head side features and tail side features Then, both are simultaneously input into the grouped bilinear function. Interactive modeling is performed: Let the grouping size of the grouped bilinear function be... ,but The function performs pairwise interactions on features within each group and concatenates the interaction results. Therefore, the output dimension of this function is... Then, the modeled results are processed using the model-learnable projection matrix. Perform mapping to obtain entity pairs Entity pairs with initial features As shown in the formula below:

[0135] ;

[0136] in, It is also located in the entity pair mesh to be constructed later. The initial characteristics of the grid cells.

[0137] S4.2 Construct an initial entity pair mesh using the initial entity pair features of each entity pair.

[0138] Set document Total of An entity, which Each entity pair of an entity has its corresponding initial features, based on the head entity index. End-of-Pack Entity Index This corresponds to the corresponding cells in the entity pair mesh, thus constructing the initial entity pair mesh. :

[0139] ;

[0140] in, Represents entity pairs The entity pairs have initial features.

[0141] Therefore, each entity pair Corresponding initial entity pair mesh China and Israel For the header entity index, with This is a grid cell indexed by the tail entity. All grid cells in a row share the head entity. , No. All grid cells in a column share the tail entity. The initial entity constructed is a mesh. This is used for subsequent information interaction and entity-guided axial attention mechanism-based feature updates of the mesh.

[0142] S5. Update the entity pair mesh based on the entity-guided axial attention mechanism to obtain the enhanced entity pair representation.

[0143] The axial attention refers to the aggregation of attention information along the row and column directions of the entity mesh, respectively; the entity guidance refers to the introduction of the representation of the corresponding head entity for guidance during row aggregation and the introduction of the representation of the corresponding tail entity for guidance during column aggregation; the entity-guided axial attention mechanism is an attention mechanism that combines the two.

[0144] Building Documents Entity representation matrix ;based on Stacked in series, level by level The entity-guided axial attention (EGA) mechanism uses an initial block to guide the axial attention of entities on the mesh. ,Do This is the second update;

[0145] S5.1, Building Documents Entity representation matrix .

[0146] Set document Includes An entity, by this The entities constitute an entity representation matrix. :

[0147] ;

[0148] in, entity The entity representation.

[0149] S5.2, Based on Entity Representation Matrix Construct a hierarchical update strategy for entities to the mesh.

[0150] To explicitly model the interaction relationships between different entity pairs at the entity pair level, starting from the initial entity pair mesh... Initially, based on the entity representation matrix Stacked in series, level by level The entity guides the axial attention block, for the first entity... Layer entity to mesh Overall update as follows:

[0151] ;

[0152] in, Indicates the first Layer entities guide the input entity pairs of the axial attention block to the mesh; Indicates after the first The layer entity guides the updated entity pair mesh output by the axial attention block; The layer index representing the entity-guided axial attention block; It is an empirical value representing the total number of layers of entity-guided axial attention blocks, that is... Number of blocks; Indicates to and Apply a solid to guide the axial attention block.

[0153] S5.3, in the In the layer entity guide axial attention block, first based on the head entity entity representation Construct line direction entity guide mark Then through The input entity pair mesh of this attention block The Entity pair feature sequence of a row The execution header entity guides the aggregation of row directions.

[0154] In the In the layer entity guiding axial attention block, for the head entity First, based on the head entity representation Generate the corresponding row direction entity guide mark :

[0155] ;

[0156] in, Indicates the first Layer Head Entity The line direction entity guide marker of the current line; Represents head entity Entity representation; This represents the learnable projection matrix used to generate row-direction entity guide markers, with its superscript... To indicate the entity guide mark used for generating The matrix function identifier of the learnable projection matrix, its subscript This indicates that the matrix is ​​applied in the row direction.

[0157] Guide the entity in the direction of the line. Pre-placed on solid-to-mesh The Entity pair feature sequence of a row Then, row direction attention is calculated on the combined sequence to obtain the updated row direction entity guidance marker. Entity pair feature sequences updated by row direction As shown in the formula below:

[0158] ;

[0159] in, Indicates to Perform line-direction entity guidance tagging after line-direction attention update; Indicates to After performing row direction attention update, the entity pair feature sequence of the corresponding row in the next layer entity pair grid; Indicates the first The row direction attention update operation performed by the layer entity-guided axial attention block is a component operation in the ℓth layer entity-guided axial attention block; the semicolon ";" indicates that the entity guide marker and the entity pair feature sequence are concatenated along the dimension of the sequence.

[0160] S5.4 Utilize the updated row direction entity guide marker Features of each entity pair after row direction update Continue with feature modulation.

[0161] To further inject head entity semantics into the entity pair features along the row direction, the updated row direction entity guide markers are used. Generate line direction modulation parameters, including scaling parameters. and offset parameters Furthermore, the FiLM (Feature-wise Linear Modulation) method is used to modulate the features of each entity pair after the row direction update, as shown in the following formula:

[0162] ;

[0163] ;

[0164] in, The superscript represents the learnable projection matrix used to generate line direction modulation parameters. Used to identify the projection matrix used in generating FiLM parameters; This represents the scaling parameters generated from the header entity semantics; This represents the offset parameter generated by the header entity semantics; This indicates that the feature sequence of the entity pair is located after the row-direction attention update. No. Entity pair features of the column; Indicates to Row-direction entity pair features after FiLM modulation; This represents element-wise multiplication; This represents the hyperbolic tangent activation function.

[0165] In entity to mesh In the middle, the first All grid cells in a row share the head entity. Therefore, the first Each entity in the row has a feature All correspond to the same updated line direction entity guide marker. Therefore, use Parameters generated and Modulation Each entity in the row has a feature ;

[0166] S5.5 Similarly, perform column-direction aggregation and feature modulation guided by tail entities. Entity-guided row-direction aggregation and feature modulation and tail entity-guided column-direction aggregation and feature modulation are performed simultaneously.

[0167] For tail entities According to the tail entity representation Generate the corresponding column direction entity guide mark :

[0168] ;

[0169] in, Represents tail entity The corresponding column direction entity guide marker; This represents the learnable projection matrix used to generate column-direction entity guide markers.

[0170] Guide the column direction entity with a marker. Pre-placed on solid-to-mesh The Entity pairs of feature sequences in a column And perform column-direction attention calculation to obtain the updated column-direction entity guidance marker. Entity pair feature sequences updated in column direction Subsequently, the updated column-direction entity guide markers are used. Generate the corresponding scaling parameters and offset parameters The column direction update results are then subjected to feature modulation using the same FiLM modulation method as in step S5.4 to obtain the column direction modulated entity pair features. .

[0171] S5.6, Regarding the first The input entity of the entity guiding the axial attention block is a mesh. By fusing the row and column update results of the current layer through a gating mechanism, an incremental feature grid for entity pairs is obtained. .

[0172] For the Layer entity to mesh Each grid cell in the array is now assumed to be located in... Based on the features of the upper-level entity Constructing a multilayer perceptron to predict two-dimensional gate vectors As shown in the formula below:

[0173] ;

[0174] in:

[0175] ;

[0176] Using two components in a two-dimensional gated vector and Update the results for each row direction. Column direction update results The weighted summation yields the entity pair feature increments after fusing the row and column direction updates. :

[0177] ;

[0178] in, Indicates the first In the entity-guided axial attention block, the input entity is targeted at the mesh. Central The two-dimensional gated vector corresponding to the grid cell; Indicates the gating coefficient corresponding to the row direction update result; This represents the gating coefficient corresponding to the column direction update result; This represents a multilayer perceptron used to predict the gate vector; This represents the Sigmoid activation function.

[0179] S5.7 Incremental mesh of entity pairs obtained from S5.6 The entity pair mesh is updated by adding residuals and using a position feedforward network. .

[0180] First, the entity pair feature incremental mesh obtained from S5.6 is processed. Entity to Mesh By summing the residuals, we obtain the intermediate solid pair mesh. Subsequently, the intermediate entity is paired with the mesh. Perform layer normalization, input the layer normalization result into the position feedforward network, and then obtain the first layer by summing the residuals. Layer entity to mesh Specifically, it is expressed as:

[0181] ;

[0182] ;

[0183] in, This represents an intermediate entity pair mesh formed after fusing row and column direction features; Indicates the first Layer entities guide the output of the axial attention block's entity pair mesh; Presentation layer normalization operation; This indicates the location feedforward network processing.

[0184] Repeat the above steps in sequence. Layer entities guide axial attention updates, thereby obtaining updated entity pairs in the mesh. .

[0185] S5.8. The initial features and updated features of entity pairs are fused by residual gating to obtain the final entity pair representation.

[0186] Finish After layer entity-guided axial attention update, for each mesh cell in the entity pair mesh, it is assumed that it is located in the entity pair mesh. Location, based on its entity's initial features Calculate the scalar gating coefficient and utilize For entity pairs of initial features and the process Entity pair features updated at each layer The interpolation and fusion are performed as shown in the following formula:

[0187] ;

[0188] ;

[0189] in, Represents a scalar gating coefficient that acts on the mesh of the entity pair. Grid cells; This represents the learnable parameters used to generate scalar gating coefficients; Indicates the location of the initial entity pair mesh. The initial features of the entity pairs; Indicates passing through in sequence Layer entity guided axial attention update located in the entity pair mesh Entity pair features; This indicates that the final result is located in the solid pair mesh. The entity pair feature of the location of the mesh cell, i.e., entity pair The final entity pair representation.

[0190] The final entity pair mesh, composed of the final entity pair features of all entity pairs, is represented as:

[0191] ;

[0192] in, This represents the final entity pair mesh after entity-guided axial attention update, and serves as the input for subsequent relation prediction.

[0193] S6. The relation classifier outputs the predicted scores of the relation category and threshold category corresponding to each entity pair feature in the final entity pair grid, and the relation classifier is trained using adaptive dual-focus loss.

[0194] In the relationship prediction process, only those satisfying... To determine the relationship between entities.

[0195] S6.1, Pair the final entity with the mesh. The input is fed into a relation classifier, which outputs a predefined relation category prediction score and a threshold category prediction score for each entity corresponding to the feature, thus obtaining... The predicted fraction tensor.

[0196] Let the predefined set of relation categories be And an additional threshold category is introduced. . These are auxiliary categories set up during the relation classification process. They do not correspond to the actual entity relation semantics; their purpose is to provide a learnable discrimination benchmark for each predefined relation category. The relation classifier outputs prediction scores for each entity pair for each predefined relation category and a prediction score for a threshold category. When predicting relations, each relation category... Predicted score and threshold category When compared with the predicted scores, The predicted score is higher than When predicting the score, the relationship category is used as the prediction result for the relationship category of the current entity pair.

[0197] After classifying the features of each entity pair in the final entity pair mesh, the prediction score tensor is obtained. It is represented as:

[0198] ;

[0199] in, The tensor representing the predicted score output by the relation classifier; Indicates the number of entities in the input document; Represents a predefined set of relation categories; Indicates the threshold category; This represents the size of the category set consisting of the relation category and the threshold category. For entity pairs... The relation classifier outputs each relation separately. Corresponding predicted score and threshold categories Corresponding predicted score .

[0200] S6.2. Adaptive dual-focus loss is used to train the relation classifier.

[0201] During the training phase of the relation classifier, for entity pairs Let its positive class relation set be The set of negative class relations is ,in:

[0202] ;

[0203] ;

[0204] in, , Represents entity pairs The corresponding set of positive relationships, that is, the set of relationship categories that the entity actually has; Represents entity pairs The corresponding set of negative relation classes, that is, all relation classes other than positive relation classes among all predefined relation classes; It represents the set of all predefined relation categories.

[0205] To learn adaptive threshold class scores for entity pairs under imbalanced class conditions, this implementation uses Adaptive Dual Focal Loss (ADFL) as the training objective for the relation classifier. First, the positive class probability is calculated based on the predicted scores for each relation class and the predicted scores for the threshold class. and negative class threshold probability As shown in the formula below:

[0206] ;

[0207] ;

[0208] in, Represents positive class relation In the set of positive class relations With threshold category Normalized probabilities in the set of categories; Indicates threshold category In the set of negative class relations With threshold category Normalized probabilities in the set of categories; Represents relation categories The corresponding predicted score; Indicates threshold category The corresponding predicted score; This indicates exponentiation.

[0209] In obtaining the positive class probability and negative class threshold probability Afterwards, respectively and Perform focus reweighting. For example... Figure 2 As shown. The positive class loss, negative class loss, and the final adaptive dual-focus loss are respectively expressed as:

[0210] ;

[0211] ;

[0212] ;

[0213] in, This indicates a set of positive class relations. The calculated positive class loss; This indicates a set of negative class relations. The calculated negative class loss; This represents the adaptive dual-focus loss; and These represent the weight coefficients corresponding to the positive class loss and the negative class loss, respectively. and These represent parameters used to control the focus level of the positive and negative classes, respectively. This represents logarithmic operations; the objective function for training a relation classifier is to minimize... .

[0214] By applying focus reweighting to the positive and negative class losses respectively, the adaptive dual-focus loss can be applied to both positive and negative classes simultaneously, thereby alleviating the imbalance between positive and negative classes in document-level relation extraction tasks and being used to learn the corresponding adaptive thresholds for each entity pair.

[0215] In the experimental setup of this implementation, the AdamW optimization algorithm was used to optimize the model parameters. For the Re-DocRED dataset, and All are set to 1.0. Set to 0.0, Set to 4.0; for the DWIE dataset, and All are set to 1.0. and All are set to 1.0. Among the parameters mentioned above, the parameter... The value of is determined through hyperparameter experiments, while the others are empirical values.

[0216] S7. During the inference phase of the relation classifier, global multi-threshold calibration is performed on the relation prediction results of each entity pair to obtain calibrated relation prediction results.

[0217] Although S6 passes the threshold category Adaptive threshold category scores are learned for different entity pairs, but inconsistencies in calibration may still exist in the predicted scores between different entity pairs, leading to misclassification or omission of some relations. Therefore, in the relation classifier inference stage, Global Multi-Threshold Calibration (GMTC) is used to further calibrate the relation prediction results of each entity pair. By introducing global thresholds corresponding to multiple relation sub-label spaces on top of the adaptive threshold category scores, a combination of local adaptive thresholding and global calibration is achieved. Figure 3 As shown.

[0218] S7.1, Obtain each relation category Final relation boundary score This is used as the input for subsequent global multi-threshold calibration.

[0219] For any relation category First, calculate the predicted score for the relationship category. Relative to threshold category Predicted score Boundary fractions :

[0220] ;

[0221] in, Represents relation categories Relative to threshold category Boundary fractions; Represents relation categories The corresponding predicted score; Indicates threshold category Corresponding predicted score; superscript This represents the document corresponding to the prediction result, and:

[0222] ;

[0223] in, This represents the prediction result corresponding to the original document. This represents the prediction result corresponding to the pseudo-document. Therefore, the relation boundary score corresponding to the original document is represented as... The relation boundary score corresponding to the pseudo-document is represented as .

[0224] The relation boundary score corresponding to the original document Relational boundary score corresponding to pseudo-document Perform weighted fusion to obtain relationship categories. The corresponding final relation boundary fusion score Specifically, for the union of the candidate prediction results of the original document and the pseudo-document, if the relation category... It appears in both types of documents and satisfies and This indicates that both types of documents independently support the relationship and directly accept the relationship category. .

[0225] For relation categories that are not directly accepted, the final relation boundary score is calculated as follows: :

[0226] (1) If relation class r appears in both the original document and the pseudo-document, then:

[0227] ;

[0228] (2) If relation class r only appears in the original document, then for Imposing punishment:

[0229] ;

[0230] (3) If the relation class r only appears in the pseudo-document, then for Imposing punishment:

[0231] ;

[0232] in, This represents the relation boundary score corresponding to the original document; This represents the relation boundary score corresponding to the pseudo-document. For candidate relations that are not directly accepted, their final relation boundary scores are calculated. Used for subsequent global multi-threshold calibration.

[0233] S7.2 Divide each relation category into sub-label spaces and determine the corresponding global threshold for each sub-label space.

[0234] First, the frequency of each relation category in the training data is counted, and all relation categories are sorted according to their frequency. Then, based on the preset relation subspace partitioning rules, the sorted relation categories are divided into subspaces. In each sub-label space, each relation category corresponds to a sub-label space. The space of individual tags is represented as follows:

[0235] ;

[0236] in, This indicates the number of sub-label spaces obtained from the partitioning; Indicates the index of the sub-label space; Indicates the first Each relation category is assigned to a corresponding sub-label space based on its frequency of occurrence.

[0237] For the Individual tag space Collect predicted relation categories belonging to the sub-label space on the development set. Candidate relations are sorted from highest to lowest according to their final relation boundary scores. F1 scores are then calculated for each candidate relation at different thresholds, and the candidate score that maximizes the F1 score of the development set is determined as the global threshold corresponding to the sub-label space. .

[0238] in, Indicates the first Individual tag space The corresponding global thresholds. Thus, different sub-label spaces correspond to different global thresholds, thereby using multiple global thresholds to calibrate the prediction results for different relation categories.

[0239] S7.3, Based on the final relation boundary score of relation r and the global threshold of the sub-label space to which this relationship belongs. The system determines whether to accept or reject the relationship category as the predicted relationship for the corresponding entity pair.

[0240] During the reasoning phase, for S7.1 that simultaneously satisfy... and For a relation category r, the relation category is directly accepted, and a global threshold comparison is no longer performed.

[0241] For the remaining candidate relation categories According to its belonging number Individual tag space The final relation boundary score The global threshold corresponding to this sub-label space Compare. When satisfied... When accepting relationship categories As a predicted relation for the corresponding entity pair; if the above conditions are not met, the relation category is not accepted.

[0242] Therefore, in the relationship prediction process, the threshold categories in S6 are utilized on the one hand. An adaptive relationship judgment benchmark is provided for different entity pairs, and a boundary score is obtained by the difference between the relationship category prediction score and the threshold category prediction score; on the other hand, a corresponding global threshold is set according to different relationship sub-label spaces. The prediction results between different entity pairs are globally calibrated to form a relationship determination mechanism that combines local adaptive thresholds with global multi-thresholds at the relation sub-label space level.

[0243] S8. Summarize the predicted relationships corresponding to each entity pair in the document as the document-level relationship extraction result for that document.

[0244] Based on the relation prediction results obtained from S7 after global multi-threshold calibration, the input document... Each entity summarizes its corresponding predicted relationships to generate document-level relationship extraction results.

[0245] For the head entity Tail-end entity The entity pairs constituted If S7 determines the relation category For the predicted relation of this entity pair, then the head entity... Relationship Category Tail-end entity Combining these elements yields the corresponding relation triples:

[0246] ;

[0247] in, Indicates the head entity; Indicates the tail entity; This indicates the prediction relationship determined after global multi-threshold calibration by S7; This represents a relation triple consisting of a head entity, a relation category, and a tail entity.

[0248] Since document-level relation extraction is modeled as a multi-label classification task, pairs of entities... It can correspond to one or more prediction relations. Therefore, when S7 determines that the same entity pair corresponds to multiple relation categories, relation triples corresponding to each relation category are generated respectively.

[0249] For example, when entity pairs The corresponding prediction relationships include and At that time, we obtained the following results:

[0250] ;

[0251] as well as:

[0252] ;

[0253] When entity pairs Not found belonging to the predefined set of relations When predicting a relationship, the entity corresponds to the no-relationship result NA.

[0254] The above processing is performed on the prediction relations corresponding to each entity pair in the input document to obtain the input document. The document-level relation extraction results.

[0255] Compared to existing document-level relation extraction methods, the method of this invention explicitly models the interaction relationships between entity pairs through an entity-guided axial attention mechanism, enabling the model to more effectively utilize cross-sentence information and intermediate entity information for two-hop reasoning, such as... Figure 1 As shown; the problem of imbalanced positive and negative samples is alleviated by adaptive dual-focus loss, such as... Figure 2 As shown; and by using a global multi-threshold calibration mechanism to improve the consistency of prediction results between different entity pairs, such as Figure 3 As shown in the figure. Experimental results demonstrate that the method of this invention achieves superior or competitive performance on both the DWIE and Re-DocRED document-level relation extraction datasets.

[0256] As shown in Tables 1 and 2, the method of this invention achieved good F1 and Ign-F1 scores on both the DWIE and Re-DocRED datasets, indicating that the method can effectively improve the accuracy of document-level relation extraction. Table 3 shows that, compared with existing loss functions, the adaptive dual-focus loss ADFL proposed in this invention achieved the highest F1 and Ign-F1 scores on the Re-DocRED test set, indicating that this loss function can better alleviate the class imbalance problem in document-level relation extraction and improve the model's ability to identify true relations.

[0257] Table 1 Experimental results based on the DWIE dataset

[0258]

[0259] Table 2 Experimental results based on the Re-DocRED dataset

[0260]

[0261] Table 3. Experimental results of different loss functions on the Re-DocRED test set.

[0262]

[0263] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope defined by the present invention.

Claims

1. A document-level relation extraction method based on entity-guided axial attention and dual-threshold calibration, characterized in that, The method includes: Obtain the original document, as well as the document's sentence set, entity set, and entity mention set, and preprocess the document to obtain the lexical sequence and lexical subsequence; Based on lexical subsequences, document-level lexical hidden representations and document-level attention weights are obtained; Encode each lexical subsequence to obtain fragment-level lexical hidden representations and fragment-level attention weights. Merge each fragment-level lexical hidden representation and fragment-level attention weight to obtain document-level lexical hidden representations and document-level attention weights. Entity representations and entity attention representations for each entity are constructed based on document-level lexical hiding representations and document-level attention weights; based on the entity attention representations of the head and tail entities, normalized lexical importance weights for entity pairs are constructed, and then contextual representations of entity pairs are constructed. Based on the entity representations and context representations of the head and tail entities of the entity pair, construct the initial features of each entity pair in the document, and initialize the entity pair mesh using the initial features of each entity pair. The final entity pair mesh is obtained by updating the entity pair mesh based on the entity-guided axial attention mechanism. A relation classifier is trained using adaptive dual-focus loss. During the inference phase of the relation classifier, the relation category of each entity pair in the final entity pair grid is predicted, and a global multi-threshold calibration is performed on the relation category prediction results to obtain the calibrated relation prediction results.

2. The document-level relation extraction method with entity-guided axial attention and dual-threshold calibration according to claim 1, characterized in that, The document preprocessing process, which yields a lexical sequence and lexical subsequences, includes: Input the original document into the pre-trained word segmenter to obtain a sequence of words; A sliding window is used to segment the word sequence, resulting in word subsequences for each segment.

3. The document-level relation extraction method based on entity-guided axial attention and dual-threshold calibration according to claim 2, characterized in that, The process of obtaining document-level lexical hidden representations and document-level attention weights based on lexical subsequences includes: Each word subsequence is input into a pre-trained language model for encoding to obtain the fragment-level word hidden representation and fragment-level attention weights corresponding to each word subsequence. The fragment-level lexical hidden representations and fragment-level attention weights corresponding to each lexical subsequence are aligned and merged according to the corresponding positions of each lexical in the document to obtain document-level lexical hidden representations and document-level attention weights.

4. The document-level relation extraction method with entity-guided axial attention and dual-threshold calibration according to claim 3, characterized in that, The method for constructing entity representations and entity attention representations for each entity based on document-level lexical hiding representations and document-level attention weights is as follows: Known Entities Entity mention set By summing the logarithms and the exponents Various entities mentioned Corresponding lexical hidden representation Aggregate to obtain entities entity representation Specifically, it is expressed as: ; through averaging Various entities mentioned Corresponding attention representation Aggregate to obtain entities Entity attention representation Specifically, it is expressed as: .

5. The document-level relation extraction method with entity-guided axial attention and dual-threshold calibration according to claim 4, characterized in that, The process of constructing normalized lexical importance weights for entity pairs based on the entity attention representations of the head and tail entities, and then constructing the contextual representation of the entity pairs, includes: For entity pairs Head entity Entity attention representation Tail-end entity Entity attention representation Element-wise multiplication is performed to obtain the word-level joint attention vector; the average of all word-level joint attention vectors is then performed, and the averaged vector is then... Normalization yields entity pairs Normalized lexical importance weights Specifically, it is expressed as: ,in Indicates the first [item] in the input document Each lexical unit has a corresponding lexical importance weight. Represents the word sequence of the input document; Based on the normalized lexical importance weights, the hidden representations of each lexical are weighted and pooled to obtain entity pairs. Context representation If each lexical element comes from the entire sentence of the document, then This represents the global context; if each lexical comes from a entity containing the head entity... or tail entity The sentence mentioned by the entity, then Context-aware representation of sentences.

6. The document-level relation extraction method based on entity-guided axial attention and dual-threshold calibration according to claim 5, characterized in that, The step of constructing initial entity pair features for each entity pair in the document based on the entity representations and context representations of the head and tail entities of the entity pair, and initializing the entity pair mesh using the initial entity pair features of each entity pair, includes: For entity pairs Represent the head entity Entity representation of context The feature dimension is concatenated to obtain and to A linear transformation is performed, followed by a nonlinear transformation of the result using the hyperbolic tangent activation function to obtain the head-side features. ; Represent the tail entity Entity representation of context The feature dimension is concatenated to obtain and to A linear transformation is performed, followed by a nonlinear transformation of the result using the hyperbolic tangent activation function to obtain the tail-side features. ; Head side features and tail side features Simultaneously input into the grouped bilinear function Interactive modeling is performed, and the modeling results are evaluated using a projection matrix. Perform mapping to obtain entity pairs The initial features of the entity pairs; The initial features of each entity pair in the document are indexed according to the header entity index. End-of-Pack Entity Index This corresponds to the corresponding cells in the entity pair mesh, constructing the initial entity pair mesh. .

7. The document-level relation extraction method with entity-guided axial attention and dual-threshold calibration according to claim 6, characterized in that, The entity-guided axial attention mechanism is used to update the entity pair mesh to obtain the final entity pair mesh. include: Document Included The entity representations of each entity are combined to form an entity representation matrix. ; From the initial entity pair mesh Initially, based on the entity representation matrix Stacked in series, level by level The entity guides the axial attention block, for the first entity... Layer entity to mesh Overall update to ,in, Indicates the first Layer entities guide the input entity pairs of the axial attention block to the mesh; Indicates after the first The layer entity guides the updated entity pair mesh output by the axial attention block; The layer index representing the entity-guided axial attention block; It is an empirical value representing the total number of layers of the entity-guided axial attention block; Indicates to and Apply a solid to guide the axial attention block; In the In the layer entity guide axial attention block, first based on the head entity entity representation Construct line direction entity guide mark ,in This represents the learnable projection matrix used to generate row-direction entity guide markers, and then through... The input entity pair mesh of this attention block The Entity pair feature sequence of a row Perform line direction aggregation guided by the head entity to obtain the updated line direction entity guide marker. Entity pair feature sequences updated with row direction ; Reuse Generate line direction scaling parameters and offset parameters For each entity pair feature after the row direction is updated Continue with feature modulation: ; ; in, This represents the learnable projection matrix used to generate the line direction modulation parameters; Similarly, perform column-direction aggregation and feature modulation guided by tail entities; Regarding the first The input entity of the entity guiding the axial attention block is a mesh. Each entity in the feature Constructing a multilayer perceptron to predict two-dimensional gate vectors ,use Two components and Update the results for each row direction. Column direction update results Weighting is performed to obtain the entity pair feature increments that fuse the row and column direction updates. This leads to the entity pair feature increment mesh. ; Incremental mesh of entity pairs features Entity to Mesh By summing the residuals, we obtain the intermediate solid mesh. After performing layer normalization, the result of layer normalization is input into the position feedforward network, and then the residuals are summed to obtain the first... Layer entity to mesh ; Repeat the above steps in sequence. Layer entities guide axial attention updates, thereby obtaining updated entity pairs in the mesh. ; For each mesh cell in the entity pair mesh, assume it is located in the entity pair mesh. Location, based on its entity's initial features Calculate the scalar gating coefficient ,in This represents the learnable parameters used to generate scalar gating coefficients, and utilizes... For entity pairs of initial features and the process Entity pair features updated at each layer Interpolation and fusion are performed to obtain entity pairs The final entity pair representation ; The final entity pair representations of all entity pairs are combined together to form the final entity pair mesh after entity-guided axial attention update, represented as: .

8. The document-level relation extraction method with entity-guided axial attention and dual-threshold calibration according to claim 7, characterized in that, The prediction of the relation category of each entity pair in the final entity pair grid during the relation classifier inference stage includes: Predefined set of relation categories And introduce a threshold category. The final entity will be meshed. The input is fed into a relation classifier, which outputs a predicted score for each predefined relation category and a predicted score for a threshold category for each entity pair. When predicting relations, each relation category... Predicted score and threshold category When compared with the predicted scores, The predicted score is higher than When predicting the score, the relationship category is used as the prediction result for the relationship category of the current entity pair.

9. The document-level relation extraction method based on entity-guided axial attention and dual-threshold calibration according to claim 8, characterized in that, The method for training the relation classifier using adaptive bifocal loss is as follows: Based on the predicted scores of each relation category and the predicted score of the threshold category Calculate the positive class probability respectively. and negative class threshold probability : right Re-weighting the focus yields the set of positive class relations. Calculated positive class loss ;in Represents entity pairs The corresponding set of positive relationships, that is, the set of relationship categories that the entity actually has; This represents the weighting coefficient corresponding to the positive class loss; This represents a parameter used to control the degree of focus for positive classes; right Re-weighting the focus yields the set of negative class relations. The calculated negative class loss ;in Represents entity pairs The corresponding set of negative relation classes, that is, all relation classes other than positive relation classes among all predefined relation classes; This represents the weighting coefficients corresponding to the negative class loss; This represents a parameter used to control the degree of focus on negative classes; The objective function for training the relation classifier is to minimize .

10. The document-level relation extraction method with entity-guided axial attention and dual-threshold calibration according to claim 8, characterized in that, During the inference phase of the relation classifier, a global multi-threshold calibration is performed on the relation category prediction results to obtain calibrated relation prediction results, including: For any relation category Calculation in the original document Predicted score Relative to threshold category Predicted score relation boundary score And in pseudo-documents Predicted score Relative to threshold category Predicted score relation boundary score If relation class r appears in both the original document and the pseudo-document, then let relation class r be... Final relation boundary score If the relation class r only appears in the original document, then for Impose punishment, make If the relation class r only appears in the pseudo-document, then for Impose punishment, make ; The frequency of each relation category in the training data is counted, and all relation categories are sorted according to their frequency. Then, based on the preset relation subspace partitioning rules, the sorted relation categories are divided into subspaces. In each sub-label space, each relation category corresponds to a sub-label space. The sub-label space is represented as ,in, Indicates the index of the sub-label space; Indicates the first Individual tag space; For the Calculate the global threshold in the sub-label space. For the first Individual tag space Collect predicted relation categories on the development set. Candidate relations are sorted from highest to lowest according to their final relation boundary scores. F1 scores are then calculated for each candidate relation at different thresholds. The final relation boundary score of the candidate relation that maximizes the F1 score in the open set is taken as the final relation boundary score. global threshold ; During the reasoning phase, for simultaneously satisfying and For the relation category r, r is directly accepted as the relation prediction result for the corresponding entity pair; for the remaining candidate relation categories... , and its final relation boundary score With its own Global threshold of sub-label space Compare, if satisfied If the relation class r is true, then the relation class r is accepted; otherwise, r is not accepted.