A document-level relation extraction method and system based on information gain and prototype contrastive learning

Abstract

This invention belongs to the field of natural language processing in computer intelligent information processing. It discloses a document-level relation extraction method and system based on information gain and prototype contrastive learning. Specifically addressing the document-level relation extraction task, this invention proposes a graph-based document-level relation extraction model that considers both extracting more accurate node features and alleviating data imbalance. Existing document-level relation extraction models typically employ graph-based models, which inherently face the problem of data imbalance. Current research suffers from issues such as noise interference from irrelevant nodes and edges during node feature updates, insufficient model learning ability for true relations due to excessive negative samples in documents, and the inability of multi-label classification to accurately predict all different relation types.

Description

Technical Field

[0001] This invention belongs to the field of natural language processing in computer intelligent information processing, specifically involving a document-level relation extraction method and system based on information gain and prototype contrast learning. Background Technology

[0002] With the rapid development of large-scale text data such as social media, news reports, and academic literature, document-level relation extraction (DocRE) has become a highly anticipated research task in the field of natural language processing (NLP). By identifying the relationships between entities in documents, we can gain a more comprehensive understanding of complex events, social interactions, and the expression of opinions, thereby driving technological advancements in information extraction and knowledge management, and providing important support for fields such as intelligent search, public opinion analysis, and knowledge graph construction.

[0003] In the field of document-level relation extraction, processing large-scale text data presents increasingly complex scenarios and demands. To better understand the relationships between entities in documents, existing models can be divided into two main lines: Transformer-based models and graph-based models.

[0004] Transformer-based models accept only word sequences from a document as input and utilize Transformer to implicitly capture long-term contextual dependencies between entities. Specifically, they leverage the Transformer architecture and attention mechanisms to capture relationships between entities in text, effectively capturing long-distance dependencies and thus modeling complex interactions between entities across the entire document. By modeling the global context, Transformer-based models can better understand the context and associations of entities within a document, thereby improving the accuracy and efficiency of document-level relation extraction tasks. These models typically use pre-trained language models (such as BERT and RoBERTa) for representation learning and combine task-specific fine-tuning methods to achieve document-level relation extraction. Their advantage lies in their ability to handle longer texts and capture global relationships, thus possessing significant application potential in document-level relation extraction tasks. Tang et al. proposed a hierarchical reasoning network that obtains entity-level reasoning information through translation constraints and bilinear transformations, models the reasoning relationships between entities and sentences, and uses hierarchical aggregation to obtain document-level reasoning information. Ye et al. proposed the CorefBERT model, which improves text coherence understanding by explicitly modeling referential relationships, demonstrating outstanding performance in referential reasoning tasks. Zhou et al. proposed adaptive thresholding and local context pooling techniques. The former learns entity-related thresholds to optimize multi-label classification, while the latter utilizes local context information to improve relation extraction accuracy, significantly improving model performance.

[0005] Graph-based models represent documents as graphs, where nodes represent entities and edges represent relationships between entities. These models utilize graph neural networks for information transfer and aggregation, capture inter-entity relationships through multi-head attention mechanisms, iteratively update node representations, and achieve global reasoning. This allows for a better understanding of the complex relationships between entities in a document, improving the accuracy and robustness of relation extraction. Zeng et al. proposed the Graph Aggregation Reasoning Network (GAIN), which constructs a mention-level graph (MG) and an entity-level graph (EG), combining path reasoning mechanisms to achieve document-level relation extraction. Wei et al. utilized language tools to construct graph structures such as coreference edges to capture cross-sentence and intra-sentence dependencies. Li et al. proposed the Graph Augmentation Dual Attention Network (GEDA), which interacts with heterogeneous graph convolutional networks through S2R / R2S attention mechanisms and designs an evidence-based regularizer to improve performance.

[0006] Graph-based models acquire current entity features by iteratively updating the current node feature representation using a graph network (GCN) based on the nodes and edges in the graph structure. However, since all surrounding information is incorporated into the current node encoding, it is inevitably affected by irrelevant information from irrelevant nodes and edges. Furthermore, due to the multi-sentence nature of cross-sentence reasoning for entity pairs, acquiring the contextual features of the current entity pair is inevitably interfered with by factors in the sentence that are irrelevant to the current entity pair. To address these issues, we propose a node and edge filtering module based on information gain and an attention filtering module based on contextual features. In this way, the model can focus more on the current entity features and the more effective contextual features for judging the relationship between the current entity pairs. More specifically, regarding the first issue, to enable the model to acquire more realistic and effective entity features, we calculate the similarity of node embeddings in the graph structure, convert it into a probability distribution, and then calculate the information entropy of each edge. By filtering out node pairs without edges using the adjacency matrix and limiting the entropy value to [0,1], the entropy value of each edge in the final output graph is used to measure the determinism of the relationship between nodes. For the second problem, in order to effectively filter out noise information in the context features of the current entity pair, the combination of the mean and standard deviation of two common statistical measures in the statistical data is used as a dynamic threshold to effectively filter the attention matrix output by the pre-trained model, thereby reducing the attention that the model allocates to invalid information in the context.

[0007] For document-level relation extraction, a multi-class classification task, the key issue is how to obtain more accurate and richer entity representations from documents containing a large number of negative samples for relation classification.

[0008] Processing document-level corpora means that models need to handle complex, large-scale text datasets. This process mainly presents two major challenges. The first is the imbalanced data problem, stemming from the long-tailed distribution of relation categories in real-world corpora. The core challenge lies in mitigating the dominant effect of high-frequency relations on model optimization while enhancing the ability to learn representations of low-frequency relations. The second is the multi-label classification problem, arising from the fact that the same entity pair can simultaneously satisfy multiple non-mutually exclusive relations in complex contexts. The core challenge lies in accurately modeling label coexistence and dynamically adjusting classification thresholds.

[0009] Addressing the inherent data imbalance problem in the document-level relation extraction field presents challenges in three main aspects: First, low-frequency relation embedding vectors struggle to form compact clusters in high-dimensional space due to insufficient training samples, leading to blurred decision boundaries; second, loss gradient updates during training are dominated by the head class, causing the model to tend to assign excessively high prior probabilities to high-frequency relations, thereby reducing prediction accuracy; third, low-frequency relations are often distributed in complex contexts requiring multi-hop reasoning, making it difficult for traditional DocRE models to capture such distributional evidence chains.

[0010] In the course of development in this field, researchers have proposed modeling relation representations, resulting in numerous models addressing the data imbalance problem in the DocRE domain. Wang et al. employed a pre-trained model fine-tuning strategy to predict the existence and fine-grained types of entity relations in stages. Sun et al. proposed the TDGAT framework, using a two-stage dynamic graph attention network to achieve relation detection and classification, and reconstructing the document graph using prior information. Xu et al. modeled entity relations based on attention mechanisms, combining graph attention and multi-head attention networks. Liu et al. proposed a dual-graph guided model, obtaining long-distance dependencies through the document graph, enhancing relation propagation through a constraint graph, and designing an entity fusion module to enrich the representation.

[0011] However, most of these methods still neglect the correlation between relation types, which is crucial for improving the accuracy of triple extraction. Furthermore, static graph attention-based methods fail to separate relation-irrelevant noise during entity feature fusion, making tail relation embeddings susceptible to head category interference.

[0012] Furthermore, the challenges of multi-label classification tasks in this field are mainly reflected in two aspects: First, the traditional global thresholding method cannot adapt to the differences in contextual complexity of different entity pairs; second, a large number of tail relationships often coexist with head relationships as secondary labels, causing the model to tend to predict only the dominant label.

[0013] In the DocRE field, research on multi-label classification tasks mainly revolves around loss functions. Zhou et al. proposed an adaptive thresholding technique to address the multi-label problem, specifically replacing the global threshold used in previous multi-label classification work with a learnable entity dependency threshold. Xu et al. employed a novel multi-label loss function that improves prediction performance by increasing the difference in label confidence scores between each predefined category and the "unrelated" category.

[0014] However, traditional methods, which employ fixed thresholds or coarse-grained dynamic adjustment strategies, cannot accurately quantify the dynamic shift characteristics of confidence distribution in document-level contexts for different relation types. Summary of the Invention

[0015] Existing document-level relation extraction models typically employ graph-based models, which inherently suffer from data imbalance. Current research exhibits several problems: node feature updates are subject to noise interference from irrelevant nodes and edges; excessive negative samples in documents lead to insufficient model learning of true relations; and multi-label classification cannot accurately predict all different relation types. This invention provides a document-level relation extraction method and system based on information gain and prototype contrastive learning. It proposes a graph-based document-level relation extraction model that addresses both the extraction of more accurate node features and the mitigation of data imbalance.

[0016] To achieve the above objectives, the present invention provides the following solution:

[0017] A document-level relation extraction method based on information gain and prototype contrastive learning, the method comprising:

[0018] Step 1: Input the document into the pre-trained model PLM to obtain the token sequence output TK and attention matrix A. At the same time, use the external tool SpaCy to extract all anaphoric words in the document, construct the initial graph structure of the document, and obtain the node embedding representation and adjacency matrix.

[0019] Step 2: Construct the corresponding relation prototype vector representation for each type of relation in the DocRED and Re-DocRED datasets, and obtain the global document representation by averaging TK. Filter A to obtain the attention matrix A' to remove redundant information, and obtain the context representation and head and tail entity embeddings that meet the preset accuracy requirements.

[0020] Step 3: Input the relation prototype vector representation, the document global representation, and the node embedding representation and adjacency matrix obtained through the document graph structure into the graph attention network composed of graph convolutional network and multi-head attention structure for node feature update. The node features are filtered by considering the information gain of nodes and edges, that is, the model no longer pays attention to irrelevant nodes and edges during feature update. Finally, the mention representation after feature update by graph network is obtained. The mention representation is then pooled with logsumexp to obtain the head and tail entity representations.

[0021] Step 4: Introduce a dynamic threshold mechanism to adaptively adjust the classification boundary for each relation type, and fuse the context representation, head and tail entity embeddings, and head and tail entity representations output by the PLM and the graph network to obtain the logical output logits of the current entity pair for all other relations.

[0022] Step 5: Train the model using the contrastive learning loss function and the adaptive threshold loss function to obtain a relational representation that meets the preset accuracy requirements.

[0023] Preferably, the implementation process of step three includes:

[0024] Computation node embedding h i and h j Cosine similarity between sim(h) i ,h j This allows us to obtain a similarity matrix S for a document, and then extract the elements s from the matrix. ij The corresponding similarity is converted into a probability distribution p ij The entropy of node i is then calculated using the following formula:

[0025]

[0026] Where n represents the number of nodes;

[0027] The similarity between pairs of nodes in the graph is calculated by dot product and converted into a corresponding probability distribution. The entropy of each edge is calculated to measure its information content, thus obtaining the set of entropy values ​​for edges between each pair of nodes. Then, ρ is set as a threshold, and edges with entropy values ​​lower than ρ are selected. The adjacency matrix is ​​updated based on the selection, resulting in a filtered adjacency matrix and node embeddings. A multi-head attention mechanism is then used to dynamically learn the graph structure to distinguish the importance of different edges. Specifically, when the PLM dimension is d and the set of edge types in the graph is denoted as U, the adjacency matrix... The calculation formula is:

[0028]

[0029] Among them, TK V Represented as nodes, and M is a learnable parameter matrix and bias. u Let be a binary adjacency matrix of u-type edges;

[0030] A graph convolutional network (GCN) is constructed on a dynamic document graph to model the interactions between nodes. The information transmission process is mediated by the adjacency matrix learned by the model. Guidance is provided to ensure that information is transmitted between nodes along important paths determined by weights learned through learning. and Representing node v respectively i The input and output representations of the m-th layer GCN are as follows: The calculation formula is:

[0031]

[0032] in, This represents the node v learned by the model. i With node v j The adjacency matrix weights between them Represents node v j The input representation of the m-th layer GCN, W m and b m σ is the learnable parameter matrix and bias, and σ is the activation function.

[0033] Preferably, the implementation process of step four includes:

[0034] A feature vector is constructed to generate a dynamic threshold, the feature vector being represented by concatenating header and footer entities E. h E t and the contextual information r captured by the pre-trained model PLM. h,t The threshold is obtained and then fed into a fully connected layer as input to generate a dynamic threshold. It is then normalized using an activation function. The calculation process is as follows:

[0035] dyn_TH=σ([mean(E h )||mean(E t )||mean(r h,t )]·W TH +b TH );

[0036] Where dyn_TH is the dynamic threshold, W TH and b TH It is a learnable parameter matrix and bias, where σ is the activation function;

[0037] The entity representation output by PLM is concatenated with contextual information and graph network output features to obtain a further enhanced entity representation c. h c t Then, for ch Contrast prototype regularization is performed to enhance the distinguishability of entity representations. Specifically, the head entity c is computed. h The cosine similarity between the relation and the prototype feature vector weight matrix P is used to obtain a similarity score. After performing a softmax operation, the weight of each relation category in the dataset is obtained. Finally, the head entity representation is updated by weighted summation of the prototype feature vector weight matrix. The calculation process is as follows:

[0038]

[0039] Where P is the learnable relation prototype matrix, and α is the prototype influence strength;

[0040] After dividing the representation of the head and tail entities into blocks, the similarity of each block is calculated, and the final logits are calculated using a bilinear transformation function. The generated dynamic threshold dyn_TH is then used to dynamically adjust the logits. The calculation process is as follows:

[0041]

[0042] Where h is the result of dynamic adjustment, W r b r These are learnable parameters.

[0043] Preferably, the implementation process of step five includes:

[0044] In the loss calculation stage, an adaptive threshold loss is introduced, treating document-level relation classification as a multi-label classification problem. The loss function for relation prediction is formally expressed as:

[0045]

[0046] Where Pos and Neg represent the values ​​for an entity pair T = (e h ,e t The positive and negative categories of ) are defined by TH, which is a threshold relationship used to distinguish between Pos and Neg;

[0047] In the stage of calculating the contrastive learning loss function, firstly, the average representation of the sequence output is calculated through average pooling, and the labels are converted into class indices. Secondly, the prototype memory is updated based on a gating mechanism, and the relation prototype representation is dynamically updated through the current features. The cosine similarity between the input sequence and each prototype in the prototype memory is calculated, and a temperature factor is introduced to control the similarity scaling to adjust the smoothness of the softmax function. Then, the labels are converted into boolean masks, and pos_mask identifies samples belonging to the positive class. The similarity of positive samples is extracted through the boolean mask pos_mask, and the similarity of negative samples is extracted by taking the inverse mask ~pos_mask. The calculation process is as follows:

[0048]

[0049] S + ={(i,c)|y ic =1};S - ={(i,c)|y ic =0};

[0050]

[0051] Among them, Loss contrast To compare the loss functions, i is the sample index in the batch, c is the class label, k is the index variable, and y is the index variable. ic Let P be the learnable relation prototype matrix, fea be the mean of the current batch features, and s be the label. ic Let S be the cosine similarity, τ be the learnable temperature coefficient, and S be the temperature coefficient. + S is the set of positive samples. - Given a negative sample set; integrate the loss functions: Loss = Loss cls +λ·Loss contrast , where λ is a coefficient that enables the model to effectively distinguish between positive and negative relationships in the input data, thus alleviating the data imbalance problem in multi-label classification scenarios.

[0052] The present invention also provides a document-level relation extraction system based on information gain and prototype contrastive learning. The system is used to implement the aforementioned method and includes: a first construction module, a second construction module, a filtering module, a fusion module, and a training module.

[0053] The first construction module is used to input the document into the pre-trained model PLM to obtain the token sequence output TK and the attention matrix A. At the same time, it uses the external tool SpaCy to extract all anaphoric words in the document, and after constructing the initial graph structure of the document, it obtains the node embedding representation and the adjacency matrix.

[0054] The second construction module is used to construct the corresponding relation prototype vector representation for each type of relation in the DocRED and Re-DocRED datasets, and obtain the global document representation by averaging TK, filter A to obtain the attention matrix A' to remove redundant information, and obtain the context representation and head and tail entity embedding that meet the preset accuracy requirements.

[0055] The filtering module is used to input the relation prototype vector representation, the document global representation, and the node embedding representation and adjacency matrix obtained through the document graph structure into the graph attention network composed of graph convolutional network and multi-head attention structure for node feature update. The node features are filtered by considering the information gain of nodes and edges, that is, the model no longer pays attention to irrelevant nodes and edges when updating features, and finally the mention representation after the graph network feature update is obtained. The mention representation is then pooled by logsumexp to obtain the head and tail entity representations.

[0056] The fusion module is used to introduce a dynamic threshold mechanism to adaptively adjust the classification boundary for each relation type, and to fuse the context representation, head and tail entity embedding, and head and tail entity representation output by the PLM and the graph network to obtain the logical output logits of the current entity pair to all other relations.

[0057] The training module is used to train the model using a contrastive learning loss function and an adaptive threshold loss function to obtain a relational representation that meets preset accuracy requirements.

[0058] Preferably, the filtering module includes: a calculation unit, an update unit, and a transmission unit;

[0059] The computing unit is used to calculate the node embedding h. i and h j Cosine similarity between sim(h) i ,h j This allows us to obtain a similarity matrix S for a document, and then convert the obtained similarity into a probability distribution p. ij The entropy of node i is then calculated using the following formula:

[0060]

[0061] Where n represents the number of nodes;

[0062] The update unit is used to calculate the similarity between pairs of nodes in the graph through dot product and convert it into a corresponding probability distribution. It measures the information content of each edge by calculating its entropy, thus obtaining the set of entropy values ​​for edges between each pair of nodes in the graph. Then, it sets ρ as a threshold, selects edges with entropy values ​​lower than ρ, and updates the adjacency matrix based on the selection. After obtaining the filtered adjacency matrix and node embeddings, a multi-head attention mechanism is used to dynamically learn the graph structure to distinguish the importance of different edges. Specifically, when the PLM dimension is d and the set of edge types in the graph is denoted as U, the adjacency matrix... The calculation formula is:

[0063]

[0064] Among them, TK V Represented as nodes, and M is a learnable parameter matrix and bias. u Let be a binary adjacency matrix of u-type edges;

[0065] The transmission unit is used to construct the interactions between nodes modeled by the Graph Convolutional Network (GCN) on the dynamic document graph. The information transmission process is carried out by the adjacency matrix learned by the model. Guidance is provided to ensure that information is transmitted between nodes along important paths determined by weights learned through learning. and Representing node v respectively i The input and output representations of the m-th layer GCN are as follows: The calculation formula is:

[0066]

[0067] in, This represents the node v learned by the model. i With node v j The adjacency matrix weights between them Represents node v j The input representation of the m-th layer GCN, W m and b m σ is the learnable parameter matrix and bias, and σ is the activation function.

[0068] Preferably, the fusion module includes: a capture unit, a serial unit, and an adjustment unit;

[0069] The capture unit is used to construct a feature vector for generating a dynamic threshold, and the feature vector is represented by splicing together header and footer entities to represent E. h E t and the contextual information r captured by the pre-trained model PLM. h,t The threshold is obtained and then fed into a fully connected layer as input to generate a dynamic threshold. It is then normalized using an activation function. The calculation process is as follows:

[0070] dyn_TH=σ([mean(E h )||mean(E t )||mean(r h,t )]·W TH +b TH );

[0071] Where dyn_TH is the dynamic threshold, W TH and b TH It is a learnable parameter matrix and bias, where σ is the activation function;

[0072] The concatenation unit is used to concatenate the entity representation output by the PLM with context information and the features output by the graph network to obtain a further enhanced entity representation c. h c t Then, for c h Contrast prototype regularization is performed to enhance the distinguishability of entity representations. Specifically, the head entity c is computed. h The cosine similarity between the relation and the prototype feature vector weight matrix P is used to obtain a similarity score. After performing a softmax operation, the weight of each relation category in the dataset is obtained. Finally, the head entity representation is updated by weighted summation of the prototype feature vector weight matrix. The calculation process is as follows:

[0073]

[0074] Where P is the learnable relation prototype matrix, and α is the prototype influence strength;

[0075] The adjustment unit is used to divide the representation of the head and tail entities into blocks, calculate the similarity of each block, calculate the final logits using a bilinear transformation function, and dynamically adjust the logits using a generated dynamic threshold dyn_TH. The calculation process is as follows:

[0076]

[0077] Where f is the result of dynamic adjustment, W r b r These are learnable parameters.

[0078] Preferably, the training module includes: an adaptive threshold loss unit, a contrastive learning loss unit, and an integration unit;

[0079] The adaptive threshold loss unit is used to introduce adaptive threshold loss during the loss calculation stage, that is, to treat document-level relationship classification as a multi-label classification problem, and the loss function of the relationship prediction part is formally expressed as:

[0080]

[0081] Where Pos and Neg represent the values ​​for an entity pair T = (e h ,e t The positive and negative categories of ) are defined by TH, which is a threshold relationship used to distinguish between Pos and Neg;

[0082] The contrastive learning loss unit, during the contrastive learning loss function calculation stage, first calculates the average representation of the sequence output through average pooling and converts the labels into category indices. Second, it updates the prototype memory based on a gating mechanism, dynamically updating the relation prototype representation using the current features. It calculates the cosine similarity between the input sequence and each prototype in the prototype memory, and introduces a temperature factor to control similarity scaling, adjusting the smoothness of the softmax function. Then, it converts the labels into Boolean masks, with `pos_mask` identifying samples belonging to the positive class. Positive sample similarity is extracted using the Boolean mask `pos_mask`, and negative sample similarity is extracted by taking the inverse mask `pos_mask`. The calculation process is as follows:

[0083]

[0084] S + ={(i,c)|y ic =1};S - ={(i,c)|y ic =0};

[0085]

[0086] Among them, Loss contrast To compare the loss functions, i is the sample index in the batch, c is the class label, k is the index variable, and y is the index variable. ic Let P be the learnable relation prototype matrix, fea be the mean of the current batch features, and s be the label. ic Let S be the cosine similarity, τ be the learnable temperature coefficient, and S be the temperature coefficient. + S is the set of positive samples. - The set of negative samples;

[0087] The integration unit is used to integrate the loss functions Loss = Loss cls +λ·Loss contrast , where λ is a coefficient that enables the model to effectively distinguish between positive and negative relationships in the input data, thus alleviating the data imbalance problem in multi-label classification scenarios.

[0088] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0089] For document-level relation extraction tasks, this invention proposes a graph-based document-level relation extraction model that addresses both the need for more accurate node feature extraction and the need to alleviate data imbalance. Existing document-level relation extraction models typically employ graph-based models, which inherently suffer from data imbalance. Current research faces challenges such as noise interference from irrelevant nodes and edges during node feature updates, insufficient model learning of true relations due to excessive negative samples in documents, and the inability of multi-label classification to accurately predict all different relation types. By designing a graph-based modeling method with attention filtering and relation awareness, this invention effectively suppresses semantically irrelevant edge information and guides the model to focus more on semantically similar but less frequently trained long-tail relation types through a dynamic edge selection strategy. This improves relation recognition accuracy while significantly enhancing the model's generalization ability to low-frequency relations. Attached Figure Description

[0090] To more clearly illustrate the technical solution of the present invention, the drawings used in the embodiments are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0091] Figure 1 This is a schematic diagram of a document-level relation extraction method based on information gain and prototype contrast learning according to an embodiment of the present invention.

[0092] Figure 2 This is a schematic diagram comparing the performance of different layers in the DocRED dataset GCN according to an embodiment of the present invention;

[0093] Figure 3 This is a schematic diagram comparing the performance of different layers in the Re-DocRED dataset GCN according to an embodiment of the present invention;

[0094] Figure 4 This is a schematic diagram comparing the attention visualization on the DocRED and Re-DocRED datasets in an embodiment of the present invention, wherein (a) is a schematic diagram of the attention visualization on the DocRED dataset; and (b) is a schematic diagram of the attention visualization on the Re-DocRED dataset. Detailed Implementation

[0095] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0096] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0097] Example 1

[0098] like Figure 1 As shown, this invention provides a document-level relation extraction method based on information gain and prototype contrastive learning, the method comprising:

[0099] Step 1: Input a document containing t tokens into the pre-trained model PLM to obtain the token sequence output TK and attention matrix A: TK, A = PLM([t1, t2, ... t2)). t Simultaneously, all anaphoric words in the document were extracted using the external tool SpaCy. After constructing the initial graph structure of the document, two types of node representations were obtained: a node embedding representation consisting of mention nodes and anaphoric nodes, and an adjacency matrix containing three types of edges. Specifically, this adjacency matrix includes: mention-anaphoric edges, coreference edges, and inter-entity edges.

[0100] Step 2: Construct a corresponding relation prototype vector representation for each type of relation in the DocRED and Re-DocRED datasets, and average it over TK. Obtain the global representation of the document, where TK t Let A represent the embedding of the t-th token in the document. Then, calculate the mean and standard deviation of all elements in A, and filter by setting the difference between the mean and standard deviation as a dynamic threshold to obtain the attention matrix after removing redundant information. Where a ij The elements in the attention matrix are dyn_th, which is a dynamic threshold, thus obtaining a more accurate context representation and head and tail entity embeddings.

[0101] Step 3: Input the above relation prototype vector representation, document global representation, node embedding representation and adjacency matrix obtained through the document graph structure into a graph attention network composed of a graph convolutional network and a multi-head attention structure for node feature update. This process filters by considering the information gain of nodes and edges, that is, when updating features, the model no longer pays attention to irrelevant nodes and edges. Finally, the mention representation after feature update by the graph network is obtained, and the head and tail entity representations are obtained after pooling with logsumexp.

[0102] Step 4: The context representation, head and tail entity embeddings, and head and tail entity representations output by the PLM are fused to obtain the logical output logits of the current entity pair to all other relations. In the above process, a dynamic threshold mechanism is introduced to adaptively adjust the classification boundary for each relation type, enhance the model's ability to distinguish different relation types, and alleviate the problem of the model's weak perception of multi-label entity pairs.

[0103] Step 5: Train the model using a contrastive learning loss function and an adaptive threshold loss function to obtain a more accurate representation of the relationship.

[0104] For existing graph models, the model first encodes the document using PLM to obtain the token embedding TK and attention matrix A. Typically, after constructing the document graph structure and obtaining the corresponding embedding representation and adjacency matrix, the data is directly input into the graph network to update node features. This invention addresses the problems of noise introduced by irrelevant nodes and edges, as well as redundant information in the context, during feature updates. It filters the nodes and edges in the document graph and then combines a multi-head attention mechanism and GCN to learn information from the graph structure, further filtering A. Specifically, A' is obtained by setting a dynamic threshold using the mean and standard deviation, and the context embedding representation is calculated using TK and A'. Furthermore, if a document contains n nodes, the dynamic graph containing n nodes uses an n×n adjacency matrix. To represent, in which elements Within the range [0,1], it represents two connected nodes v. i and v j The importance of edges between nodes. First, calculate the node embedding h. i and h j Cosine similarity between Thus, for a document, a similarity matrix S is obtained, and the elements s in the matrix are... ij The corresponding similarity is converted into a probability distribution p ij = The entropy of node i is then calculated as shown in the following formula:

[0105]

[0106] Next, to filter effective edges in the graph, the similarity between pairs of nodes is calculated using dot product and converted into a corresponding probability distribution. The entropy of each edge is calculated to measure its information content, thus obtaining the set of entropy values ​​for edges between each pair of nodes in the graph. Then, ρ is set as a threshold, and edges with entropy values ​​lower than ρ are selected, and the adjacency matrix is ​​updated based on the selection. After obtaining the filtered adjacency matrix and node embeddings, a multi-head attention mechanism is used to dynamically learn the graph structure to distinguish the importance of different edges. Specifically, when the PLM dimension is d and the set of edge types in the graph is denoted as U, The calculation is shown in the following formula:

[0107]

[0108] TK V Let U be the node representation, and U be the set of edge types. and M is a learnable parameter matrix and bias. u Let d be the binary adjacency matrix of u-type edges, and d be the PLM dimension.

[0109] Then, a graph convolutional network (GCN) is constructed on the dynamic document graph to model the interactions between nodes. The information transmission process is mediated by the adjacency matrix learned by the model. Guidance allows information to be transferred between nodes along important paths determined by learned weights. Let... and Representing node v respectively i The input and output representations of the m-th layer GCN are as follows: The calculation is shown in the following formula:

[0110]

[0111] in, This represents the node v learned by the model. i With node v j The adjacency matrix weights between them Represents node v j The input representation of the m-th layer GCN. W m and b m σ is the learnable parameter matrix and bias, and σ is the ReLU activation function.

[0112] Furthermore, to alleviate the inherent data imbalance problem in the field of document-level relation extraction and to optimize multi-label relation classification, this invention introduces relation prototype contrastive learning and a hierarchical feature representation fusion mechanism. Specifically, firstly, the average representation of each token in TK is taken. This allows for the acquisition of overall semantic features, enabling effective modeling of the document's global semantic information. Secondly, the logsumexp function is used to perform a weighted summation of mention embeddings, thereby generating a more stable and discriminative entity representation. Formally, the representation of entity e is EN. e The calculation is shown in the following formula:

[0113]

[0114] Where C represents the number of mentions contained in entity e.

[0115] Then, a prototype vector is created for each relation category in the dataset. After introducing relation-level and document-level feature information into the graph network, updated relation-level feature representations `rel_graph_fea` and `doc_graph_fea` are obtained, and then fused with entity features. Specifically, for each graph structure node feature `graph_fea`, it is superimposed with the global document features. Then, the entity representation fused with the global features is concatenated with the relation features. This process is illustrated in the following formula:

[0116] graph_fea=graph_fea+doc_graph_fea[0];

[0117] final_graph_fea=[graph_fea||rel_graph_fea];

[0118] Similarly, to obtain the entity embedding representation, it is necessary to traverse each entity mention and use the logsumexp function to perform a weighted summation of the mention embeddings, ultimately obtaining the head and tail entity representation EN. h With EN t .

[0119] In the relation classification stage, to enable the model to better distinguish different relation types and simultaneously consider multi-label classification, this invention introduces a dynamic threshold mechanism. Specifically, it first constructs a header and footer entity representation E by splicing together the header and footer entity representations. h E t and the contextual information r captured by the pre-trained model (PLM). h,t A feature vector is obtained and used to generate a dynamic threshold. This feature vector is then fed into a fully connected layer as input to generate the dynamic threshold, which is then normalized using an activation function. The calculation process is shown in the following formula:

[0120] dyn_TH=σ([mean(E h )||mean(E t )||mean(r h,t )]·W TH +b TH );

[0121] Where dyn_TH is the dynamic threshold, W TH and b TH It is a learnable parameter matrix and bias, where σ is the Sigmoid activation function.

[0122] To preserve the contextual information captured by the pre-trained language model, the entity representation output by the PLM is concatenated with the contextual information and the features output by the graph network to obtain a further enhanced entity representation c. h c tThe above process is shown in the following formula:

[0123] c h =tanh(W h [E h ||r h,t ||EN h ])+b;

[0124] c t =tanh(W t [E t ||r h,t ||EN t ])+b;

[0125] Among them, W h ∈R d×d W t ∈R d×d All of these are learnable parameters.

[0126] Then, for c h Contrast prototype regularization is performed to enhance the distinguishability of entity representations. Specifically, the head entity c is computed. h The cosine similarity between the relation and the prototype feature vector weight matrix P is used to obtain a similarity score. After performing a softmax operation, the weights of each relation category in the dataset are obtained. Finally, the head entity representation is updated by weighted summation of the prototype feature vector weight matrix. This method helps align the head entity representation with the corresponding category prototype. The above process is shown in the following formula:

[0127]

[0128] Where P is the learnable relation prototype matrix, and α is the prototype influence strength.

[0129] Finally, to adaptively adjust the predicted values ​​according to different relation types, the representations of the head and tail entities are divided into blocks, their similarity is calculated, and the final logits are calculated using a bilinear transformation function. The generated dynamic threshold dyn_TH is then used to dynamically adjust the logits. The calculation process is shown in the following formula:

[0130]

[0131] Where f is the result of dynamic adjustment, W r b r These are learnable parameters.

[0132] In the loss calculation stage, this invention introduces the adaptive threshold loss previously proposed, that is, treating document-level relation classification as a multi-label classification problem, and the loss function for the relation prediction part is Loss. cls This can be formally expressed as shown in the following formula:

[0133]

[0134] Where Pos and Neg represent the values ​​for an entity pair T = (e h ,e t The positive and negative categories are defined by TH, which is a threshold relationship used to distinguish between Pos and Neg.

[0135] The second part calculates the contrastive learning loss function. First, average pooling is used to calculate the average representation of the sequence output, and the labels are converted into class indices. Next, a gating mechanism is used to update the prototype memory, dynamically updating the relation prototype representation based on the current features. The cosine similarity between the input sequence and each prototype in the prototype memory is calculated, and a temperature factor is introduced to control the similarity scaling, adjusting the smoothness of the softmax function. Then, the labels are converted into Boolean masks, where `pos_mask` identifies which samples belong to the positive class (i.e., label 1). Positive sample similarity is extracted using the Boolean mask `pos_mask`, and negative sample similarity is extracted by taking the inverse mask `pos_mask`. The above calculation process is shown in the following formula:

[0136]

[0137] S + ={(i,c)|y ic =1};S - ={(i,c)|y ic =0};

[0138]

[0139] Among them, Loss contrast To compare the loss functions, i is the sample index in the batch, c is the class label, k is the index variable, and y is the index variable. ic For labels. P is the learnable relation prototype matrix, fea is the mean of the current batch features, and s ic Let S be the cosine similarity, and τ be the learnable temperature coefficient. + S is the set of positive samples. - This is the set of negative samples.

[0140] Finally, the loss functions are integrated: Loss = Loss cls +λ·Loss contrast , where λ is a coefficient that enables the model to effectively distinguish between positive and negative relationships in the input data, thus alleviating the data imbalance problem in multi-label classification scenarios.

[0141] This invention uses the English datasets DocRED and Re-DocRED, downloaded from GitHub, as datasets for document-level relation extraction. DocRED is a standard dataset for the DocRE task proposed by Yao et al., containing over 10,000 long documents from Wikipedia, with 97 relation types (including NA relations). The task requires the model to extract cross-sentence entity relations from long texts, facing problems such as complex semantic reasoning and long-distance dependencies. The dataset's annotations include multiple entity categories, such as people, places, and organizations, as well as various relation labels, such as "located in" and "leader." Re-DocRED is an extended dataset of DocRED proposed by Tan et al., which has been re-annotated, increasing the number of relational facts in DocRED and adding core referential resolution information. Re-DocRED is a re-annotated dataset of DocRED, adding missing false negative relation triples. Since the original test set of DocRED is not publicly available, they divided 1,000 evaluation documents into a development set and a test set. The specific data of DocRED and Re-DocRED are shown in Table 1.

[0142] Table 1. Statistics of the DocRED and Re-DocRED datasets

[0143]

[0144] This invention was developed using an A40 graphics card with 48GB of video memory. The operating system was Linux release 8.2.2004, the development language was Python 3.10, and the deep learning framework was PyTorch 2.6.

[0145] The specific parameter settings are shown in Table 2.

[0146] Table 2 Parameter Settings

[0147]

[0148]

[0149] This invention uses the PRF (Precision, Recall, F1) metric to analyze and evaluate experimental results, where P represents precision, R represents recall, and F is the F1 score. The F1 score is calculated by macro-averaging P and R, reflecting the overall performance of the model, and is expressed as F1 = 2 * P * R / (P + R). Furthermore, since overlap in relational facts between the training and development / test sets is unavoidable, as many common relational facts may be shared across different documents, the DocRE task also introduces an F1 score excluding shared relational facts between the training and development / test sets, denoted as the Ign F1 score. This evaluation metric makes the assessment more convincing because the model must truly understand and learn unseen relations, rather than simply memorizing known relations.

[0150] The model AFPCL of this invention was compared with several DocRE models in recent years, and the results are shown in Tables 3 and 4. Table 3 shows the results obtained after training on the DocRED dataset using Bert-base-cased and Roberta-large pre-trained models. Overall, the model outperforms all baseline models. When using Bert-base-cased as the PLM, the F1 and Ign F1 scores reached 61.94% and 60.23%, respectively; when using Roberta-large as the PLM, the F1 and Ign F1 scores reached 64.25% and 62.36%, respectively. Table 4 shows the results obtained after training on the Re-DocRED dataset using the Roberta-large pre-trained model. Overall, the model outperforms all baseline models, with F1 and Ign F1 scores reaching 78.69% and 78.13%, respectively.

[0151] Table 3. Comparison Results of DocRED Experiments

[0152]

[0153]

[0154] Table 4. Comparison Results of Re-DocRED Experiments

[0155]

[0156] The model designed in this invention, based on information gain and prototype contrastive learning, optimizes current methods from two perspectives: 1) Noise reduction (NR), which filters edges in the graph by calculating information gain and sets a dynamic threshold to filter the attention matrix; 2) Mitigation of data imbalance (MDI), which constructs a relation prototype vector for each relation and calculates its similarity to samples, and improves the accuracy of multi-label classification through a dynamic threshold mechanism. The above two parts were removed from the model and trained on DocRED and Re-DocRED using different pre-trained models, respectively, and the experimental results are shown in Tables 5 and 6.

[0157] Table 5. Results of DocRED ablation experiments

[0158]

[0159] Table 6. Results of Re-DocRED ablation experiments

[0160]

[0161] All experimental results in Tables 5 and 6 were obtained under the same seed settings. The F1 and Ign F1 scores of the model in this invention, based on the DocRED dataset and the BERT pre-trained model, were 61.94% and 88.9%, respectively. Removing the denoising method decreased the F1 score by 0.5%; removing the data imbalance mitigation method decreased the F1 score by 0.54%; and removing both decreased the F1 score by 0.84%. This demonstrates that both denoising and data imbalance mitigation methods can improve upon the shortcomings of current methods from different perspectives.

[0162] The purpose of this experiment is to observe the impact of hyperparameters on the results. This model is a graph-based method, and its core technology involves updating node features in the graph structure through graph convolution combined with a multi-head attention mechanism to obtain entity representations. Therefore, further experiments were conducted on the DocRED validation set and the Re-DocRED test set to verify the impact of the number of GCN layers on model performance. Experimental results are as follows: Figure 2 , Figure 3 As shown.

[0163] from Figure 2 , Figure 3As can be observed, the AFPCL model performs best when the number of GCN layers is 2. This is because the number of GCN layers determines the depth of node feature propagation: in document-level relation extraction tasks, entities need sufficient neighborhood information aggregation to capture long-distance dependencies while maintaining feature discriminability. When the number of layers is too small, the model can only capture the first-order neighborhood features of entities, making it difficult to model the complex semantic interactions across sentences in the document, resulting in insufficient perception of the global topological structure; while too many layers (≥3 layers) can lead to oversmoothing, causing different node feature representations to converge, weakening the discriminative ability of classification boundaries. In addition, deep GCNs introduce more noise propagation, especially when processing document-level graph structures, which may redundantly fuse irrelevant node features. Experiments show that the 2-layer GCN structure achieves the optimal balance between node feature expression capabilities and can more effectively utilize the relational information and global document information captured by this invention.

[0164] To verify the effectiveness of the attention filtering mechanism, the attention distribution of 50 tokens from randomly selected documents was visualized. Attention scores were uniformly mapped to the interval [0, 0.1] for comparison. Figure 4 As shown.

[0165] This invention compares the filtered attention distribution generated based on a dynamic thresholding strategy with the original attention distribution directly output by a pre-trained language model (PLM). Visualization results show that attention filtering under the dynamic thresholding mechanism can effectively remove semantically irrelevant tokens while retaining key information, allowing the model to focus more on the effective context required for relational reasoning, thereby enhancing entity semantic representation and improving relation extraction performance.

[0166] Example 2

[0167] The present invention also provides a document-level relation extraction system based on information gain and prototype contrastive learning. The system is used to implement the method described in Embodiment 1. The system includes: a first construction module, a second construction module, a filtering module, a fusion module, and a training module.

[0168] The first construction module is used to input the document into the pre-trained model PLM to obtain the token sequence output TK and the attention matrix A. At the same time, it uses the external tool SpaCy to extract all anaphoric words in the document, and after constructing the initial graph structure of the document, it obtains the node embedding representation and the adjacency matrix.

[0169] The second construction module is used to construct the corresponding relation prototype vector representation for each type of relation in the DocRED and Re-DocRED datasets, and obtain the global document representation by averaging TK, filter A to obtain the attention matrix A' to remove redundant information, and obtain the context representation and head and tail entity embedding that meet the preset accuracy requirements.

[0170] The filtering module is used to input the relation prototype vector representation, the document global representation, and the node embedding representation and adjacency matrix obtained through the document graph structure into the graph attention network composed of graph convolutional network and multi-head attention structure for node feature update. The node features are filtered by considering the information gain of nodes and edges, that is, the model no longer pays attention to irrelevant nodes and edges during feature update, and finally the mention representation after feature update by graph network is obtained. The mention representation is then pooled by logsumexp to obtain the head and tail entity representations.

[0171] The fusion module is used to introduce a dynamic threshold mechanism to adaptively adjust the classification boundary for each relation type. It fuses the context representation, head and tail entity embeddings, and head and tail entity representations output by the PLM and the graph network to obtain the logical output logits of the current entity pair to all other relations.

[0172] The training module is used to train the model using a contrastive learning loss function and an adaptive threshold loss function to obtain a relational representation that meets the preset accuracy requirements.

[0173] In this embodiment, the filtering module includes: a calculation unit, an update unit, and a transmission unit;

[0174] Computational unit, used to compute node embedding h i and h j Cosine similarity between sim(h) i ,h j This allows us to obtain a similarity matrix S for a document, and then convert the obtained similarity into a probability distribution p. ij The entropy of node i is then calculated using the following formula:

[0175]

[0176] Where n represents the number of nodes;

[0177] The update unit calculates the similarity between pairs of nodes in the graph using dot product and converts it into a corresponding probability distribution. It measures the information content of each edge by calculating its entropy, thus obtaining the set of entropy values ​​for each pair of nodes in the graph. Then, it sets ρ as a threshold, selecting edges with entropy values ​​lower than ρ, and updates the adjacency matrix based on the selection. After obtaining the filtered adjacency matrix and node embeddings, a multi-head attention mechanism is used to dynamically learn the graph structure to distinguish the importance of different edges. Specifically, when the PLM dimension is d and the set of edge types in the graph is denoted as U, the adjacency matrix... The calculation formula is:

[0178]

[0179] Among them, TK VRepresented as nodes, and M is a learnable parameter matrix and bias. u Let be a binary adjacency matrix of u-type edges;

[0180] Transmission units are used to construct the interactions between nodes in a graph convolutional network (GCN) on a dynamic document graph. The information transmission process is based on the adjacency matrix learned by the model. Guidance is provided to ensure that information is transmitted between nodes along important paths determined by weights learned through learning. and Representing node v respectively i The input and output representations of the m-th layer GCN are as follows: The calculation formula is:

[0181]

[0182] in, This represents the node v learned by the model. i With node v j The adjacency matrix weights between them Represents node v j The input representation of the m-th layer GCN, W m and b m σ is the learnable parameter matrix and bias, and σ is the ReLU activation function.

[0183] In this embodiment, the fusion module includes: a capture unit, a serial unit, and an adjustment unit;

[0184] A capture unit is used to construct a feature vector for generating a dynamic threshold, wherein the feature vector is represented by splicing together header and tail entities E. h E t and the contextual information r captured by the pre-trained model PLM. h,t The threshold is obtained and then fed into a fully connected layer as input to generate a dynamic threshold. It is then normalized using an activation function. The calculation process is as follows:

[0185] dyn_TH=σ([mean(E h )||mean(E t )||mean(r h,t )]·W TH +b TH );

[0186] Where dyn_TH is the dynamic threshold, W TH and b TH It is a learnable parameter matrix and bias, where σ is the Sigmoid activation function;

[0187] The concatenation unit is used to concatenate the entity representation output by the PLM with context information and the features output by the graph network to obtain a further enhanced entity representation c. h c t Then, for c h Contrast prototype regularization is performed to enhance the distinguishability of entity representations. Specifically, the head entity c is computed. h The cosine similarity between the relation and the prototype feature vector weight matrix P is used to obtain a similarity score. After performing a softmax operation, the weight of each relation category in the dataset is obtained. Finally, the head entity representation is updated by weighted summation of the prototype feature vector weight matrix. The calculation process is as follows:

[0188]

[0189] Where P is the learnable relation prototype matrix, and α is the prototype influence strength;

[0190] The adjustment unit is used to divide the representation of the head and tail entities into blocks, calculate the similarity of each block, calculate the final logits using a bilinear transformation function, and dynamically adjust the logits using a generated dynamic threshold dyn_TH. The calculation process is as follows:

[0191]

[0192] Where f is the result of dynamic adjustment, W r b r These are learnable parameters.

[0193] In this embodiment, the training module includes: an adaptive threshold loss unit, a contrastive learning loss unit, and an integration unit;

[0194] The adaptive threshold loss unit is used to introduce adaptive threshold loss during the loss calculation stage. This treats document-level relation classification as a multi-label classification problem, and the loss function for relation prediction is formally expressed as:

[0195]

[0196] Where Pos and Neg represent the values ​​for an entity pair T = (e h ,e t The positive and negative categories of ) are defined by TH, which is a threshold relationship used to distinguish between Pos and Neg;

[0197] The contrastive learning loss unit, used in the contrastive learning loss function calculation stage, first calculates the average representation of the sequence output through average pooling and converts the labels into class indices. Next, it updates the prototype memory based on a gating mechanism, dynamically updating the relation prototype representation using the current features. It calculates the cosine similarity between the input sequence and each prototype in the prototype memory, and introduces a temperature factor to control similarity scaling, adjusting the smoothness of the softmax function. Then, it converts the labels into a Boolean mask, with `pos_mask` identifying samples belonging to the positive class. Positive sample similarity is extracted using the Boolean mask `pos_mask`, and negative sample similarity is extracted by taking the inverse mask `pos_mask`. The calculation process is as follows:

[0198]

[0199] S + ={(i,c)|y ic =1};S - ={(i,c)|y ic =0};

[0200]

[0201] Among them, Loss contrast To compare the loss functions, i is the sample index in the batch, c is the class label, k is the index variable, and y is the index variable. ic Let P be the learnable relation prototype matrix, fea be the mean of the current batch features, and s be the label. ic Let S be the cosine similarity, τ be the learnable temperature coefficient, and S be the temperature coefficient. + S is the set of positive samples. - The negative sample set; the integration unit, used to integrate the loss functions Loss = Loss cls +λ·Loss contrast , where λ is a coefficient that enables the model to effectively distinguish between positive and negative relationships in the input data, thus alleviating the data imbalance problem in multi-label classification scenarios.

[0202] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Various modifications and improvements made to the technical solutions of the present invention by those skilled in the art without departing from the spirit of the present invention should fall within the protection scope defined by the claims of the present invention.

Claims

1. A document-level relation extraction method based on information gain and prototype contrastive learning, characterized in that, The method includes: Step 1: Input the document into the pre-trained model PLM to obtain the token sequence output TK and attention matrix A. At the same time, use the external tool SpaCy to extract all anaphoric words in the document, construct the initial graph structure of the document, and obtain the node embedding representation and adjacency matrix. Step 2: Construct the corresponding relation prototype vector representation for each type of relation in the DocRED and Re-DocRED datasets, and obtain the global document representation by averaging TK. Filter A to obtain the attention matrix A' to remove redundant information, and obtain the context representation and head and tail entity embeddings that meet the preset accuracy requirements. Step 3: Input the relation prototype vector representation, the document global representation, and the node embedding representation and adjacency matrix obtained through the document graph structure into the graph attention network composed of graph convolutional network and multi-head attention structure for node feature update. The node features are filtered by considering the information gain of nodes and edges, that is, the model no longer pays attention to irrelevant nodes and edges during feature update. Finally, the mention representation after feature update by graph network is obtained. The mention representation is then pooled with logsumexp to obtain the head and tail entity representations. Step 4: Introduce a dynamic threshold mechanism to adaptively adjust the classification boundary for each relation type, and fuse the context representation, head and tail entity embeddings, and head and tail entity representations output by the PLM and the graph network to obtain the logical output logits of the current entity pair for all other relations. Step 5: Train the model using the contrastive learning loss function and the adaptive threshold loss function to obtain a relational representation that meets the preset accuracy requirements; Step 3 includes the following steps: Cosine similarity between embedding of a node and embedding of another node And Cosine similarity between embedding of a node and embedding of another node So that for a document we get a similarity matrix Convert the similarity in the matrix To a probability distribution Then compute the entropy of the node The formula is: ； wherein denotes the number of nodes; The similarity between each pair of nodes in the graph is calculated by point multiplication, and is converted into a corresponding probability distribution. The information amount of each edge is measured by calculating the entropy of each edge, thereby obtaining a set of entropy values of the edges between each pair of nodes in the graph. Then, edges with entropy values lower than are selected as the threshold, and the adjacency matrix is updated according to the selection, thereby obtaining the adjacency matrix after filtering the edges and the node embedding. After that, the multi-head attention mechanism is used to dynamically learn the graph structure to distinguish the importance of different edges. Specifically, when the PLM dimension is d and the edge type set in the graph is denoted as U, the calculation formula of the adjacency matrix is as follows: ； in, Represented as nodes, and For learnable parameter matrices and biases, for A binary adjacency matrix of edge types; A graph convolutional network (GCN) is constructed on a dynamic document graph to model the interactions between nodes. The information transmission process is mediated by the adjacency matrix learned by the model. Guidance is provided to ensure that information is transmitted between nodes along important paths determined by weights learned through learning. and Representing nodes respectively In the The input and output representations of the layer GCN, The calculation formula is: ； in, The nodes are those learned by the model. With nodes The adjacency matrix weights between them Represents a node In the The input representation of the layer GCN, and It consists of a learnable parameter matrix and biases. It is an activation function.

2. The method according to claim 1, characterized in that, Step four includes the following steps: A feature vector is constructed to generate a dynamic threshold, and the feature vector is represented by splicing header and footer entities. , and contextual information captured by the pre-trained model PLM. The threshold is obtained and then fed into a fully connected layer as input to generate a dynamic threshold. It is then normalized using an activation function. The calculation process is as follows: ； in, For dynamic thresholds, and It consists of a learnable parameter matrix and biases. For activation functions; The entity representation output by PLM is concatenated with contextual information and graph network output features to obtain a further enhanced entity representation. , Then, for Contrast prototype regularization is performed to enhance the distinguishability of entity representations. Specifically, the head entity is computed. With the prototype eigenvector weight matrix The cosine similarity between the relationships is used to obtain a similarity score. After performing a softmax operation, the weight of each relation category in the dataset is obtained. Finally, the head entity representation is updated by weighted summation of the prototype feature vector weight matrix. The calculation process is as follows: ； in, The learnable relation prototype matrix, The strength of the influence of the prototype; After dividing the representation of the head and tail entities into blocks, the similarity of each block is calculated, and the final logits are calculated using a bilinear transformation function. The generated dynamic threshold is then used. The logits are dynamically adjusted, and the calculation process is as follows: ； in, This is the result of dynamic adjustment. These are learnable parameters.

3. The method according to claim 1, characterized in that, Step five includes the following steps: In the loss calculation stage, an adaptive threshold loss is introduced, treating document-level relation classification as a multi-label classification problem. The loss function for relation prediction is formally expressed as: ； in, and e.g., each represents a pair of entities. Positive and negative categories, It is a threshold relationship used to distinguish and e.g. In the stage of calculating the contrastive learning loss function, firstly, the average representation of the sequence output is calculated through average pooling, and the labels are converted into category indices. Secondly, the prototype memory is updated based on a gating mechanism, and the relation prototype representation is dynamically updated through the current features. The cosine similarity between the input sequence and each prototype in the prototype memory is calculated, and a temperature factor is introduced to control the similarity scaling to adjust the smoothness of the softmax function. Finally, the labels are converted into boolean type masks. Samples belonging to the positive class are identified using a Boolean mask. Extract the similarity of positive samples and invert the mask. The process for extracting negative sample similarity is as follows: ； ； ； in, To compare loss functions, It is the sample index in the batch. It's a category label. For index variables, For tags, The learnable relation prototype matrix, This represents the average feature value of the current batch. For cosine similarity, For learnable temperature coefficient, For the positive sample set, The set of negative samples; Integrate the loss function ,in, The coefficients enable the model to effectively distinguish between positive and negative relationships in the input data, alleviating the data imbalance problem in multi-label classification scenarios.

4. A document-level relation extraction system based on information gain and prototype contrastive learning, the system being used to implement the method described in any one of claims 1-3, characterized in that, The system includes: a first construction module, a second construction module, a filtering module, a fusion module, and a training module; The first construction module is used to input the document into the pre-trained model PLM to obtain the token sequence output TK and the attention matrix A. At the same time, it uses the external tool SpaCy to extract all anaphoric words in the document, and after constructing the initial graph structure of the document, it obtains the node embedding representation and the adjacency matrix. The second construction module is used to construct the corresponding relation prototype vector representation for each type of relation in the DocRED and Re-DocRED datasets, and obtain the global document representation by averaging TK, filter A to obtain the attention matrix A' to remove redundant information, and obtain the context representation and head and tail entity embedding that meet the preset accuracy requirements. The filtering module is used to input the relation prototype vector representation, the document global representation, and the node embedding representation and adjacency matrix obtained through the document graph structure into the graph attention network composed of graph convolutional network and multi-head attention structure for node feature update. The node features are filtered by considering the information gain of nodes and edges, that is, the model no longer pays attention to irrelevant nodes and edges when updating features, and finally the mention representation after the graph network feature update is obtained. The mention representation is then pooled by logsumexp to obtain the head and tail entity representations. The fusion module is used to introduce a dynamic threshold mechanism to adaptively adjust the classification boundary for each relation type, and to fuse the context representation, head and tail entity embedding, and head and tail entity representation output by the PLM and the graph network to obtain the logical output logits of the current entity pair to all other relations. The training module is used to train the model using a contrastive learning loss function and an adaptive threshold loss function to obtain a relational representation that meets the preset accuracy requirements.

5. The system according to claim 4, characterized in that, The filtering module includes: a calculation unit, an update unit, and a transmission unit; The computing unit is used to compute node embedding. and Cosine similarity between Thus, a similarity matrix is ​​obtained for a document. The obtained similarity is converted into a probability distribution. Post-compute node The entropy, the formula is: ； in, Indicates the number of nodes; The update unit is used to calculate the similarity between pairs of nodes in the graph through dot product and convert it into a corresponding probability distribution. It measures the information content of each edge by calculating the entropy of each edge, thereby obtaining the set of entropy values ​​for the edges between each pair of nodes in the graph, and then sets... As a threshold, select values ​​with entropy values ​​lower than [a certain threshold]. The edges are selected, and the adjacency matrix is ​​updated according to the selection. After obtaining the adjacency matrix after filtering edges and node embeddings, a multi-head attention mechanism is used to dynamically learn the graph structure to distinguish the importance of different edges. Specifically, when the PLM dimension is d and the set of edge types in the graph is denoted as U, the adjacency matrix... The calculation formula is: ； in, Represented as nodes, and For learnable parameter matrices and biases, for A binary adjacency matrix of edge types; The transmission unit is used to construct the interactions between nodes modeled by the Graph Convolutional Network (GCN) on the dynamic document graph. The information transmission process is carried out by the adjacency matrix learned by the model. Guidance is provided to ensure that information is transmitted between nodes along important paths determined by weights learned through learning. and Representing nodes respectively In the The input and output representations of the layer GCN, The calculation formula is: ； in, The nodes are those learned by the model. With nodes The adjacency matrix weights between them Represents a node In the The input representation of the layer GCN, and It consists of a learnable parameter matrix and biases. It is an activation function.

6. The system according to claim 4, characterized in that, The fusion module includes: a capture unit, a serial unit, and an adjustment unit; The capture unit is used to construct a feature vector for generating a dynamic threshold, and the feature vector is represented by splicing header and footer entities. , and contextual information captured by the pre-trained model PLM. The threshold is obtained and then fed into a fully connected layer as input to generate a dynamic threshold. It is then normalized using an activation function. The calculation process is as follows: ； in, For dynamic thresholds, and It consists of a learnable parameter matrix and biases. For activation functions; The concatenation unit is used to concatenate the entity representation output by the PLM with the context information and the features output by the graph network to obtain a further enhanced entity representation. , Then, for Contrast prototype regularization is performed to enhance the distinguishability of entity representations. Specifically, the head entity is computed. With the prototype eigenvector weight matrix The cosine similarity between the relationships is used to obtain a similarity score. After performing a softmax operation, the weight of each relation category in the dataset is obtained. Finally, the head entity representation is updated by weighted summation of the prototype feature vector weight matrix. The calculation process is as follows: ； in, The learnable relation prototype matrix, The strength of the influence of the prototype; The adjustment unit is used to divide the representations of the head and tail entities into blocks, calculate the similarity of each block, calculate the final logits using a bilinear transformation function, and use the generated dynamic threshold. The logits are dynamically adjusted, and the calculation process is as follows: ； in, This is the result of dynamic adjustment. These are learnable parameters.

7. The system according to claim 4, characterized in that, The training module includes: an adaptive threshold loss unit, a contrastive learning loss unit, and an integration unit; The adaptive threshold loss unit is used to introduce adaptive threshold loss during the loss calculation stage, that is, to treat document-level relationship classification as a multi-label classification problem, and the loss function of the relationship prediction part is formally expressed as: ； in, and e.g., each represents a pair of entities. Positive and negative categories, It is a threshold relationship used to distinguish and e.g. The contrastive learning loss unit, during the contrastive learning loss function calculation stage, first calculates the average representation of the sequence output through average pooling and converts the labels into category indices. Second, it updates the prototype memory based on a gating mechanism, dynamically updating the relation prototype representation using the current features. It calculates the cosine similarity between the input sequence and each prototype in the prototype memory, and introduces a temperature factor to control similarity scaling, adjusting the smoothness of the softmax function. Finally, it converts the labels into Boolean type masks. Samples belonging to the positive class are identified using a Boolean mask. Extract the similarity of positive samples and invert the mask. The process for extracting negative sample similarity is as follows: ； ； ； in, To compare loss functions, It is the sample index in the batch. It's a category label. For index variables, For tags, The learnable relation prototype matrix, This represents the average feature value of the current batch. For cosine similarity, For learnable temperature coefficient, For the positive sample set, The set of negative samples; The integration unit is used to integrate the loss functions. ,in, The coefficients enable the model to effectively distinguish between positive and negative relationships in the input data, alleviating the data imbalance problem in multi-label classification scenarios.

Images