Document-level relation extraction method and system based on information gain and prototype comparative learning

Abstract

The invention belongs to the field of natural language processing in computer intelligent information processing, and discloses a document level relation extraction method and system based on information gain and prototype comparative learning. The invention provides a document-level relation extraction model based on a graph structure, which considers two aspects of extracting more accurate node features and relieving data imbalance. The problem that an existing document-level relation extraction model generally adopts a graph-based model and faces inherent data imbalance is solved. At present, the problems that noise interference is caused by irrelevant nodes and edges in the node feature updating process, the learning ability of a model to a real relation is insufficient due to too many negative samples in a document, and all different relation types cannot be accurately predicted through multi-label classification exist in research.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of natural language processing in computer intelligent information processing, and particularly relates to a document-level relation extraction method and system based on information gain and prototype contrast learning. BACKGROUND

[0002] With the booming development of large-scale text data such as social media, news reports and academic literature, document-level relation extraction (DocRE) has become a research task that attracts much attention in the field of natural language processing (NLP). By identifying the relationships between entities in a document, a more comprehensive understanding of complex events, social interactions and opinion expression can be achieved, thereby promoting the technological progress of information extraction and knowledge management, and providing important support for fields such as intelligent search, public opinion analysis and knowledge graph construction.

[0003] In the field of document-level relation extraction, there are increasingly complex scenarios and demands when dealing with large-scale text data. In order to better understand the associations between entities in a document, the existing models can be divided into two main lines: Transformer-based models and graph-based models.

[0004] Transformer-based models only accept the word sequence of the document as input and use Transformer to implicitly capture the long context dependency relationship between entities. Specifically, it uses the Transformer architecture and attention mechanism to capture the relationships between entities in the text, which can effectively capture long-distance dependencies, thereby modeling the complex interactions between entities within the entire document. By modeling the global context, Transformer-based models can better understand the context and associations of entities in the document, thereby improving the accuracy and efficiency of the document-level relation extraction task. These models usually perform representation learning through pre-trained language models (such as BERT, RoBERTa, etc.) and combine task-specific fine-tuning methods to achieve document-level relation extraction. The advantage is that it can handle longer texts and capture global relationships, so it has important application potential in the task of document-level relation extraction. Tang et al. proposed a hierarchical reasoning network that obtains entity-level reasoning information through translation constraints and bilinear transformations, models the reasoning relationship between entities and sentences, and obtains document-level reasoning information through hierarchical aggregation. Ye et al. proposed the CorefBERT model, which improves the understanding of text coherence by explicitly modeling the reference relationship, and performs outstandingly in the reference reasoning task. Zhou et al. proposed adaptive threshold and local context pooling techniques, the former learns entity-related thresholds to optimize multi-label classification, and the latter uses local context information to improve relation extraction accuracy, which significantly improves the performance of the model.

[0005] The graph-based model represents the document as a graph, where the nodes represent the entities in the document, and the edges represent the relationships between entities. The graph-based model uses graph neural networks for information propagation and aggregation, captures the relevance between entities through multi-head attention mechanism, iteratively updates the node representation, and implements global reasoning, so as to better understand the complex relationships between entities in the document and improve the accuracy and robustness of relationship extraction. Zeng et al. proposed a graph aggregation inference network (GAIN), which constructs a mention-level graph (MG) and an entity-level graph (EG), and combines a path reasoning mechanism to realize document-level relationship extraction. Wei et al. use language tools to construct co-reference edges and other graph structures to capture cross-sentence and intra-sentence dependencies. Li et al. proposed a graph-enhanced dual attention network (GEDA), which interacts with a heterogeneous graph convolutional network through S2R / R2S attention mechanisms, and designs an evidence-based regularizer to improve performance.

[0006] When the graph-based model obtains the current entity feature, it needs to update the current node feature representation iteratively according to the nodes and edges in the graph structure through GCN. However, since all surrounding information will be included in the current node encoding, it will inevitably be affected by irrelevant information from irrelevant nodes and edges. In addition, since there is a multi-sentence nature when performing cross-sentence reasoning on entity pairs, the context feature of the current entity pair will inevitably be disturbed by irrelevant information factors in the sentence. To solve the above problems, we propose a node and edge screening module based on the information gain mechanism and an attention filtering module based on the context feature. In this way, the model can pay more attention to the current entity feature and the context feature that is more effective for judging the current entity pair relationship. More specifically, to solve the first problem, in order to make the model obtain more real and effective entity features, the similarity of node embedding in the graph structure is calculated and converted into a probability distribution, and then the information entropy of each edge is calculated. Through the adjacency matrix, the node pair without edge is filtered out, and the entropy value is limited between [0, 1], and finally the entropy value of each edge in the graph is output to measure the certainty of the relationship between nodes; to solve the second problem, in order to effectively filter out the noise information in the context feature of the current entity pair, the combination of two common statistics in statistical data, mean and standard deviation, is used as a dynamic threshold to effectively filter the attention matrix output by the pre-training model, thereby reducing the attention of the model to the invalid information in the context.

[0007] For the multi-classification task of document-level relationship extraction, the key problem is how to obtain more correct and rich entity representations from documents containing a large number of negative samples for relationship classification.

[0008] Processing document-level corpus means that the model needs to process complex large-scale text corpus, and there are mainly two problems in this process. The first is the problem of data imbalance, which is caused by the long-tail distribution characteristics of relationship categories in real corpus. The core challenge is how to alleviate the dominant effect of high-frequency relationships on model optimization while enhancing the representation learning ability of low-frequency relationships. The second is the problem of multi-label classification, which is caused by the fact that the same entity pair can satisfy multiple non-mutually exclusive relationships in complex context. The core challenge is how to accurately model label coexistence and dynamically adjust the classification threshold.

[0009] The challenges caused by the inherent data imbalance problem in the field of document-level relationship extraction mainly lie in three aspects: first, the embedding vector of low-frequency relationship is difficult to form a compact cluster in high-dimensional space due to insufficient training samples, resulting in blurred decision boundary; second, the loss gradient update is dominated by the head category during training, leading the model to assign too high prior probability to high-frequency relationships, thus reducing the prediction accuracy; third, low-frequency relationships are often distributed in complex context that requires multi-hop reasoning, and traditional DocRE models are difficult to capture such distribution evidence chains.

[0010] During the development of this field, researchers have proposed to consider modeling relationship representation, and many models have been developed to solve the data imbalance problem in the field of DocRE. Wang et al. adopted the pre-training model fine-tuning strategy to predict the existence and fine-grained type of entity relationships in stages. Sun et al. proposed the TDGAT framework to realize relationship detection and classification through a two-stage dynamic graph attention network, and reconstructed the document graph using prior information. Xu et al. based on attention mechanism, combined graph attention and multi-head attention network to model entity relationships. Liu et al. proposed a dual-graph guided model to obtain long-range dependencies through a document graph, constrain the graph to enhance relationship propagation, and designed an entity fusion module to enrich the representation.

[0011] However, most of these methods still ignore the relevance between relationship types, which is crucial for improving the accuracy of triple extraction. In addition, the method based on static graph attention fails to separate relationship-independent noise in the entity feature fusion process, making the tail relationship embedding susceptible to interference from head categories.

[0012] In addition, the multi-label classification task in this field mainly has two difficulties: first, the traditional global threshold method cannot adapt to the differences in context complexity of different entity pairs; second, a large number of tail relationships often coexist with head relationships as secondary labels, leading the model to tend to predict only the dominant label.

[0013] The research in the field of DocRE mainly focuses on the loss function for the multi-label classification task: Zhou et al. proposed an adaptive threshold technique to solve the multi-label problem, that is, replacing the global threshold used in previous work for multi-label classification with a learnable entity-dependent threshold. Xu et al. adopted a novel multi-label loss function that improves prediction performance by increasing the difference in label confidence scores between each predefined class and the "no relation" class.

[0014] However, the traditional method adopts a fixed threshold or a coarse-grained dynamic adjustment strategy, and cannot accurately quantify the dynamic shift characteristics of the confidence distribution of different relationship types in the document-level context. SUMMARY

[0015] For the existing document-level relationship extraction model, a graph-based model is usually adopted, and the inherent data imbalance problem is faced. In the current research, the node feature updating process is disturbed by noise from irrelevant nodes and edges, there are too many negative samples in the document, which leads to insufficient learning ability of the model for real relationships, and the multi-label classification cannot accurately predict all different relationship types. The present application provides a document-level relationship extraction method and system based on information gain and prototype contrast learning, and proposes a graph structure-based document-level relationship extraction model considering the extraction of more accurate node features and the alleviation of data imbalance from two aspects.

[0016] To achieve the above object, the present application provides the following scheme:

[0017] A document-level relationship extraction method based on information gain and prototype contrast learning, the method comprising:

[0018] Step one: input the document into the pre-training model PLM to obtain the token sequence output TK and the attention matrix A, and at the same time extract all the anaphora in the document through the external tool SpaCy, obtain the node embedding representation and the adjacency matrix after constructing the initial graph structure of the document;

[0019] Step two: construct the corresponding relationship prototype vector representation for each type of relationship in the DocRED and Re-DocRED data sets, and obtain the document global representation by averaging TK, and obtain the attention matrix A' that removes redundant information by filtering A, and obtain the context representation and head and tail entity embedding that meet the preset accuracy requirement;

[0020] Step three: input the relationship prototype vector representation, document global representation, and node embedding representation and adjacency matrix obtained through the document graph structure into the graph attention network composed of graph convolution network and multi-head attention structure for node feature updating, filter the node features by considering the information gain of nodes and edges, that is, make the model no longer pay attention to irrelevant nodes and edges during feature updating, and finally obtain the mention representation after graph network feature updating, and obtain the head and tail entity representation after logsumexp pooling of the mention representation;

[0021] Step four: introduce a dynamic threshold mechanism to adaptively adjust the classification boundary for each relationship type, and fuse the context representation output by the PLM, the head and tail entity embedding, and the head and tail entity representation output by the graph network to obtain the logical output logits of the current entity pair for all other relationships;

[0022] Step five: train the model with the contrastive learning loss function and the adaptive threshold loss function to obtain the relationship representation that meets the preset accuracy requirement.

[0023] Preferably, the implementation process of step three includes:

[0024] Calculate the cosine similarity sim(h i ,h j ) between the node embedding h i and h j , so as to obtain a similarity matrix S for a document, and convert the similarity of the corresponding element s ij in the matrix into a probability distribution p ij , then calculate the entropy of node i, the formula is:

[0025]

[0026] Wherein, n represents the number of nodes;

[0027] The similarity between two nodes in the graph is calculated by dot product, and the corresponding probability distribution is converted, the information amount of the edge is measured by calculating the entropy of each edge, so as to obtain the entropy value set of the edge between each pair of nodes in the graph, then set ρ as the threshold, select the edge with entropy value lower than ρ, and update the adjacency matrix according to the selection, obtain the adjacency matrix after filtering the edge and the node embedding, and use the multi-head attention mechanism to dynamically learn the graph structure to distinguish the importance of different edges; Specifically, when the PLM dimension is d, the edge type set in the graph is denoted as U, the calculation formula of the adjacency matrix is:

[0028]

[0029] Wherein, TK V is the node representation, and ​are learnable parameter matrix and bias, M u is a binary adjacency matrix of u-type edges;

[0030] The graph convolutional network (GCN) is constructed on the dynamic document graph to model the interaction between nodes, and the information transmission process is determined by the adjacency matrix learned by the model Guided, so that information is transmitted between nodes along important paths determined by learned weights, set and respectively represent the node v i The input and output representations of the mth layer GCN are represented as The calculation formula is:

[0031]

[0032] wherein, represents the adjacency matrix weight between nodes v i and v j learned by the model, represents the input representation of node v j in the mth layer GCN, W m and b m are learnable parameter matrix and bias, and σ is the activation function.

[0033] Preferably, the implementation process of step four comprises:

[0034] A feature vector is constructed for generating a dynamic threshold, which is obtained by concatenating the head entity representation E h , the tail entity representation E t , and the context information r h,t captured by the pre-trained model PLM, and then inputting it into a fully connected layer to generate a dynamic threshold, and normalizing it through an activation function, the calculation process is:

[0035] dyn_TH=σ([mean(E h )||mean(E t )||mean(r h,t )]·W TH +b TH );

[0036] Wherein, dyn_TH is the dynamic threshold, W TH and b TH are learnable parameter matrix and bias, and σ is the activation function;

[0037] The entity representation output by the PLM is concatenated with the context information and the output feature of the graph network to obtain a further enhanced entity representation c h , c t , and then ch Contrastive prototype regularization is performed to enhance the distinguishability of entity representations. Specifically, the head entity c h The cosine similarity between the prototype feature vector weight matrix P and the head entity c

[0038]

[0039] Where P is a learnable relation prototype matrix, and a is the prototype influence strength.

[0040] After the representations of the head and tail entities are divided into blocks, the similarity of each block is calculated, and the final logits are calculated through a bilinear transformation function. The generated dynamic threshold dyn_TH is used to dynamically adjust the logits, and the calculation process is as follows:

[0041]

[0042] Where h is the result of dynamic adjustment, W r , and b r are learnable parameters.

[0043] Preferably, the implementation process of step five includes:

[0044] In the loss calculation stage, an adaptive threshold loss is introduced, i.e. the document-level relation classification is regarded as a multi-label classification problem, and the loss function of the relation prediction part is formalized as:

[0045]

[0046] Where Pos and Neg represent the positive and negative classes for an entity pair T = (e h , e t ), and TH is a threshold relationship used to distinguish Pos and Neg.

[0047] In the calculation of the contrastive learning loss function, first, the average representation of the sequence output is calculated through average pooling, and the label is converted into a class index. Second, the prototype memory bank is updated based on the gating mechanism, the relation prototype representation is dynamically updated through the current feature, the cosine similarity between the input sequence and each prototype in the prototype memory bank is calculated, and a temperature factor is introduced to control the similarity scaling, which is used to adjust the smoothness of the softmax function. Then the label is converted into a Boolean type mask, pos_mask identifies the samples belonging to the positive class, the positive sample similarity is extracted through the Boolean mask pos_mask, and the negative sample similarity is extracted by taking the inverse mask ~ pos_mask. The calculation process is as follows:

[0048]

[0049] S + ={(i,c)|y ic =1};S - ={(i,c)|y ic =0};

[0050]

[0051] wherein, Loss contrast is a contrastive loss function, i is a sample index in a batch, c is a class label, k is an index variable, y ic is a label, P is a learnable relation prototype matrix, fea is a current batch feature mean, s ic is a cosine similarity, tau is a learnable temperature coefficient, S + is a positive sample set, S - is a negative sample set; the loss function is integrated Loss=Loss cls +lambda.Loss contrast , wherein, lambda is a coefficient, so that the model effectively distinguishes positive and negative relationships in the input data, and alleviates the data imbalance problem in the multi-label classification scenario.

[0052] The application also provides a document-level relation extraction system based on information gain and prototype contrastive learning, which is used to implement the foregoing method, and comprises a first construction module, a second construction module, a screening module, a fusion module and a training module.

[0053] The first construction module is used for inputting a document into a pre-training model PLM to obtain a token sequence output TK and an attention matrix A, and simultaneously extracting all anaphoric words in the document through an external tool SpaCy, obtaining node embedding representation and an adjacency matrix after constructing an initial graph structure of the document.

[0054] The second construction module is used for constructing a corresponding relation prototype vector representation for each type of relation in the DocRED and Re-DocRED data sets, obtaining a document global representation by averaging TK, filtering A to obtain an attention matrix A' that removes redundant information, and obtaining a context representation and head and tail entity embeddings that meet a preset accuracy requirement.

[0055] The screening module is configured to input the relational prototype vector representation, the document global representation, and the node embedding representation and the adjacency matrix obtained through the document graph structure into a graph attention network composed of a graph convolution network and a multi-head attention structure for node feature updating, screen the node features by considering the information gain of the nodes and edges, i.e., make the model no longer focus on irrelevant nodes and edges during feature updating, and finally obtain mention representations after graph network feature updating.

[0056] The fusion module is configured to introduce a dynamic threshold mechanism to adaptively adjust the classification boundary for each relationship type, fuse the context representation output by the PLM, the head and tail entity embedding, and the head and tail entity representation output by the graph network, and obtain the logical output logits of the current entity pair for all other relationships.

[0057] The training module is configured to train the model by using a contrastive learning loss function and an adaptive threshold loss function, and obtain a relationship representation satisfying a preset accuracy requirement.

[0058] Preferably, the screening module comprises a calculation unit, an updating unit, and a transmission unit.

[0059] The calculation unit is configured to calculate the cosine similarity sim(h i ,h j ) between the node embedding h i and h j , so as to obtain a similarity matrix S for a document, convert the obtained similarity into a probability distribution p ij , and then calculate the entropy of the node i, with the formula being:

[0060]

[0061] wherein n represents the number of nodes.

[0062] The updating unit is configured to calculate the similarity between two nodes in the graph by dot multiplication, convert the similarity into a corresponding probability distribution, measure the information amount of the edge by calculating the entropy of each edge, so as to obtain a set of entropy values of the edge between each pair of nodes in the graph, set ρ as a threshold, select the edges with an entropy value lower than ρ, and update the adjacency matrix according to the selection, so as to obtain the adjacency matrix after filtering the edges and the node embedding, and use a multi-head attention mechanism to dynamically learn the graph structure to distinguish the importance of different edges. Specifically, when the PLM dimension is d and the set of edge types in the graph is denoted as U, the calculation formula of the adjacency matrix is:

[0063]

[0064] wherein TK V is the node representation. and are learnable parameter matrices and bias, M u is a binary adjacency matrix of u-type edges;

[0065] The transmission unit is configured to construct a graph convolution network (GCN) on the dynamic document graph to model the interaction between nodes, and an information transmission process is determined by an adjacency matrix learned by the model Guides so that information is transmitted between nodes along important paths determined by learned weights, and the information transmission process is determined by an adjacency matrix learned by the model and respectively represent nodes v i The input and output representations of the mth layer GCN are respectively represented by The calculation formula of is as follows:

[0066]

[0067] wherein, represents the adjacency matrix weight between nodes v i and v j . represents the input representation of node v j in the mth layer GCN, W m and b m are learnable parameter matrices and bias, and σ is an activation function.

[0068] Preferably, the fusion module comprises a capturing unit, a concatenation unit and an adjustment unit.

[0069] The capturing unit is configured to construct a feature vector for generating a dynamic threshold value, and the feature vector is obtained by splicing the head entity representation E h , the tail entity representation E t and the context information r h,t captured by the pre-trained model (PLM), and then inputting the feature vector into a full connection layer to generate the dynamic threshold value and performing normalization through an activation function, and the calculation process is as follows:

[0070] dyn_TH=σ([mean(E h )||mean(E t )||mean(r h,t )]·W TH +b TH );

[0071] wherein, dyn_TH is a dynamic threshold value, W TH and b TH are learnable parameter matrices and bias, and σ is an activation function.

[0072] The serial unit is configured to concatenate the entity representation output by the PLM with the context information and the graph network output features to obtain a further enhanced entity representation c h t Then, the c h is subjected to contrastive prototype regularization to enhance the distinguishability of the entity representation. Specifically, cosine similarity between the head entity c h and a prototype feature vector weight matrix P is calculated to obtain a similarity score. After a softmax operation, the weight of each relationship category in the data set is obtained. Finally, the head entity representation is updated by weighted summation of the prototype feature vector weight matrix. The calculation process is as follows:

[0073]

[0074] wherein P is a learnable relationship prototype matrix, and a is a prototype influence strength.

[0075] The adjustment unit is configured to calculate the similarity of each block after dividing the representation of the head and tail entities, and calculate the final logits by a bilinear transformation function. The generated dynamic threshold dyn_TH is used to dynamically adjust the logits. The calculation process is as follows:

[0076]

[0077] wherein f is the result of dynamic adjustment, W r , b r are learnable parameters.

[0078] Preferably, the training module comprises an adaptive threshold loss unit, a contrastive learning loss unit, and an integration unit.

[0079] The adaptive threshold loss unit is configured to introduce adaptive threshold loss in the loss calculation stage, i.e., to regard the document-level relationship classification as a multi-label classification problem. The loss function of the relationship prediction part is formalized as follows:

[0080]

[0081] wherein Pos and Neg represent the positive class and the negative class for an entity pair T=(e h ,e t ), respectively, and TH is a threshold relationship used to distinguish Pos and Neg.

[0082] ​The contrastive learning loss unit is configured to, in a stage of calculating a contrastive learning loss function, first calculate an average representation of the sequence output through average pooling, and convert the label into a category index, second update a prototype memory bank based on a gating mechanism, dynamically update a relationship prototype representation through a current feature, calculate a cosine similarity between the input sequence and each prototype in the prototype memory bank, and introduce a temperature factor to control similarity scaling, which is used to adjust the smoothness of a softmax function, then convert the label into a Boolean type mask, a pos_mask identifies samples belonging to a positive class, extract positive sample similarities through the Boolean mask pos_mask, extract negative sample similarities through an inverse mask ~pos_mask, and the calculation process is as follows:

[0083]

[0084] S + ={(i,c)|y ic =1};S - ={(i,c)|y ic =0};

[0085]

[0086] wherein, Loss contrast is a contrastive loss function, i is a sample index in a batch, c is a category label, k is an index variable, y ic is a label, P is a learnable relationship prototype matrix, fea is a current batch feature mean, s ic is a cosine similarity, tau is a learnable temperature coefficient, S + is a positive sample set, and S - is a negative sample set.

[0087] The integration unit is configured to integrate the loss functions as follows: Loss = Loss cls + lambda * Loss contrast wherein, lambda is a coefficient, so that the model can effectively distinguish positive and negative relationships in the input data, and alleviate the data imbalance problem in the multi-label classification scenario.

[0088] Compared with the prior art, the present application has the following advantages:

[0089] For the document-level relation extraction task, the application proposes a graph structure-based document-level relation extraction model considering the extraction of more accurate node features and the alleviation of data imbalance. The existing document-level relation extraction model usually adopts a graph-based model and faces the inherent data imbalance problem. In the current research, the node feature updating process is disturbed by noise from irrelevant nodes and edges, there are too many negative samples in the document, which leads to insufficient learning ability of the model for real relations, and the multi-label classification cannot accurately predict all different relation types. By designing a graph structure modeling method with attention filtering mechanism and relation perception, the semantic irrelevant edge information can be effectively suppressed, and the dynamic edge screening strategy is combined to guide the model to pay more attention to the long-tail relation types with similar semantics but low training frequency. Thus, the relation recognition accuracy is improved, and the generalization ability of the model for low-frequency relations is effectively enhanced. BRIEF DESCRIPTION OF DRAWINGS

[0090] In order to more clearly illustrate the technical solutions of the present application, the following briefly introduces the drawings needed to be used in the embodiments of the present application. Obviously, the drawings described in the following embodiments are only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor.

[0091] Figure 1 For the embodiment of the present application, a document-level relation extraction method based on information gain and prototype contrast learning is shown in the flowchart.

[0092] Figure 2 For the embodiment of the present application, the performance comparison diagram of GCN with different layers of DocRED dataset is shown.

[0093] Figure 3 For the embodiment of the present application, the performance comparison diagram of GCN with different layers of Re-DocRED dataset is shown.

[0094] Figure 4 For the embodiment of the present application, the attention visualization comparison diagram of DocRED and Re-DocRED datasets is shown, wherein (a) is the attention visualization diagram of DocRED dataset; (b) is the attention visualization diagram of Re-DocRED dataset. DETAILED DESCRIPTION

[0095] The technical solutions in the embodiments of the present application will be described in detail below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0096] In order to make the above objectives, characteristics and advantages of the present application more apparent, comprehensible and easier to understand, the present application will be described in further detail below with reference to the accompanying drawings and specific embodiments.

[0097] Embodiment one

[0098] As Figure 1 shown, the present application provides a document-level relation extraction method based on information gain and prototype contrastive learning, which comprises:

[0099] Step one: input a document containing t tokens into a pre-training model PLM to obtain token sequence output TK and attention matrix A: TK, A = PLM ([t1, t2,... t t ]). At the same time, all anaphors in the document are extracted through an external tool SpaCy, and after constructing the initial graph structure of the document, two types of node representations are obtained: node embedding representation composed of mention nodes and anaphor nodes, and adjacency matrix containing three types of edges. The adjacency matrix specifically includes: mention-anaphor edge, co-reference edge and entity-to-entity edge;

[0100] Step two: construct the corresponding relation prototype vector representation for each type of relation in the DocRED and Re-DocRED data sets, and obtain the document global representation by taking the average of TK , wherein TK t represents the tth token embedding representation in the document. Then the mean and standard deviation of all elements in A are calculated, and the difference between the mean and the standard deviation is set as a dynamic threshold to filter the attention matrix , wherein a ij is an element in the attention matrix, and dyn_th is a dynamic threshold, so as to obtain more accurate context representation and head-tail entity embedding;

[0101] Step three: input the above relation prototype vector representation, document global representation, node embedding representation and adjacency matrix obtained through the document graph structure into the graph attention network composed of graph convolution network and multi-head attention structure to update the node features. This process filters by considering the information gain of nodes and edges, that is, the model no longer pays attention to irrelevant nodes and edges during feature updating, and finally obtains the mention representation after graph network feature updating, and obtains the head-tail entity representation after logsumexp pooling;

[0102] Step 4: The context representation, head and tail entity embeddings, and head and tail entity representations output by the PLM are fused to obtain the logical output logits of the current entity pair to all other relations. In the above process, a dynamic threshold mechanism is introduced to adaptively adjust the classification boundary for each relation type, enhance the model's ability to distinguish different relation types, and alleviate the problem of the model's weak perception of multi-label entity pairs.

[0103] Step 5: Train the model using a contrastive learning loss function and an adaptive threshold loss function to obtain a more accurate representation of the relationship.

[0104] For existing graph models, the model first encodes the document using PLM to obtain the token embedding TK and attention matrix A. Typically, after constructing the document graph structure and obtaining the corresponding embedding representation and adjacency matrix, the data is directly input into the graph network to update node features. This invention addresses the problems of noise introduced by irrelevant nodes and edges, as well as redundant information in the context, during feature updates. It filters the nodes and edges in the document graph and then combines a multi-head attention mechanism and GCN to learn information from the graph structure, further filtering A. Specifically, A' is obtained by setting a dynamic threshold using the mean and standard deviation, and the context embedding representation is calculated using TK and A'. Furthermore, if a document contains n nodes, the dynamic graph containing n nodes uses an n×n adjacency matrix. To represent, in which elements Within the range [0,1], it represents two connected nodes v. i and v j The importance of edges between nodes. First, calculate the node embedding h. i and h j Cosine similarity between Thus, for a document, a similarity matrix S is obtained, and the elements s in the matrix are... ij The corresponding similarity is converted into a probability distribution p ij = The entropy of node i is then calculated as shown in the following formula:

[0105]

[0106] Next, to filter effective edges in the graph, the similarity between pairs of nodes is calculated using dot product and converted into a corresponding probability distribution. The entropy of each edge is calculated to measure its information content, thus obtaining the set of entropy values ​​for edges between each pair of nodes in the graph. Then, ρ is set as a threshold, and edges with entropy values ​​lower than ρ are selected, and the adjacency matrix is ​​updated based on the selection. After obtaining the filtered adjacency matrix and node embeddings, a multi-head attention mechanism is used to dynamically learn the graph structure to distinguish the importance of different edges. Specifically, when the PLM dimension is d and the set of edge types in the graph is denoted as U, is calculated as shown in the following formula:

[0107]

[0108] where TK V is the node representation, U is the set of edge types, and are learnable parameter matrices and bias, M u is the binary adjacency matrix of u-type edges, and d is the dimension of the PLM.

[0109] Then, a graph convolutional network (GCN) is constructed on the dynamic document graph to model the interaction between nodes, and the information transmission process is guided by the adjacency matrix learned by the model , so that information can be transmitted between nodes along important paths determined by the learned weights. Let and represent the input and output representations of node v i at the m-th layer GCN, respectively. is calculated as shown in the following formula:

[0110]

[0111] where, represents the adjacency matrix weight between node v i and node v j , represents the input representation of node v j at the m-th layer GCN. W m and b m are learnable parameter matrices and bias, and σ is the Relu activation function.

[0112] In addition, in order to alleviate the inherent data imbalance problem in the field of document-level relation extraction, and optimize the multi-label classification of relations, the application introduces relation prototype contrast learning and hierarchical feature representation fusion mechanism. Specifically, first, take the average of each token representation in TK to obtain the overall semantic feature, effectively modeling the global semantic information of the document. Second, use the logsumexp function to weight the sum of the mention embedding, thereby generating a more stable and discriminative entity representation. Formally, the representation EN e of entity e is calculated as shown in the following formula:

[0113]

[0114] where C is the number of mentions contained by entity e.

[0115] Then, the prototype vector of each relation category in the data set is established, the relation level and the document level feature information are introduced into the graph network, and the updated relation level feature representation rel_graph_fea and the document level feature representation doc_graph_fea are obtained, and are fused with the entity feature. The specific fusion method is that the document global feature is superimposed for each graph structure node feature graph_fea, and then the entity representation fused with the global feature is spliced with the relation feature. The process is shown in the following formula:

[0116] graph_fea = graph_fea + doc_graph_fea[0];

[0117] final_graph_fea = [graph_fea||rel_graph_fea];

[0118] Similarly, in order to obtain the entity embedding representation, each entity mention needs to be traversed, and the logsumexp function is used to weight and sum the mention embedding, and finally the head and tail entity representations EN h and EN t are obtained.

[0119] In the relation classification stage, in order to make the model better distinguish different relation types, and at the same time consider multi-label classification, the dynamic threshold mechanism is introduced. Specifically, first, a feature vector is obtained by splicing the head and tail entity representations E h , E t , and the context information r h,t captured by the pre-trained model (PLM), which is used to generate a dynamic threshold, and then the above feature vector is input into the full connection layer to generate a dynamic threshold, and is normalized through an activation function. The calculation process is shown in the following formula:

[0120] dyn_TH = σ ([mean(E h )||mean(E t )||mean(r h,t )]·W TH +b TH );

[0121] Where, dyn_TH is the dynamic threshold, W TH and b TH are learnable parameter matrix and bias, and sigma is the Sigmoid activation function.

[0122] In order to retain the context information captured by the pre-trained language model, the entity representation output by the PLM is concatenated with the context information and the graph network output feature to obtain the further enhanced entity representation c h , c tThe above process is shown in the following formula:

[0123] c h =tanh(W h [E h ||r h,t ||EN h ])+b;

[0124] c t =tanh(W t [E t ||r h,t ||EN t ])+b;

[0125] Wherein, W h ∈R d×d , W t ∈R d×d , are all learnable parameters.

[0126] Then, the c h is compared with the prototype regularization, and the distinguishability of the entity representation is enhanced. Specifically, the cosine similarity between the head entity c h and the prototype feature vector weight matrix P is calculated to obtain the similarity score. After the softmax operation, the weight of each relationship category in the data set is obtained. Finally, the head entity representation is updated by weighted sum of the prototype feature vector weight matrix. This way helps to align the head entity representation with the corresponding category prototype. The above process is shown in the following formula:

[0127]

[0128] Wherein, P is a learnable relationship prototype matrix, and alpha is the prototype influence strength.

[0129] Finally, in order to adaptively adjust the prediction value according to different relationship types, the representations of the head and tail entities are divided into blocks, their similarities are calculated, and the final logits are calculated through a bilinear transformation function. The generated dynamic threshold dyn_TH is used to dynamically adjust the logits, and the calculation process is shown in the following formula:

[0130]

[0131] Wherein, f is the result of dynamic adjustment, W r , b r are learnable parameters.

[0132] In the loss calculation stage, the adaptive threshold loss proposed by the previous person is introduced, that is, the document-level relationship classification is regarded as a multi-label classification problem, and the loss function Loss cls of the relationship prediction part can be formally expressed as shown in the following formula:

[0133]

[0134] where Pos and Neg represent the positive and negative classes for an entity pair T = (e h ,e t ) respectively. TH is a threshold relation to distinguish Pos and Neg.

[0135] The second part is to calculate the contrastive learning loss function. First, the average representation of the sequence output is calculated by average pooling, and the label is converted into a class index. Second, the prototype memory bank is updated based on the gating mechanism, which dynamically updates the relational prototype representation through the current features. The cosine similarity between the input sequence and each prototype in the prototype memory bank is calculated, and a temperature factor is introduced to control the similarity scaling, which is used to adjust the smoothness of the softmax function. Then the label is converted into a boolean type mask, pos_mask identifies which samples belong to the positive class (i.e. the label is 1), and the positive sample similarity is extracted through the boolean mask pos_mask, and the inverse mask ~ pos_mask is used to extract the negative sample similarity. The above calculation process is shown in the following formula:

[0136]

[0137] S + ={(i,c)|y ic =1}; S - ={(i,c)|y ic =0};

[0138]

[0139] where Loss contrast is the contrastive loss function, i is the sample index in the batch, c is the class label, k is the index variable, y ic is the label. P is the learnable relational prototype matrix, fea is the current batch feature mean, s ic is the cosine similarity, τ is the learnable temperature coefficient. S + is the positive sample set, S - is the negative sample set.

[0140] Finally, the loss function is integrated Loss = Loss cls + λ·Loss contrast , where λ is a coefficient, so that the model can effectively distinguish the positive and negative relationships in the input data, and alleviate the data imbalance problem in the multi-label classification scenario.

[0141] The application adopts English dataset DocRED and Re-DocRED as the dataset of document-level relation extraction downloaded from github. DocRED is a standard dataset proposed by Yao et al. for the DocRE task, containing more than 10,000 long documents from Wikipedia, containing 97 relationship types (including NA relationship). The task requires the model to extract cross-sentence entity relationships from long text, facing problems such as complex semantic reasoning and long-distance dependence. The annotation of the dataset includes multiple entity categories such as persons, places and organizations, and multiple relationship labels such as "located in", "led" and the like. Re-DocRED is a dataset extended by Tan et al. from DocRED, which is re-labeled and expands the number of relationship facts in DocRED, and adds core core reference resolution information. Re-DocRED is a re-labeled dataset of DocRED, adding missing false negative relationship triples. Since the original test set of DocRED is not public, they divided 1,000 evaluation documents into development and test sets. The specific data of DocRED and Re-DocRED are shown in Table 1.

[0142] Table 1 DocRED and Re-DocRED dataset statistics

[0143]

[0144] The application is carried out in the hardware environment of a graphics card A40 with 48G of video memory. The system is Linux release 8.2.2004, the development language is Python 3.10, and the deep learning framework is Pytorch 2.6.

[0145] The specific parameter settings are shown in Table 2.

[0146] Table 2 Parameter settings

[0147]

[0148]

[0149] The application uses evaluation index PRF value to analyze and evaluate the experimental results, wherein P is precision, R is recall, and F is F1 value. F1 value is the harmonic mean of P and R by macro average, which is used to reflect the comprehensive performance of the model, and the formula is F1=2*P*R / (P+R). In addition, since the relationship between the training set and the development / test set is inevitable, because many common relationship facts may be shared in different documents. Therefore, the DocRE task also introduces the F1 value after excluding the relationship facts shared by the training set and the development / test set, which is denoted as Ign F1 value. This evaluation index makes the evaluation more convincing, because the model must truly understand and learn the unseen relationship, rather than simply remember the known relationship.

[0150] The application model AFPCL is compared with several DocRE models in this field in recent years, and the results are shown in Tables 3 and 4. Table 3 shows the results obtained after training using Bert-base-cased and Roberta-large pre-training models on the DocRED dataset. From the final overall results, the effect is better than all baseline models, and the F1 value and Ign F1 value are 61.94% and 60.23% respectively when using Bert-base-cased as PLM; the F1 value and Ign F1 value are 64.25% and 62.36% respectively when using Roberta-large as PLM. Table 4 shows the results obtained after training using Roberta-large pre-training model on Re-DocRED dataset. From the final overall results, the effect is better than all baseline models, and the F1 value and Ign F1 value are 78.69% and 78.13% respectively.

[0151] Table 3 Comparison results of DocRED experiment

[0152]

[0153]

[0154] Table 4 Comparison results of Re-DocRED experiment

[0155]

[0156] The model based on information gain and prototype contrast learning designed by the application optimizes the current method from two angles: 1) the noise reduction angle (NR), which screens the edges in the graph by calculating the information gain and sets a dynamic threshold to filter the attention matrix; 2) the data imbalance relief angle (MDI), which constructs a relationship prototype vector for each relationship and calculates the similarity between it and the sample, and improves the accuracy of multi-label classification through a dynamic threshold mechanism. The above two parts are removed from the model respectively for training using different pre-training models on DocRED and Re-DocRED, respectively, to obtain the following experimental results, as shown in Tables 5 and 6.

[0157] Table 5 DocRED ablation experiment results

[0158]

[0159] Table 6 Re-DocRED ablation experiment results

[0160]

[0161] All experimental results in Tables 5 and 6 are completed under the same seed setting. The F1 value and Ign F1 value of the model of the application are 61.94% and 88.9% respectively under the setting of the DocRED dataset and the BERT pre-training model. When the noise reduction method is removed, the F1 value decreases by 0.5%; when the data imbalance relief method is removed, the F1 value decreases by 0.54%; and when both are removed, the F1 value decreases by 0.84%. It can be seen that the noise reduction and data imbalance relief methods can improve the defects in the current method from different angles.

[0162] The purpose of this experiment is to observe the influence of hyperparameters on the results. The model is a graph-based method, and the core technology is to update the node features in the graph structure by combining graph convolution with a multi-head attention mechanism to obtain entity representations. Therefore, further experiments were conducted on the DocRED validation set and the Re-DocRED test set to verify the influence of the number of GCN layers on the performance of the model. The experimental results are shown in Tables 7 and 8. Figure 2 、 Figure 3

[0163] From Figure 2 、 Figure 3 ​It can be observed that the AFPCL model performs best when the number of GCN layers is 2. This is because the number of GCN layers determines the depth of node feature propagation: in the document-level relation extraction task, both sufficient neighborhood information aggregation to capture long-range dependencies and the discriminability of features are needed between entities. When the number of layers is too small, the model can only capture the first-order neighborhood features of entities, and it is difficult to model the complex semantic interaction across sentences in the document, resulting in insufficient perception of the global topology structure; when the number of layers is too large (≥ 3 layers), over-smoothing phenomenon occurs, which causes different node feature representations to converge, and weakens the discriminability of the classification boundary. In addition, deep GCN will introduce more noise propagation, especially when processing document-level graph structures, which may redundantly integrate irrelevant node features. Experiments show that the 2-layer GCN structure is optimal in terms of node feature expression ability, and can more effectively utilize the correlation information between relationships and the global information of the document captured by the application.

[0164] To verify the effectiveness of the attention filtering mechanism, the attention distribution of 50 tokens randomly selected from a document is visualized, and the attention scores are uniformly mapped to the interval [0, 0.1] for comparison, as shown in Figure 4 .

[0165] The application compares the filtered attention distribution generated based on the dynamic threshold strategy with the original attention distribution directly output by the pre-trained language model (PLM). The visualization results show that the attention filtering under the dynamic threshold mechanism can effectively remove semantically irrelevant tokens while retaining key information, enabling the model to focus more on the effective context required for relation reasoning, thereby enhancing entity semantic expression and improving relation extraction performance.

[0166] Embodiment Two

[0167] The application also provides a document-level relation extraction system based on information gain and prototype contrast learning, which is used to implement the method of embodiment one, and the system comprises a first construction module, a second construction module, a screening module, a fusion module and a training module.

[0168] The first construction module is used to input the document into the pre-trained model PLM to obtain the token sequence output TK and the attention matrix A, and simultaneously extract all anaphors in the document through an external tool SpaCy to obtain the node embedding representation and the adjacency matrix after constructing the initial graph structure of the document.

[0169] The second construction module is used to construct the corresponding relation prototype vector representation for each type of relation in the DocRED and Re-DocRED datasets, and obtain the document global representation by averaging TK, and obtain the attention matrix A' after filtering A to remove redundant information, and obtain the context representation and head and tail entity embeddings that meet the preset accuracy requirements.

[0170] The filtering module is used to input the relation prototype vector representation, the document global representation, and the node embedding representation and adjacency matrix obtained through the document graph structure into the graph attention network composed of graph convolutional network and multi-head attention structure for node feature update. The node features are filtered by considering the information gain of nodes and edges, that is, the model no longer pays attention to irrelevant nodes and edges during feature update, and finally the mention representation after feature update by graph network is obtained. The mention representation is then pooled by logsumexp to obtain the head and tail entity representations.

[0171] The fusion module is used to introduce a dynamic threshold mechanism to adaptively adjust the classification boundary for each relation type. It fuses the context representation, head and tail entity embeddings, and head and tail entity representations output by the PLM and the graph network to obtain the logical output logits of the current entity pair to all other relations.

[0172] The training module is used to train the model using a contrastive learning loss function and an adaptive threshold loss function to obtain a relational representation that meets the preset accuracy requirements.

[0173] In this embodiment, the filtering module includes: a calculation unit, an update unit, and a transmission unit;

[0174] Computational unit, used to compute node embedding h i and h j Cosine similarity between sim(h) i ,h j This allows us to obtain a similarity matrix S for a document, and then convert the obtained similarity into a probability distribution p. ij The entropy of node i is then calculated using the following formula:

[0175]

[0176] Where n represents the number of nodes;

[0177] The update unit calculates the similarity between pairs of nodes in the graph using dot product and converts it into a corresponding probability distribution. It measures the information content of each edge by calculating its entropy, thus obtaining the set of entropy values ​​for each pair of nodes in the graph. Then, it sets ρ as a threshold, selecting edges with entropy values ​​lower than ρ, and updates the adjacency matrix based on the selection. After obtaining the filtered adjacency matrix and node embeddings, a multi-head attention mechanism is used to dynamically learn the graph structure to distinguish the importance of different edges. Specifically, when the PLM dimension is d and the set of edge types in the graph is denoted as U, the adjacency matrix... The calculation formula is:

[0178]

[0179] Among them, TK Vis a node representation, and are learnable parameter matrices and biases, M u is a binary adjacency matrix of u-type edges;

[0180] a propagation unit for constructing a graph convolutional network (GCN) on a dynamic document graph to model the interaction between nodes, and the information transmission process is determined by the adjacency matrix learned by the model guiding so that information is transmitted between nodes along important paths determined by learned weights, set and respectively represent the input and output representations of node v i in the mth layer GCN, The calculation formula is:

[0181]

[0182] wherein, represents the adjacency matrix weight between nodes v i and v j , represents the input representation of node v j in the mth layer GCN, W m and b m are learnable parameter matrices and biases, and σ is a Relu activation function.

[0183] In the embodiment, the fusion module includes a capture unit, a concatenation unit, and an adjustment unit.

[0184] The capture unit is configured to construct a feature vector for generating a dynamic threshold value, wherein the feature vector is obtained by splicing head entity representation E h , tail entity representation E t , and context information r h,t captured by a pre-trained model (PLM), and then inputting the feature vector into a fully connected layer to generate the dynamic threshold value and performing normalization through an activation function, and the calculation process is:

[0185] dyn_TH=σ([mean(E h )||mean(E t )||mean(r h,t )]·W TH +b TH );

[0186] wherein, dyn_TH is a dynamic threshold value, W TH and b TH are learnable parameter matrices and biases, and σ is a Sigmoid activation function.

[0187] A concatenation unit is configured to concatenate the entity representation output by the PLM with the context information and the graph network output features to obtain a further enhanced entity representation c h t Then, the c h is subjected to a contrast prototype regularization to enhance the distinguishability of the entity representation. Specifically, a cosine similarity between the head entity c h and a prototype feature vector weight matrix P is calculated to obtain a similarity score. After a softmax operation, a weight of each relationship category in the data set is obtained. Finally, the head entity representation is updated by weighted summation of the prototype feature vector weight matrix. The calculation process is as follows:

[0188]

[0189] wherein P is a learnable relationship prototype matrix, and a is a prototype influence strength.

[0190] An adjustment unit is configured to calculate the similarity of each block after the representation of the head and tail entities is blocked, calculate the final logits by a bilinear transformation function, and dynamically adjust the logits by using a generated dynamic threshold dyn_TH. The calculation process is as follows:

[0191]

[0192] wherein f is the result of dynamic adjustment, W r r and b are learnable parameters.

[0193] In the embodiment, the training module includes an adaptive threshold loss unit, a contrast learning loss unit, and an integration unit.

[0194] The adaptive threshold loss unit is configured to introduce an adaptive threshold loss in the loss calculation stage, that is, to regard the document-level relationship classification as a multi-label classification problem. The loss function of the relationship prediction part is formalized as follows:

[0195]

[0196] wherein Pos and Neg represent the positive class and the negative class for an entity pair T=(e h t ), respectively, and TH is a threshold relationship used to distinguish Pos and Neg.

[0197] ​​​The contrast learning loss unit is used for calculating the contrast learning loss function, first calculating the average representation of the sequence output through average pooling, and converting the label into a class index, second updating the prototype memory library based on a gating mechanism, dynamically updating the relationship prototype representation through the current feature, calculating the cosine similarity between the input sequence and each prototype in the prototype memory library, and introducing a temperature factor to control the similarity scaling, which is used to adjust the smoothness of the softmax function, then converting the label into a Boolean type mask, the pos_mask identifies the samples belonging to the positive class, the positive sample similarity is extracted through the Boolean mask pos_mask, and the negative sample similarity is extracted by taking the inverse mask ~ pos_mask, and the calculation process is:

[0198]

[0199] S + ={(i,c)|y ic =1};S - ={(i,c)|y ic =0};

[0200]

[0201] Wherein, Loss contrast is the contrast loss function, i is the sample index in the batch, c is the class label, k is the index variable, y ic is the label, P is the learnable relationship prototype matrix, fea is the current batch feature mean, s ic is the cosine similarity, tau is the learnable temperature coefficient, S + is the positive sample set, S - is the negative sample set; the integration unit is used for integrating the loss function Loss = Loss cls + lambda * Loss contrast , wherein lambda is a coefficient, so that the model can effectively distinguish the positive and negative relationships in the input data, and alleviate the data imbalance problem in the multi-label classification scenario.

[0202] The above-described embodiments are only descriptions of the preferred modes of the present application, and do not limit the scope of the present application, and various modifications and improvements to the technical solutions of the present application made by those skilled in the art without departing from the design spirit of the present application shall fall within the protection scope of the claims of the present application.

Claims

1. A document-level relation extraction method based on information gain and prototype contrastive learning, characterized in that, The method includes: Step 1: Input the document into the pre-trained model PLM to obtain the token sequence output TK and attention matrix A. At the same time, use the external tool SpaCy to extract all anaphoric words in the document, construct the initial graph structure of the document, and obtain the node embedding representation and adjacency matrix. Step 2: Construct the corresponding relation prototype vector representation for each type of relation in the DocRED and Re-DocRED datasets, and obtain the global document representation by averaging TK. Filter A to obtain the attention matrix A' to remove redundant information, and obtain the context representation and head and tail entity embeddings that meet the preset accuracy requirements. Step 3: Input the relation prototype vector representation, the document global representation, and the node embedding representation and adjacency matrix obtained through the document graph structure into the graph attention network composed of graph convolutional network and multi-head attention structure for node feature update. The node features are filtered by considering the information gain of nodes and edges, that is, the model no longer pays attention to irrelevant nodes and edges during feature update. Finally, the mention representation after feature update by graph network is obtained. The mention representation is then pooled with logsumexp to obtain the head and tail entity representations. Step 4: Introduce a dynamic threshold mechanism to adaptively adjust the classification boundary for each relation type, and fuse the context representation, head and tail entity embeddings, and head and tail entity representations output by the PLM and the graph network to obtain the logical output logits of the current entity pair for all other relations. Step 5: Train the model using the contrastive learning loss function and the adaptive threshold loss function to obtain a relational representation that meets the preset accuracy requirements.

2. The method according to claim 1, characterized in that, Step three includes the following steps: Computation node embedding h i and h j Cosine similarity between sim(h) i ,h j This allows us to obtain a similarity matrix S for a document, and then extract the elements s from the matrix. ij The corresponding similarity is converted into a probability distribution p ij The entropy of node i is then calculated using the following formula: Where n represents the number of nodes; The similarity between pairs of nodes in the graph is calculated by dot product and converted into a corresponding probability distribution. The entropy of each edge is calculated to measure its information content, thus obtaining the set of entropy values ​​for edges between each pair of nodes. Then, ρ is set as a threshold, and edges with entropy values ​​lower than ρ are selected. The adjacency matrix is ​​updated based on the selection, resulting in a filtered adjacency matrix and node embeddings. A multi-head attention mechanism is then used to dynamically learn the graph structure to distinguish the importance of different edges. Specifically, when the PLM dimension is d and the set of edge types in the graph is denoted as U, the adjacency matrix... The calculation formula is: Among them, TK V Represented as nodes, and M is a learnable parameter matrix and bias. u Let be a binary adjacency matrix of u-type edges; A graph convolutional network (GCN) is constructed on a dynamic document graph to model the interactions between nodes. The information transmission process is mediated by the adjacency matrix learned by the model. Guidance is provided to ensure that information is transmitted between nodes along important paths determined by weights learned through learning. and They represent nodes v respectively i The input and output representations of the m-th layer GCN are as follows: The calculation formula is: in, This represents the node v learned by the model. i With node v j The adjacency matrix weights between them Represents node v j The input representation of the m-th layer GCN, W m and b m σ is the learnable parameter matrix and bias, and σ is the activation function.

3. The method according to claim 1, characterized in that, Step four includes the following steps: A feature vector is constructed to generate a dynamic threshold, the feature vector being represented by concatenating header and footer entities E. h E t and the contextual information r captured by the pre-trained model PLM. h,t The threshold is obtained and then fed into a fully connected layer as input to generate a dynamic threshold. It is then normalized using an activation function. The calculation process is as follows: dyn_TH=σ([mean(E h )||mean(E t )||mean(r h,t )]·W TH +b TH ); Where dyn_TH is the dynamic threshold, W TH and b TH It is a learnable parameter matrix and bias, where σ is the activation function; The entity representation output by PLM is concatenated with contextual information and graph network output features to obtain a further enhanced entity representation c. h c t Then, for c h Contrast prototype regularization is performed to enhance the distinguishability of entity representations. Specifically, the head entity c is computed. h The cosine similarity between the relation and the prototype feature vector weight matrix P is used to obtain a similarity score. After performing a softmax operation, the weight of each relation category in the dataset is obtained. Finally, the head entity representation is updated by weighted summation of the prototype feature vector weight matrix. The calculation process is as follows: Where P is the learnable relation prototype matrix, and α is the prototype influence strength; After dividing the representation of the head and tail entities into blocks, the similarity of each block is calculated, and the final logits are calculated using a bilinear transformation function. The generated dynamic threshold dyn_TH is then used to dynamically adjust the logits. The calculation process is as follows: Where f is the result of dynamic adjustment, W r b r These are learnable parameters.

4. The method according to claim 1, characterized in that, Step five includes the following steps: In the loss calculation stage, an adaptive threshold loss is introduced, treating document-level relation classification as a multi-label classification problem. The loss function for relation prediction is formally expressed as: Where Pos and Neg represent the values ​​for an entity pair T = (e h ,e t The positive and negative categories of ) are defined by TH, which is a threshold relationship used to distinguish between Pos and Neg; In the stage of calculating the contrastive learning loss function, firstly, the average representation of the sequence output is calculated through average pooling, and the labels are converted into class indices. Secondly, the prototype memory is updated based on a gating mechanism, and the relation prototype representation is dynamically updated through the current features. The cosine similarity between the input sequence and each prototype in the prototype memory is calculated, and a temperature factor is introduced to control the similarity scaling to adjust the smoothness of the softmax function. Then, the labels are converted into boolean masks, and pos_mask identifies samples belonging to the positive class. The similarity of positive samples is extracted through the boolean mask pos_mask, and the similarity of negative samples is extracted by taking the inverse mask ~pos_mask. The calculation process is as follows: S + ={(i,c)|y ic =1};S - ={(i,c)|y ic =0}; Among them, Loss contrast To compare the loss functions, i is the sample index in the batch, c is the class label, k is the index variable, and y is the index variable. ic Let P be the learnable relation prototype matrix, fea be the mean of the current batch features, and s be the label. ic Let S be the cosine similarity, τ be the learnable temperature coefficient, and S be the temperature coefficient. + S is the set of positive samples. - The set of negative samples; Integrate the loss functions: Loss = Loss cls +λ·Loss contrast , where λ is a coefficient that enables the model to effectively distinguish between positive and negative relationships in the input data, thus alleviating the data imbalance problem in multi-label classification scenarios.

5. A document-level relation extraction system based on information gain and prototype contrastive learning, the system being used to implement the method described in any one of claims 1-4, characterized in that, The system includes: a first construction module, a second construction module, a filtering module, a fusion module, and a training module; The first construction module is used to input the document into the pre-trained model PLM to obtain the token sequence output TK and the attention matrix A. At the same time, it uses the external tool SpaCy to extract all anaphoric words in the document, and after constructing the initial graph structure of the document, it obtains the node embedding representation and the adjacency matrix. The second construction module is used to construct the corresponding relation prototype vector representation for each type of relation in the DocRED and Re-DocRED datasets, and obtain the global document representation by averaging TK, filter A to obtain the attention matrix A' to remove redundant information, and obtain the context representation and head and tail entity embedding that meet the preset accuracy requirements. The filtering module is used to input the relation prototype vector representation, the document global representation, and the node embedding representation and adjacency matrix obtained through the document graph structure into the graph attention network composed of graph convolutional network and multi-head attention structure for node feature update. The node features are filtered by considering the information gain of nodes and edges, that is, the model no longer pays attention to irrelevant nodes and edges when updating features, and finally the mention representation after the graph network feature update is obtained. The mention representation is then pooled by logsumexp to obtain the head and tail entity representations. The fusion module is used to introduce a dynamic threshold mechanism to adaptively adjust the classification boundary for each relation type, and to fuse the context representation, head and tail entity embedding, and head and tail entity representation output by the PLM and the graph network to obtain the logical output logits of the current entity pair to all other relations. The training module is used to train the model using a contrastive learning loss function and an adaptive threshold loss function to obtain a relational representation that meets preset accuracy requirements.

6. The system according to claim 5, characterized in that, The filtering module includes: a calculation unit, an update unit, and a transmission unit; The computing unit is used to calculate the node embedding h. i and h j Cosine similarity between sim(h) i ,h j This allows us to obtain a similarity matrix S for a document, and then convert the obtained similarity into a probability distribution p. ij The entropy of node i is then calculated using the following formula: Where n represents the number of nodes; The update unit is used to calculate the similarity between pairs of nodes in the graph through dot product and convert it into a corresponding probability distribution. It measures the information content of each edge by calculating its entropy, thus obtaining the set of entropy values ​​for edges between each pair of nodes in the graph. Then, it sets ρ as a threshold, selects edges with entropy values ​​lower than ρ, and updates the adjacency matrix based on the selection. After obtaining the filtered adjacency matrix and node embeddings, a multi-head attention mechanism is used to dynamically learn the graph structure to distinguish the importance of different edges. Specifically, when the PLM dimension is d and the set of edge types in the graph is denoted as U, the adjacency matrix... The calculation formula is: Among them, TK V Represented as nodes, and M is a learnable parameter matrix and bias. u Let be a binary adjacency matrix of u-type edges; The transmission unit is used to construct the interactions between nodes modeled by the Graph Convolutional Network (GCN) on the dynamic document graph. The information transmission process is carried out by the adjacency matrix learned by the model. Guidance is provided to ensure that information is transmitted between nodes along important paths determined by weights learned through learning. and They represent nodes v respectively i The input and output representations of the m-th layer GCN are as follows: The calculation formula is: in, This represents the node v learned by the model. i With node v j The adjacency matrix weights between them Represents node v j The input representation of the m-th layer GCN, W m and b m σ is the learnable parameter matrix and bias, and σ is the activation function.

7. The system according to claim 5, characterized in that, The fusion module includes: a capture unit, a serial unit, and an adjustment unit; The capture unit is used to construct a feature vector for generating a dynamic threshold, and the feature vector is represented by splicing together header and footer entities to represent E. h E t and the contextual information r captured by the pre-trained model PLM. h,t The threshold is obtained and then fed into a fully connected layer as input to generate a dynamic threshold. It is then normalized using an activation function. The calculation process is as follows: dyn_TH=σ([mean(E h )||mean(E t )||mean(r h,t )]·W TH +b TH ); Where dyn_TH is the dynamic threshold, W TH and b TH It is a learnable parameter matrix and bias, where σ is the activation function; The concatenation unit is used to concatenate the entity representation output by the PLM with context information and the features output by the graph network to obtain a further enhanced entity representation c. h c t Then, for c h Contrast prototype regularization is performed to enhance the distinguishability of entity representations. Specifically, the head entity c is computed. h The cosine similarity between the relation and the prototype feature vector weight matrix P is used to obtain a similarity score. After performing a softmax operation, the weight of each relation category in the dataset is obtained. Finally, the head entity representation is updated by weighted summation of the prototype feature vector weight matrix. The calculation process is as follows: Where P is the learnable relation prototype matrix, and α is the prototype influence strength; The adjustment unit is used to divide the representation of the head and tail entities into blocks, calculate the similarity of each block, calculate the final logits using a bilinear transformation function, and dynamically adjust the logits using a generated dynamic threshold dyn_TH. The calculation process is as follows: Where f is the result of dynamic adjustment, W r b r These are learnable parameters.

8. The system according to claim 5, characterized in that, The training module includes: an adaptive threshold loss unit, a contrastive learning loss unit, and an integration unit; The adaptive threshold loss unit is used to introduce adaptive threshold loss during the loss calculation stage, that is, to treat document-level relationship classification as a multi-label classification problem, and the loss function of the relationship prediction part is formally expressed as: Where Pos and Neg represent the values ​​for an entity pair T = (e h ,e t The positive and negative categories of ) are defined by TH, which is a threshold relationship used to distinguish between Pos and Neg; The contrastive learning loss unit, during the contrastive learning loss function calculation stage, first calculates the average representation of the sequence output through average pooling and converts the labels into category indices. Second, it updates the prototype memory based on a gating mechanism, dynamically updating the relation prototype representation using the current features. It calculates the cosine similarity between the input sequence and each prototype in the prototype memory, and introduces a temperature factor to control similarity scaling, adjusting the smoothness of the softmax function. Then, it converts the labels into Boolean masks, with `pos_mask` identifying samples belonging to the positive class. Positive sample similarity is extracted using the Boolean mask `pos_mask`, and negative sample similarity is extracted by taking the inverse mask `pos_mask`. The calculation process is as follows: S + ={(i,c)|y ic =1};S - ={(i,c)|y ic =0}; Among them, Loss contrast To compare the loss functions, i is the sample index in the batch, c is the class label, k is the index variable, and y is the index variable. ic Let P be the learnable relation prototype matrix, fea be the mean of the current batch features, and s be the label. ic Let S be the cosine similarity, τ be the learnable temperature coefficient, and S be the temperature coefficient. + S is the set of positive samples. - The set of negative samples; The integration unit is used to integrate the loss functions Loss = Loss cls +λ·Loss contrast , where λ is a coefficient that enables the model to effectively distinguish between positive and negative relationships in the input data, thus alleviating the data imbalance problem in multi-label classification scenarios.

Images