A multi-granularity contrast learning-based distant supervision relation extraction method

Abstract

The application provides a remote supervision relationship extraction method based on multi-granularity contrast learning, and relates to the technical field of information extraction. The method comprises the following steps: obtaining a sentence in a data sample; segmenting the sentence into a word sequence; marking the head and tail of the word sequence respectively to obtain a marked sentence; obtaining initial sentence features by using a bidirectional encoder; obtaining initial positive sample features by using a positive example pair encoder; obtaining constraint features of a relationship and an entity pair according to a graph encoder and fusing the constraint features into corresponding initial sentence features and initial positive sample features to obtain enhanced sentence features and enhanced positive sample features; fusing the enhanced sentence features into bag-level features; inputting the bag-level features and the enhanced positive sample features into a contrast learning relationship classifier to extract the relationship of the data sample and obtain a classified relationship. The application solves the problem of low accuracy and efficiency of relationship extraction and weak long-tail relationship extraction capability caused by ignoring the interaction between different hierarchical features.

Description

Technical Field

[0001] This invention relates to the field of information extraction technology, and in particular to a method for extracting remote supervised relations based on multi-granularity contrastive learning. Background Technology

[0002] Relation extraction is a fundamental task in natural language processing and a key subtask of information extraction. The purpose of relation extraction is to automatically identify entities (such as names, locations, and organizations) and the semantic relationships between them from text. These relationships are typically represented as triples of [head entity, relation, tail entity] or structures of [subject, predicate, object]. The semantic information obtained from relation extraction provides support for knowledge graph construction, information retrieval, and intelligent question-answering systems, and is one of the key technologies driving the development of the Semantic Web and artificial intelligence.

[0003] However, supervised relation extraction methods have significant limitations, often relying on large amounts of manually labeled text data. In the era of big data, manual labeling is not only time-consuming and labor-intensive but also extremely costly. Furthermore, as knowledge expands and updates rapidly, manually labeled datasets quickly become outdated, failing to meet the demands of real-time updates. Therefore, remote supervision methods are being applied to relation extraction tasks.

[0004] Remote supervision is a method that pairs structured knowledge bases with unstructured corpora to automatically generate labeled data (e.g., Figure 1 Remote supervision follows a fundamental assumption: if two entities have a relation record in a knowledge base, then any text containing these two entities likely expresses that relation. This assumption allows remote supervision to automatically annotate text data using existing structured knowledge bases (such as Freebase, Wikidata, etc.), reducing the workload of manual annotation, facilitating the expansion and updating of training data, enabling relation extraction models to be trained on large amounts of data, and improving the scalability and efficiency of relation extraction.

[0005] While remote supervision methods can automatically label training data, they can also introduce mislabeling. This is because in real-world datasets, not all sentences containing specific entity pairs accurately represent the relationships in the knowledge base. For example, entities may have multiple relationships, entity references in the text may be ambiguous, or the relationships corresponding to entities may be outdated. Such mislabeling becomes noise in the training data, negatively impacting the learning performance of the relation extraction model.

[0006] To mitigate the negative impact of noisy data on model performance, past research has primarily employed long-range supervised relation extraction methods that combine multi-instance learning (MIM) with attention mechanisms. For example, Zeng et al. were the first to combine MMI and deep learning neural networks in the field of long-range supervised relation extraction, reducing the negative impact of mislabeled data in long-range supervised datasets on the model. MMI allows entity relations to be learned from "instance bags" containing multiple text instances, thus mitigating the negative impact of noisy labeling. However, long-range supervised relation extraction models based on MMI often process data with coarse granularity, focusing on bag-level representations while failing to fully utilize information at other levels and ignoring the importance of interactions between features at different levels. This strategy leads to low data utilization efficiency because the semantic information of features at other levels and the structured information between entity relations are not fully explored.

[0007] In relation extraction tasks, models typically leverage contextual information to understand relationships between entities. Focusing on only one level of feature information will neglect information from other levels, leading to information loss. For example, sentence-level features can provide sentence structure information and semantic information between entities, while bag-level features can provide context and background information about the relation triples. Utilizing only one level of feature information will result in the loss of feature information at another level, thus failing to capture the more complex semantic information between entities and ultimately degrading model performance.

[0008] Furthermore, remotely supervised datasets consistently suffer from a severe long-tail problem. That is, most sentence instances correspond to a small number of relations (i.e., head relations), while most relations (i.e., long-tail relations) correspond to a relatively small number of sentence instances. For example, in the NYT10 dataset, the most commonly used dataset in the field of remotely supervised relation extraction, over 60% of the relations are long-tail relations, with fewer than 100 corresponding instances. However, this long-tail problem is often overlooked in the field of remotely supervised relation extraction, and even state-of-the-art models fail to handle it well. Summary of the Invention

[0009] To overcome the shortcomings of existing technologies, the purpose of this invention is to provide a remote supervised relation extraction method based on multi-granularity contrastive learning. This invention solves the problems of low accuracy and efficiency of relation extraction and weak long-tail relation extraction capability caused by ignoring the interaction between features at different levels in existing technologies.

[0010] To achieve the above objectives, the present invention provides the following solution:

[0011] A method for extracting remote supervised relations based on multi-granularity contrastive learning is applied to a remote supervised relation extraction model. The model includes a positive example pair encoder, a graph encoder, and a bag encoder. The positive example pair encoder is used to obtain initial positive sample features. The graph encoder is used to extract constraint features of relations and entity pairs in the constraint graph and fuse these constraint features into the corresponding initial sentence features and initial positive sample features to obtain enhanced sentence features and enhanced positive sample features. The bag encoder is used to fuse the enhanced sentence features into bag-level features. The method includes:

[0012] Extract sentences from the data sample;

[0013] The sentence is segmented into a sequence of words;

[0014] The first and last parts of the word sequence are marked to obtain the marked sentence;

[0015] The tagged sentence is encoded using BERT (bidirectional encoder), and the first and last words of the word sequence are concatenated to obtain the initial sentence features;

[0016] Based on the sentence, the encoder obtains initial positive sample features using the positive examples;

[0017] The graph encoder obtains constraint features of relations and entity pairs and fuses these constraint features into the corresponding initial sentence features and initial positive sample features to obtain enhanced sentence features and enhanced positive sample features.

[0018] The enhanced sentence features are fused into packet-level features using the packet encoder.

[0019] The package-level features and the enhanced positive sample features are input into the contrastive learning relation classifier to extract the relationships between the data samples and obtain the classification relationships.

[0020] Preferably, the method for constructing the encoder for the positive example is as follows:

[0021] Obtain the first PLM encoder, the second PLM encoder, the first multi-instance attention layer, and the second multi-instance attention layer to construct a positive example pair encoder;

[0022] A positive example encoder is constructed based on the first PLM encoder, the second PLM encoder, the first multi-instance attention layer, and the second multi-instance attention layer, wherein the first PLM encoder is connected to the first multi-instance attention layer, and both the first multi-instance attention layer and the second PLM encoder are connected to the second multi-instance attention layer.

[0023] Preferably, the graph encoder employs a 2-layer GCN.

[0024] The present invention discloses the following technical effects:

[0025] This invention provides a method for extracting remote supervised relations based on multi-granularity contrastive learning, applied to a remote supervised relation extraction model. The remote supervised relation extraction model includes: a positive example pair encoder, a graph encoder, and a bag encoder. The positive example pair encoder is used to acquire initial positive sample features. The graph encoder is used to extract constraint features of relations and entity pairs in the constraint graph and fuse these constraint features into the corresponding initial sentence features and initial positive sample features to obtain enhanced sentence features and enhanced positive sample features. The bag encoder is used to fuse the enhanced sentence features into bag-level features. The method includes: acquiring sentences from data samples; segmenting the sentences into word sequences; and adding header and tail headers to the word sequences. The process involves several steps: First, the sentences are labeled separately to obtain labeled sentences. Then, a BERT (bidirectional encoder) is used to encode the labeled sentences, and the first and last words of the word sequence are concatenated to obtain initial sentence features. Based on the sentences, the positive example encoder is used to obtain initial positive sample features. Next, a graph encoder is used to obtain constraint features for relational and entity pairs, which are then fused into the corresponding initial sentence features and initial positive sample features to obtain enhanced sentence features and enhanced positive sample features. Finally, a bag encoder is used to fuse the enhanced sentence features into bag-level features. The bag-level features and the enhanced positive sample features are then input into a contrastive learning relation classifier to extract relations from the data samples, resulting in classification relations. This invention enhances the extraction capability of long-tail features by specially labeling the beginning and end of sentences and utilizes the interaction of features at different granularities to perform semantic interaction between features at different levels, thereby improving the accuracy of relation extraction. Attached Figure Description

[0026] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0027] Figure 1 A schematic diagram of prior art remote monitoring provided for embodiments of the present invention;

[0028] Figure 2 This is a schematic diagram of the remote supervision relationship extraction model structure provided in an embodiment of the present invention;

[0029] Figure 3 A schematic diagram of the encoder structure is provided as a positive example in an embodiment of the present invention;

[0030] Figure 4This is a schematic diagram comparing the PR curves of different models in NYT10 provided in the embodiments of the present invention;

[0031] Figure 5 This is a schematic diagram illustrating the experimental results of different models provided in the embodiments of the present invention on NYT10. Detailed Implementation

[0032] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0033] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0034] like Figure 2 As shown, this invention provides a multi-granularity contrastive learning approach for distantly-supervised relation extraction (MGCL), applied to a distantly-supervised relation extraction model. The distantly-supervised relation extraction model includes: a positive example pair encoder, a graph encoder, and a bag encoder. The positive example pair encoder is used to obtain initial positive sample features. The graph encoder is used to extract constraint features of relations and entity pairs in the constraint graph and fuse these constraint features into the corresponding initial sentence features and initial positive sample features to obtain enhanced sentence features and enhanced positive sample features. The bag encoder is used to fuse the enhanced sentence features into bag-level features. The method includes:

[0035] Step 100: Obtain sentences from the data sample;

[0036] Step 200: Segment the sentence into a sequence of words;

[0037] Step 300: Mark the beginning and end of the word sequence respectively to obtain the marked sentence;

[0038] Step 400: Encode the tagged sentence using a bidirectional encoder and concatenate the first and last words of the word sequence to obtain initial sentence features;

[0039] Step 500: Based on the sentence, use the positive examples to obtain initial positive sample features for the encoder;

[0040] Step 600: Obtain the constraint features of relations and entity pairs according to the graph encoder, and fuse the constraint features into the corresponding initial sentence features and initial positive sample features to obtain enhanced sentence features and enhanced positive sample features;

[0041] Step 700: Use the packet encoder to fuse the enhanced sentence features into packet-level features;

[0042] Step 800: Input the package-level features and the enhanced positive sample features into the contrastive learning relation classifier to extract the relations of the data samples and obtain the classification relations.

[0043] Specifically, based on multi-instance learning and contrastive learning, this method utilizes a comprehensive modeling approach combining contrastive learning and constraint graphs to achieve interaction among four granular features: entities, sentences, instance packages, and entity relation constraints. The remote supervised relation extraction method MGCL, based on multi-granularity interactive contrast, aims to improve the accuracy and efficiency of relation extraction through multi-dimensional information interaction and contrastive learning. MGCL's core innovation lies in its unique multi-granularity feature utilization strategy and contrastive learning framework. By combining feature information of different granularities, it achieves a deep understanding and efficient execution of remote supervised relation extraction tasks. During the construction of contrastive learning samples, MGCL leverages the interaction of features of different granularities to perform semantic interaction between features at different levels. It also employs an asymmetric contrastive learning strategy to conduct cross-comparison of multi-granularity features, thereby providing the relation extraction model with richer and more multi-dimensional learning signals and refining the accuracy of relation classification. Furthermore, this method utilizes the constraint relationships between entity classes and relation classes to enrich the semantic representation of rare nodes, further enhancing the model's ability to extract long-tail relations.

[0044] Specifically, MGCL constructs a contrastive learning positive example pair encoder that interacts with bag-level features during data augmentation and sentence-level positive example pair feature construction. It leverages the "multi-instance context" information in instance bags to reduce noise introduced during positive example pair construction. MGCL introduces a constraint graph and graph convolutional networks (GCNs) to extract interaction information between entities and relations in the constraint graph. GCNs can capture the global structural properties of entities in the constraint graph and increase information about rare entities through information propagation from neighboring nodes. We interact the entity constraint features obtained by GCNs with sentence features to enhance the entity information in the sentence features. Simultaneously, entity constraint features contain the mutual constraints between different entities; feature interaction can also highlight the correlation between entities, facilitating information sharing among different sentence instances within the multi-instance learning framework. This improves the performance of the multi-instance learning process, enabling it to more accurately filter and correct noisy labels.

[0045] We treat the interaction between global and local information as an implicit ensemble learning method to achieve better comprehensive modeling results. MGCL's packet encoder integrates local information (sentence-level features) from text data and global information (entity relationship constraint features) from constraint graphs, capturing and integrating relational cues scattered across different instances. This enhances MGCL's ability to distinguish between relevant and irrelevant instances in complex datasets and reduces the impact of noisy annotations.

[0046] Specifically, such as Figure 3 As shown, the Positive Pair Encoder is used to fuse and interact with sentence-level features and bag-level features; the Graph Encoder is used to extract constraint features of relations and entities in the Constraint Graph, and interacts with sentence features using entity constraint features to enhance entity information in sentence features; the Bag Encoder integrates sentence-level features and entity relation constraint features, and uses entity and relation constraints to adjust the attention layer of the Bag Encoder.

[0047] In MGCL, the pre-trained language model encoder (PLM encoder) uses BERT (Bidirectional Encoder Representation from Transformers), and the classifier uses a contrastive learning classifier. The constraint graph data comes from the existing OntoNotes 5.0. The PLM encoder is used to extract sentence S. ij Feature information S is obtained from the encoder using the improved positive examples. ij Features of positive sample pairs S ij This indicates package B. j The i-th sentence in the text, B j It represents the set of all sentences that contain the same entity pair.

[0048] Graph encoders learn the embedded representations of nodes by propagating node features, which in turn learns the constraint features of relationships and entity pairs from the constraint graph.

[0049] Graph encoders learn the embedded representations of nodes by propagating node features, which in turn learns the constraint features of relationships and entity pairs from the constraint graph. After that and Integrate into the corresponding sentence features and positive sample features Obtain enhanced sentence features and positive sample features Within the same instance package, a package encoder is used to fuse all enhanced sentence features into package-level features. Sentence S ij Use package-level features from other packages as negative sample pairs.

[0050] In the text preprocessing stage, we follow the currently specified standard workflow and use BertTokenizer to convert S... ij Segmented into word sequences (t1, t2, ..., e1, ..., e2, ..., t l In this model, e1 and e2 represent tokens for two entities in the sentence, and l represents the sentence length. Two tokens, [CLS] and [SEP], are added at the beginning and end of the sentence to help the model better understand the structure and meaning of the input text. Furthermore, to enhance the model's entity perception capability, this study adds four special tokens ([H-CLS], [H-SEP]) and ([T-CLS], [T-SEP]) before and after the head and tail entities to mark their positions, allowing the model to encode sentences using an entity-aware approach.

[0051] Using Bert on S ij The encoding process involves multiple layers of a Transformer structure to progressively extract deeper semantic information. The final sentence is transformed into a series of context-sensitive vector representations. By concatenating the vector representations of the head entity e1 and the tail entity e2, an entity-aware sentence embedding is obtained.

[0052]

[0053] Furthermore, the method for constructing the encoder for the positive example is as follows:

[0054] Obtain the first PLM encoder, the second PLM encoder, the first multi-instance attention layer, and the second multi-instance attention layer to construct a positive example pair encoder;

[0055] A positive example encoder is constructed based on the first PLM encoder, the second PLM encoder, the first multi-instance attention layer, and the second multi-instance attention layer, wherein the first PLM encoder is connected to the first multi-instance attention layer, and both the first multi-instance attention layer and the second PLM encoder are connected to the second multi-instance attention layer.

[0056] Using data augmentation to generate a sentence as similar as possible to the current sentence instance as a positive example is a common technique in NLP contrastive learning methods. This chapter utilizes the Dropout mechanism in neural networks to perform data augmentation on sentence instances in the packet at the encoding level. Because the Dropout method can be considered the smallest unit of data augmentation at the encoding level, the generated positive sample maintains a semantic consistency with the original sample, thus introducing as little additional noise to the model as possible.

[0057] Furthermore, to avoid the problem that contrastive learning methods tend to assume that two sentences with the same encoding length are semantically more similar when using only Dropout for data augmentation, and to achieve information interaction with bag-level features, we combine Dropout and multi-instance attention to implement a novel data augmentation method, which is constructed as a PositivePairEncoder, such as... Figure 4 As shown:

[0058] First, by utilizing the Dropout mechanism in BERT, different Dropout masks for hidden layers are used in the two forward passes, allowing sentence S to... ij Obtain two feature representations that have the same semantics but different forms. and in As a characteristic of the sentence itself.

[0059] To prevent the model from over-relying on auxiliary information and avoid overfitting, this study calculates the similarity between the relation labels and auxiliary information before feature concatenation, and designs this similarity as a modulation factor. This is used to adjust the feature concatenation process, thereby obtaining an ontology-enhanced sentence representation.

[0060]

[0061] In order to obtain sentence S ij As a positive example, we will use S ij The instance package B j The remaining sentences in the text form a subpackage B' j Following existing soft attention mechanisms, we obtain the computational multi-instance attention score α for each sentence embedding. i :

[0062]

[0063] in S represents ij Its relationship r j The degree of alignment and matching between them. A is a weighted diagonal matrix, q rRepresents the predictive relation, r j corresponding feature vector This indicates that α... i The size will depend on Mapping to q r The degree of above, and r j A more matching sentence will receive a higher attention score.

[0064] Get B j Feature representation F(B') j ):

[0065]

[0066] Where K is package B j The size. Then, this multi-instance attention layer is used for processing. and F(B') j ), to obtain sentence S ij Enhanced feature representation

[0067] In the above process, the positive examples use two layers of multi-instance attention on the encoder. The first layer of attention primarily aims to extract packet-level features and filter semantic noise. The second layer of attention allows sentence-level and packet-level features to be fused, thereby enabling cross-layer data interaction. Furthermore, if the original sentence itself is noisy data, the second layer of attention can mitigate the negative impact of the original sentence's feature representation on the positive examples' encoder, causing the encoder to focus more on sub-packet B'. j Feature F(B') j Since the sentence features obtained by the sentence encoder represent the feature information of entity relations, the bag-level features learned through multi-instance learning can be regarded as the feature representation of the current entity relations. Therefore, F(B') can be used. j Construct positive examples of features to further reduce the impact of noisy data on the model.

[0068] Furthermore, the graph encoder employs a 2-layer GCN.

[0069] Specifically, this invention uses GCN to obtain constraint representations of entities and relationships. In order to capture dependencies between entities while avoiding over-parameterization, the 18 entity types in OntoNotes 5.0 are used, and NA is added as the 19th type to represent relationship types that do not belong to these 18 types.

[0070] This invention defines a directed graph G = (V, E), where V is a set of nodes representing entity types and relations. It is a set of edges, representing semantic connections between entity types and relations. Each node v∈(V) in the graph is connected to a d-dimensional vector x. v The association is the vector, which is its embedding representation, initialized as X∈R. (|V|×d) .

[0071] This invention uses a two-layer GCN. Each layer of the GCN transforms the feature representation of nodes using the following update rules:

[0072]

[0073] Among them, Z (l) It is the node feature matrix of the l-th layer. It is the sum of the adjacency matrix A and the identity matrix I of graph G, used to add self-joins to preserve the feature information of the nodes themselves. yes The degree matrix (DegreeMatrix) has diagonal elements W is the degree of node i plus one. (l) Let be the learnable weight matrix of the l-th layer, and σ be the non-linear activation function (ReLU is used in this chapter). After propagation through the L-layer GCN, the high-level feature representation H of each node is obtained. (L) Finally, Z (L) The entity type representation T and relation representation R are separated as follows:

[0074] T,R = split(z) (L) );

[0075] Here, `split` represents an operation that splits the Z nodes based on their entity type or relation. (L) The rows are divided into different matrices.

[0076] For sentence S ij Flair is used to identify the type of entity pairs, and the type representation of the entity pairs is obtained by looking up the type representation matrix T. Sentence S ij Features The connection between entity type representations and the sentence features obtained are enhanced.

[0077]

[0078] For sentence S ij Enhanced feature representation Perform the same processing to obtain positive example pairs features enhanced with constraint graphs, which are then used for positive example comparison.

[0079] Furthermore, in order to utilize the global information of entity relation constraints and the local information in sentence-level features, the entity relation constraints obtained by the graph encoder are introduced into the bag encoder based on the multi-instance bag encoder. This is done to improve the efficiency of the bag encoder and provide an additional information dimension for the model.

[0080] The constraint representation obtained by the graph encoder is defined as: A feedforward neural network (FNN) is used to process sentence features and constraints, obtaining the constraint confidence c for each sentence. i :

[0081]

[0082] Among them, W c and b c The weight matrix and bias vector are obtained through random initialization and training. Sentence S is then obtained. ij Corresponding multi-instance constrained attention score for:

[0083] Then, attention scores are constrained based on multiple instances. Design a bag-level encoder F to obtain the multi-instance constrained feature representation F(Bj) of bag Bj. j ):

[0084]

[0085] Next, we use a fully connected layer with a softmax activation function to convert F(B) into a more efficient and efficient function. j The input is fed into this fully connected layer and mapped to a conditional probability function p(r). j |F(B j ),θ):

[0086]

[0087] Where, n r Let o = MF(B) be the number of all relations. j )+b represents the final output packet B of the neural network. j With the similarity score for each relation, M is a matrix consisting of vectors of all entity relations, and b is the bias vector.

[0088] Finally, the cross-entropy loss function is defined as the objective function of the packet-level encoder.

[0089]

[0090] Because F(B) jThe graph encoder contains constraint information learned by the graph encoder. The graph encoder and the pack encoder are complementary in function, and the graph encoder also uses the above formula as the learning target.

[0091] Furthermore, the MGCL model uses InfoNCE as the target for contrastive learning of invention.

[0092]

[0093] Here, sim(a,b) is used to measure the similarity between two feature vectors a and b, and τ is used as a divisor in softmax to adjust the similarity score. This invention divides the similarity score of positive samples by the sum of the similarity scores of all positive and negative samples to obtain a standardized probability distribution.

[0094] The overall learning objectives of MGCL are as follows:

[0095]

[0096] in, γ is a function positively correlated with the training step t, which allows MGCL to focus on training the bag encoder in the early stages of training, and gradually shift the training focus to contrastive learning as training progresses. N represents the batch size, and γ M These are the parameters of the masked language loss function.

[0097] This embodiment also provides the following experiments:

[0098] Regarding the experimental data: This invention trains and evaluates the model on the NYT10 dataset, which is commonly used in the field of remote supervised relation extraction. The composition of the dataset is shown in Table 1. Unknown relations (NA) represent relations that cannot be labeled by remote supervision. Table 1 shows the statistics of the dataset, as shown below:

[0099] Table 1. Statistics of the Dataset

[0100]

[0101] Furthermore, evaluation metrics: Based on the existing experimental procedures, several commonly used evaluation metrics in the field of remote supervision relation extraction are used, including AUC, P@N, and P@M. Among them, the area under the curve (AUC) measures the overall performance of the model by calculating the area under the ROC curve, and the precision of the N (P@N) metric is used to evaluate the precision at each cutoff point.

[0102] Furthermore, regarding the experimental parameter settings: In MGCL, the sentence encoder uses BERT_base, containing 12 Transformer encoder layers and 110M parameters, with each hidden state vector having a dimension of 768. During the training phase, based on previous experience, the learning rate (lr) is set to 2e-5, and the dropout ratio (dropout_prob) is 0.1. The initial value of the temperature parameter used to control the calculation of positive and negative sample similarity is set to the default value of 0.05. For the optimizer used to update the model parameters, we use AdamW, with the learning rate (lr) set to 0.1 and eps set to the default value of 1e-8. During the generation of the masked language model (MLM), 80% of the tokens are set to be replaced with [MASK] tokens, 10% of the tokens are replaced with random words, and 10% of the tokens are left unchanged. In the positive pair encoder, the generation... and The dropout_prob value was adjusted to 0.3. In the packet encoder, the W value of the feedforward neural network... c and b c Random initialization is performed, and the weights are initialized by sampling from a relatively small standard normal distribution. This addresses the symmetry of the model and the problem of gradient nullification, and promotes the weights to evolve in different directions at the beginning of training.

[0103] For the comparative experiment:

[0104] This invention selected six baseline models for experimental comparison. The experimental results of different models on NYT10 are shown in Table 2 and 3. Figure 5 As shown in Table 2. Table 2 shows the experimental results of different models on NYT10, as shown below:

[0105] Table 2. Experimental results of different models on NYT10.

[0106]

[0107] In Table 2, the MGCL model has the highest AUC value compared to several baseline models, indicating that MGCL can more accurately identify various relation types, reduce over-reliance on mislabeled data, and can also accurately distinguish relation types with higher similarity, proving the effectiveness of MGCL. Although MGCL's P@100 metric is slightly lower than the best performance (CIL) among the baseline models, ranking only second, the number of the top 100 relation instances is relatively small, and these instances have a low probability of being noise, so it cannot reflect the overall effect of the model. P@300 and P@500 are both higher than other baselines, and the mean P@M is also the highest. The improvement in P@300 and P@500 means that MGCL has a higher proportion of positive examples in the top 300 or top 500 prediction results, indicating that MGCL can better identify positive examples and has a stronger ability to distinguish true examples. It also shows that the model's generalization ability has been improved, and it can perform well for different relation types.

[0108] exist Figure 5 In comparison to CIL, the best-performing baseline model, MGCL's PR curve outperforms CIL in most recall scenarios, indicating that MGCL improves prediction accuracy while maintaining a certain level of recall. Furthermore, the earlier peak of the PR curve suggests that MGCL maintains high accuracy even with low recall, meaning it performs better in identifying positive samples. Additionally, since the NYT10 dataset is a standard long-distance supervised dataset containing noise, the earlier peak also indicates that the model is better adapted to noisy data.

[0109] Furthermore, regarding ablation experiments:

[0110] To verify the effectiveness of various methods in the MGCL model, ablation experiments were conducted on the NYT10 dataset. The experimental results are shown in Table 3. Table 3 presents the MGCL ablation experimental results, as shown below:

[0111]

[0112]

[0113] To verify the effectiveness of the positive pair encoder in MGCL, this section removes the constraint graph and graph encoder from MGCL, leaving only the contrastive learning part of multi-instance learning. MGCL's contrastive learning method is built on the basis of CIL. The AUC value of MGCL without the graph encoder is higher than that of CIL in Table 5-1, indicating that the positive pair encoder constructed in this chapter can improve the model's ability to distinguish different samples, and the interaction between sentence-level features and bag-level features enhances the model's ability to filter noise.

[0114] Because MGCL's positive examples utilize multi-instance learning attention in the encoder, it relies more heavily on instance packages. The long-tail distribution of data affects the construction of instance packages, resulting in MGCL's P@M being lower than CIL when the graph encoder is removed. However, this also reflects that the graph encoder component in MGCL can alleviate the problems caused by long-tail data in multi-instance learning.

[0115] We removed the entity constraint information provided by MGCL for sentence-level features. MGCL's packet encoder only processes features of the sentence itself, leading to a decrease in the performance of the ablation experiments. This demonstrates that the interaction between entity constraints and sentence features can improve the model's ability to identify entity relevance. The lack of this interaction process also makes the model more susceptible to noisy annotations.

[0116] We degenerate the reconstructed bag encoder into a more general multi-instance bag encoder, removing the entity relation constraints provided by the graph encoder. This causes MGCL to only utilize sentence-level features within instance bags to obtain contextual information about entity pairs within that bag, thus reducing MGCL's performance. This demonstrates that entity relation constraints in the constraint graph, as a form of global information, can highlight the relevance of entity pairs between different sentence instances within an instance bag and provide more entity relation cues for the multi-instance bag encoder.

[0117] To further verify the effectiveness of entity relation constraints, this section modifies the feedforward neural network for obtaining constraint confidence in the MGCL packet encoder, replacing the activation function of ci from ReLU to Sigmoid. Since the output of Sigmoid ranges from 0 to 1, the output is smoother, thus reducing the impact of entity relation constraints on the packet encoder and preventing a single sentence from excessively dominating the representation of the entire packet. Experimental results show that the weakened entity relation constraints can still have a positive impact on MGCL, demonstrating the effectiveness of MGCL's packet encoder and graph encoder.

[0118] The beneficial effects of this invention are as follows:

[0119] This invention improves the accuracy and robustness of remote supervised relation extraction by combining contrastive learning and constraint graph modeling, utilizing feature information at different granularities. It employs a novel method for constructing positive pairs for contrastive learning, using bag-level features provided by multi-instance learning to reduce noise introduced into the positive pairs. It introduces constraint graphs and GCNs to extract interaction information between entities and relations in the constraint graph, improving the modeling ability of rare relations and more accurately filtering and correcting noisy labels.

[0120] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. Specific examples have been used to illustrate the principles and implementation methods of the invention. The descriptions of the embodiments above are only for the purpose of helping to understand the method and core ideas of the invention; furthermore, those skilled in the art will recognize that, based on the ideas of the invention, there will be changes in specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the invention.

Claims

1. A multi-granularity contrast learning-based distant supervision relation extraction method, characterized in that, An application is made to a remote supervised relation extraction model, which includes a positive example pair encoder, a graph encoder, and a bag encoder. The positive example pair encoder is used to obtain initial positive sample features. The graph encoder is used to extract constraint features of relations and entity pairs in the constraint graph and fuse the constraint features into the corresponding initial sentence features and initial positive sample features to obtain enhanced sentence features and enhanced positive sample features. The bag encoder is used to fuse the enhanced sentence features into bag-level features. The method includes: Extract sentences from the data sample; The sentence is segmented into a sequence of words; The first and last parts of the word sequence are marked to obtain the marked sentence; The tagged sentence is encoded using a bidirectional encoder, and the first and last words of the word sequence are concatenated to obtain the initial sentence features. Based on the sentence, the encoder obtains initial positive sample features using the positive examples; The graph encoder obtains constraint features of relations and entity pairs and fuses these constraint features into the corresponding initial sentence features and initial positive sample features to obtain enhanced sentence features and enhanced positive sample features. The enhanced sentence features are fused into packet-level features using the packet encoder. The package-level features and the enhanced positive sample features are input into the contrastive learning relation classifier to extract the relationships between the data samples and obtain the classification relationships.

2. The multi-granularity contrast learning based distant supervision relation extraction method according to claim 1, characterized in that, The method for constructing the encoder in the positive example is as follows: Obtain the first PLM encoder, the second PLM encoder, the first multi-instance attention layer, and the second multi-instance attention layer; A positive example encoder is constructed based on the first PLM encoder, the second PLM encoder, the first multi-instance attention layer, and the second multi-instance attention layer, wherein the first PLM encoder is connected to the first multi-instance attention layer, and both the first multi-instance attention layer and the second PLM encoder are connected to the second multi-instance attention layer.

3. The multi-granularity contrastive learning based distant supervision relation extraction method according to claim 1, characterized in that, The graph encoder uses a 2-layer GCN.

Images