Data-driven zero-shot relation extraction method based on selective contrastive learning

Abstract

The application discloses a data-driven zero-shot relation extraction method based on selective contrast learning, and steps include: using a pre-trained language model as a basic relation encoder in the training process; two different prompt templates are used to expand instances, and the instances are input into the language model to generate instance representations; the representations are then subjected to prompt fine-tuning and contrast learning; prompt fine-tuning is used to find the most suitable labels to fill in the mask positions in the prompt templates, and contrast learning is used to find the corresponding relations of each instance; further relation reasoning is performed, and a learned good relation encoder is used to generate instance representations of unseen relations; a medical knowledge graph is constructed based on the extracted relations between entities; and semantic search is performed based on the constructed medical knowledge graph to obtain recommended medical encyclopedic knowledge.

Description

Technical Field

[0001] This invention belongs to the field of medical knowledge graph technology, and in particular relates to a data-driven zero-sample relation extraction method based on selective contrastive learning. Background Technology

[0002] Knowledge graphs describe the classification, attributes, and relationships of things, possessing rich semantic information that provides significant underlying support for semantic search. Semantic search based on medical knowledge graphs is currently used for semantic extraction from medical encyclopedias, clinical guidelines / literature, and healthcare information. Relation extraction (RE) aims to infer the relationship between a pair of entities from text, a crucial step in automatically constructing knowledge graphs. Recent supervised learning methods have attempted to define relation extraction as a multi-class classification task, selecting the most appropriate relation labels for target entity pairs from a predefined set of relations. However, in practical applications, training data is far from covering all relations; therefore, supervised relation extraction methods are not well-suited for identifying unknown relations during the training process.

[0003] To avoid the laborious annotation of newly emerging relations, unsupervised relation extraction (URE) was first proposed, discovering new relations through clustering without prior knowledge of the relation range and distribution. However, URE methods cannot utilize the relevant knowledge in previously accumulated relations. To address this limitation, zero-shot relation extraction (ZSRE) emerged as a remedy, training a model on historical relations with annotated instances and generalizing it to extract relations never seen during training. Since the relations used for training and testing under zero-shot conditions are disjoint, ZSRE models cannot obtain supervisory signals from test relations. Therefore, how to transfer knowledge from training data to test data remains a long-standing challenge for zero-shot learning.

[0004] To address the challenges of knowledge transfer, previous studies have tackled it by transforming ZSRE into proxy tasks such as reading comprehension, textual entailment, and attribute representation matching. Since these methods require defining response templates or relation descriptions, finding suitable and effective options can be arduous. Furthermore, these approaches rely on the assumption that humans know in advance which new relations need to be extracted, which often contradicts practical applications, as new relations are always derived from corresponding newly emerging instances. In other words, we believe that ZSRE models should be data-driven, not human-guided. In previous research, Wang et al. adopted this idea but failed to fully define how to perform data-driven ZSRE.

[0005] The key to solving the data-driven ZSRE problem lies in obtaining an efficient projection function that can map nearby instances of the same relation to distant instances of different relations in the embedding space. Given the success of contrastive learning in capturing discriminative representations, Wang et al. utilized an instance-level self-supervised contrastive approach to train a relation encoder to better learn subtle differences between instances. However, existing methods still suffer from three drawbacks when dealing with the aforementioned ZSRE paradigm: Disadvantage 1: Most existing relation extraction models use entity tag encoders to obtain the semantics of instances, which is insufficient in obtaining relevant knowledge embedded in pre-trained language models through prompts.

[0006] Disadvantage 2: Although contrastive learning has been successfully applied to relation extraction, previous work has failed to distinguish between different contrastive negative samples and has not been able to further improve the generalization ability to encode unknown relations.

[0007] Disadvantage 3: During the reasoning process, the test samples are divided into several groups, and each group will inevitably contain some false positive samples; however, few studies have yet attempted to filter out these false positives. Summary of the Invention

[0008] In view of this, this application designs a new selective contrastive learning framework, namely SCL, to handle data-driven ZSRE. (1) In the training phase, cue learning is adopted and adapted to the ZSRE task. By combining the pre-training objective and the downstream task, cue tuning can make full use of the knowledge obtained in the pre-trained language model. In addition, in order to better achieve the separation of relations, this application performs batch selective contrastive learning. For a set of instances, this application uses different cue word templates to enhance them to construct another view as positive contrastive instances. It is worth noting that this application assigns selective weights to various contrastive instances of anchors, in which this application emphasizes the hard-to-distinguish negative samples, because they are more useful for learning discriminative representations. (2) In the testing phase, following the method of Wang et al., this application uses a pre-trained model to project all test samples into the representation space and divides them into several clusters using the K-Means algorithm. This application summarizes the high-quality samples around each cluster center and uses them as the prediction results for all instances. According to observation, in each cluster, the distance of false positive samples from their cluster center is always larger, so this application proposes a post-processing method for detecting false positive instances. Specifically, this application designs a threshold criterion to determine whether a test instance is a false positive. Previous ZSRE work has never investigated improving performance through such a non-parametric method, indicating that this seemingly simple idea is not easy to implement.

[0009] The data-driven zero-shot relation extraction method based on selective contrastive learning disclosed in this application includes the following steps: Acquire medical drug data, which includes multiple entities, such as drug names or disease names. Label the relationships between drug names or disease names in the medical drug data to obtain a set of labeled relationship tags. During training, a pre-trained language model is used as the basic relation encoder; instances of entities are augmented with two different cue templates and then input into the language model to generate instance representations. The representation is then subjected to cue tuning and contrastive learning; the cue tuning is used to find the most suitable labels to fill the masking positions in the cue template, and the contrastive learning is used to find the correspondences for each instance; during the cue tuning process, the language model is expanded with a set of additional labels to represent the seen relations; in order to highlight the role of hard-to-contrast sample pairs, these hard-to-contrast sample pairs are dynamically emphasized to improve the discriminative ability of relations within the representation space. Further relational reasoning is performed by generating instance representations of unseen relations using a well-learned relational encoder; these instance representations are then divided into multiple clusters using the K-Means algorithm; for all clusters, the top k samples closest to the center point are selected and aggregated to find surfacing relations that can be used as predicted labels. If false positives are distributed on the boundaries of each cluster, a new cluster boundary is assigned to the target relation to exclude false positives. Based on the extracted relationships between entities, a medical knowledge graph is constructed; semantic search is then performed based on the constructed medical knowledge graph to obtain recommended medical encyclopedia knowledge.

[0010] Preferably, the zero-sample relation extraction task refers to: given a paragraph mentioning a pair of entities (e.g., ... s e o The text of the relation, also known as instances s, aims to identify the main entity e based on contextual clues. s Heke Entity e o The semantic relationship r between e is represented by the symbol t. s and e o The entity tuples that make up the entity.

[0011] Preferably, a template T(·) is added to the data sample to prompt instance s, wherein the prompt template contains a [MASK] tag, and the prompt template defined in the model is: ;

[0012] Among them, e s and e oThese are the head and tail of the entity, respectively; 's' are loaded into the template sequentially, and then a prompt input is generated: ; Will (e) s e o Inputting a pre-trained language model yields the hidden vector corresponding to the [MASK] position: ; Simplify the notation to r = h [MASK] It represents the instance relation representation used for cueing learning and contrastive learning.

[0013] Preferably, during the prompting optimization process, the task of the language model is to predict which word is suitable to fill in the [MASK] position for relation extraction; in order to extract the relation based on the relation representation r = h [MASK] To predict the target relation, we extend Ԑ with a set of learnable dummy relation labels V to fully represent the corresponding known relation R. s That is, using each virtual tag v i ∈V describes the relation r i ∈R s The implicit semantics, rather than a conventional mapper that maps a relation label to a label word in the vocabulary; To fully utilize relation r i The semantic information in the relation name is used to initialize the virtual word vector; Using the MLM head layer in a language model, [MASK] tags are recovered from the relation tag set based on inner product similarity: ; The MLM head layer corresponds to the relation classifier g(·); Obtain the training set The language model is optimized by minimizing the cross-entropy loss. ; r i It is the label of the i-th relation in the relation set.

[0014] Preferably, in the comparative learning process, a set of instances {s} is given. i}B i=1, construct an enhanced view for instance-level contrastive learning; construct contrastive positive instances by concatenating different cue templates for each instance; for (e s , e o For instance s, its enhanced view is: ; Among them, T´(es e o ) is another defined template: ; By providing a set of instances Add different prompt templates, and obtain two views representing the instance using the following formula. and

[0015]

[0016] Preferably, for anchor characterization r i Only its corresponding enhanced view As positive contrast instances; all instances with different relations in the augmentation set are used as negative contrast samples, denoted as... From a mathematical perspective, the importance of a negative contrast instance is measured by calculating the distance between it and the anchor point; the smaller the distance, the more difficult it is to represent a negative contrast instance. Therefore, the weight of a contrast instance is defined as: ; in yes and The Euclidean distance between them.

[0017] Preferably, in the contrastive learning process, the selective contrastive loss is defined as: ; τ is the adjustable temperature parameter, and B is the batch size. It is the cosine similarity, where r is the relational representation vector of the samples. It is the representation vector of the enhanced view; Optimize the encoder with joint loss: ; Here, λ1 and λ2 are two weights that constitute different losses.

[0018] Preferably, during the testing phase, the input unknown relation instances are... The data is fed into the trained encoder to generate their representations. The K-Means algorithm is used to divide the representation into m groups, where m is the number of unknown relations, and denoted as Cluster(m). → {C1,... , C m}; Each cluster C i Includes a set of test samples; relationships are summarized from typical samples. i, and assign it to cluster C iIn the implementation process, the sample closest to each cluster center is selected as the typical sample; C i The test samples will be assigned as prediction relationships. .

[0019] Preferably, false positive samples exist at the edges of the clusters; for the target predicted relation, these false positive samples are considered OOD samples. During relation inference, for an instance in the cluster, its distance to the center point is calculated, and a threshold criterion is used to detect whether it is an OOD sample. For the i-th cluster , ,..., ,in ||r|| represents the L2 norm value of the relation representation vector, and the cluster center is represented as the average of all normalized instance embedding vectors: ; The Euclidean distance from an instance to its centroid is calculated as follows: ; The decision function for OOD samples is: ; In the formula, δ is the OOD boundary threshold, 1{·} is the index function, and δ is the default setting.

[0020] The beneficial effects of this application are as follows: A more powerful relation encoder is proposed, which can be fine-tuned through cue learning and selective contrastive learning. This allows the encoder to generate more compact relation representations for better separation of unknown relations.

[0021] A post-processing method is designed to filter out erroneous prediction instances in each cluster. Using a non-parametric threshold criterion, we can detect false positives, thereby improving the accuracy of our proposed ZSRE model. Attached Figure Description

[0022] Figure 1 ZSRE's data-driven paradigm includes visible relation training, unknown relation clustering, and emergent relation discovery; Figure 2 : This application presents a flowchart of the process for handling zero-sample relation extraction. Detailed Implementation

[0023] The present invention will be further described below with reference to the accompanying drawings, but this is not intended to limit the present invention in any way. Any modifications or substitutions made based on the teachings of the present invention shall fall within the protection scope of the present invention.

[0024] Artificial intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess the functions of perception, reasoning, and decision-making.

[0025] Natural Language Processing (NLP) is an important field within computer science and artificial intelligence. It studies the theories and methods for enabling effective communication between humans and computers using natural language. NLP is a science that integrates linguistics, computer science, and mathematics. Therefore, research in this field involves natural language—the language people use in daily life—and thus it has a close relationship with linguistic research. NLP techniques typically include text processing, semantic understanding, machine translation, question answering, and knowledge graphs.

[0026] The technical solutions provided in this application involve technologies such as machine learning and natural language processing in artificial intelligence, which are specifically described and illustrated through the following embodiments.

[0027] Before introducing the embodiments of this application, some terms involved in this application will be explained.

[0028] 1. Zero-shot relation extraction (RE) is defined as determining the relationship between two annotated entities in a given sentence. To avoid the cost of annotating training instances for unknown relations, recent research has begun to explore zero-shot relation extraction (ZSRE). Since there are no training instances for testing relations, existing methods rely on annotation auxiliary information as input and transform RE into proxy tasks such as reading comprehension and sentence entailment.

[0029] In order to directly utilize the knowledge in the pre-trained BERT model, Chen and Li[3] proposed a ZS-BERT model to handle ZSRE through attribute representation learning. The representations of instances and relation descriptions can be derived from the BERT model, and the instance representations are matched with the description representations through ZS-BERT. Due to the progress in cue learning, Xu et al. designed multiple cue templates for an instance and derived multiple representations. These representations are then fused through an attention mechanism and matched with the representations of relation descriptions.

[0030] 2. Cue-based learning, through its cue-based tuning paradigm, aims to narrow the gap between pre-trained and fine-tuned pre-trained language models by designing downstream tasks as "cloze"-like tasks. Driven by GPT, cue-based learning has gained popularity in a wide range of natural language processing tasks, such as text classification and entity input.

[0031] Cue learning has also been applied to relation extraction tasks, significantly improving performance compared to previous methods. Among these methods, Han et al. proposed a PTR model that creatively applies logical rules to decompose cues into multiple sub-cues. To further leverage external knowledge, the KnowPrompt model injects entity and relation knowledge into a pre-trained language model. However, these cue-based relation extraction methods are tailored for supervised or few-shot relation extraction. In this application, cue learning is applied to relation extraction under zero-shot conditions.

[0032] 3. The underlying idea of ​​contrastive learning is to cluster similar data points and separate dissimilar data points. Essentially, contrastive learning makes instance representations in neural models more compact and better separated. Recently, in the field of natural language processing, contrastive learning has also been used for pre-training language models. Inspired by SimCLR, Gao et al. proposed a sentence representation learning framework called SimCSE, in which positive contrastive sample pairs are constructed using two independently sampled elimination masks. As a more comprehensive study, ConSERT investigated different data augmentation strategies for obtaining effective sentence representations through contrastive learning.

[0033] Unlike the contrastive learning methods described above, this application uses different cue templates to augment the data, thereby constructing positive contrast positive samples. This paper further elucidates the data-driven paradigm of ZSRE, and its workflow is as follows: Figure 1 As shown in the diagram. In this paradigm, a relation encoder is first trained on known relation instances, and then the encoder is used to map test samples to the embedding space. The embedding vectors of these test samples are then clustered into several groups, and high-quality samples from each cluster are selected and summarized using manual analysis or ChatGPT to generate new relation names, which are then assigned as predicted labels to the corresponding groups.

[0034] The following section introduces the data-driven zero-sample relation extraction method of this application. First, the data-driven ZSRE task is defined, and then the SCL method proposed in this application is summarized.

[0035] ZSRE Task Definition: Given a reference to a pair of entities (e.g., ... s e o The purpose of relation extraction is to identify the main entity e based on contextual clues, from the text (also called instances s). sand customer entity e o The semantic relationship between them is r. For clarity, this application uses the symbol t to denote the relationship between e and r. s and e o The entity tuples that make up the entity.

[0036] In ZSRE, there is a set of training texts. (i.e., entity pair t) i There is a comment relationship r i (examples) and a set of target test texts , where |D| (or |T2|) is the number of instances in D (or τ). The training set D includes a set of known relations. The test set τ includes another set of unseen relationships. Where n (respect) and these two relation sets are disjoint, i.e., R s ∩ R u = .

[0037] The method for solving the ZSRE task in this application consists of three steps. The first step aims to train a relation extraction model on the training data D, which comprises an encoder E and a classifier g(·), i.e., g(Ԑ(si)) → r∈Rs. In the second stage, the trained encoder E is used to extract the relation from the test set... Mapped to the embedding space, it is represented as Ԑ(τ) → These instances are embedded into m groups using a clustering algorithm, i.e., Cluster( →{C1,..., C m In the third stage, select some values ​​that are closest to each C. i Typical samples of the centroid are summarized manually or using ChatGPT to generate a relation name, denoted as Summarize(C i ) → r ′. Then the generated relation name is used as the group C. i The tags for all instances in the list.

[0038] This application proposes a novel Selective Contrastive Learning (SCL) framework to address the relation extraction problem under zero-shot conditions. As shown in Figure 2, it consists of three key components: a cue tuning module, a contrastive learning module, and a relation inference module.

[0039] During training, a pre-trained language model is used as the skeleton of the relation encoder. Given a batch of instances involving two entities, the model is augmented with two different cue templates and input into the language model to generate instance representations. These representations are then fed into a cue tuning module and a contrastive learning module. In the cue tuning module, the language model is expanded with a set of additional labels to represent seen relations. The goal of cue tuning is to find the most suitable labels to fill the masking positions in the cue templates. The goal of the contrastive learning module is to find the corresponding relation for each instance. To highlight the role of hard-contrasting sample pairs, this application dynamically emphasizes these hard-contrasting sample pairs, which helps improve the discriminative power of relations within the representation space. In the inference phase, instance representations of unseen relations are first generated using the well-trained relation encoder. These instance representations can be divided into m clusters using the K-Means algorithm. For all clusters, the top k samples closest to the center point are selected for aggregation to find surfacing relations as predicted labels. Furthermore, it was observed that due to the subtle differences among various relations, false positive relations tend to be distributed on the boundaries of each cluster. Therefore, for the target relation, a smaller cluster boundary is reassigned to exclude false positive relations.

[0040] The following section details the method proposed in this application for zero-sample relation extraction, namely SCL.

[0041] Prompt encoder This application employs a pre-trained language model as the basic relation encoder, denoted by Ԑ. Since cue-based optimization demonstrates a powerful ability to facilitate the acquisition of relational knowledge embedded in the language model, this work utilizes a cue-based encoder and adapts it for ZSRE.

[0042] To better optimize the prompts, the RE is designed as a cloze-style masked language prediction task. To achieve this, an appropriate template T(·) is added to the data samples to prompt instances s, where the prompt template must contain a [MASK] tag. The manually defined prompt template in the model is as follows: (1); Among them, e s and e o These are the head and tail of the entity, respectively. Load 's' into the template sequentially, then generate the input prompt: (2); Will (e) s e o By inputting a pre-trained language model, we obtain the hidden vector corresponding to the [MASK] position: (3); For clarity, the notation is simplified to r = h [MASK] It represents the instance relation representation used for cueing learning and contrastive learning.

[0043] Tips for optimization In the prompting optimization module, the task of the language model is to predict which word is suitable to fill in the [MASK] position for relation extraction. This is based on the relation representation r = h in equation (3). [MASK] To predict target relationships, it is necessary to introduce relationship tags to represent the relationship labels that need to be detected; To avoid time-consuming and laborious language engineering, this application extends Ԑ with a set of learnable virtual relation notations V to fully represent the corresponding known relation R. s In other words, we set each virtual tag v i ∈V can describe the relation r i ∈R s The implicit semantics, rather than a conventional mapper that maps a relation label to a label word in the vocabulary. To fully utilize relation r i The semantic information in the relation is used to initialize the virtual word vectors with the average word vector of the relation name. For example, the virtual token v1 corresponding to the relation "born in" is initialized as: v1 = (Emb born + Emb in ) / 2.

[0044] Using the MLM head layer in a language model, [MASK] tags can be recovered from a set of relation tags based on inner product similarity: (4); The MLM head layer corresponds to the relation classifier g(·) mentioned in Section 3; With training set Therefore, the language model can be optimized by minimizing the cross-entropy loss. (5).

[0045] Comparative learning In the contrastive learning module, given a set of instances This application constructs an enhanced view for instance-level contrastive learning. It builds contrastive positive instances by concatenating different cue templates for each instance. (Regarding (e) s , e o Taking instance s as an example, its original view is shown in formula (2), and its enhanced view is: (6); Among them, T´(e s e o () is another manually defined template: (7); By providing a set of instances By adding different prompt templates, two views representing the instance can be obtained using formula (3). and .

[0046] The initial instance-level contrastive learning defined the contrastive learning loss by using a cross-entropy objective function with temperature scaling of in-batch negative samples: (8); Where τ is the adjustable temperature parameter, and B is the batch size. It is cosine similarity. Unlike the original contrastive learning method, the selective contrastive learning in this application not only considers the label signal, but also selectively assigns different weights to negative samples.

[0047] Specifically, for anchor representation r i This application only provides its corresponding enhanced view. As positive contrast instances, all instances with different relations in the augmentation set are used as negative contrast samples, denoted as... From a mathematical perspective, this application measures the importance of a negative contrast instance by calculating the distance between the negative contrast instance and the anchor point; the smaller the distance, the more difficult it is to represent a negative contrast instance. Therefore, this application defines the weight of a contrast instance as: (9); in yes and The Euclidean distance between them. After obtaining the weights, the selective contrast loss is defined as: (10); Finally, the ZSRE model is optimized, especially for the encoder with joint loss: (11); Here, λ1 and λ2 are two weights that constitute different losses.

[0048] Relational Reasoning During the testing phase, the input will be an instance of an unknown relation. The data is fed into the trained encoder to generate their representations. The K-Means algorithm is used to divide the representation into m groups, where m is the number of unknown relations, denoted as Cluster( → {C1,... , C m Each cluster C i It includes a set of test samples. Since the relation names for each cluster are unknown, this application will use manual methods or ChatGPT to summarize the relations from typical samples. and assign it to cluster C i In the implementation process, this application selects the sample closest to each cluster center as the typical sample. i The test samples will be assigned as prediction relationships. .

[0049] Visualization in two-dimensional space revealed numerous false positive samples distributed along the edges of clusters. These false positives are considered out-of-distribution (OOD) samples for the target predicted relation. Therefore, detecting these OOD samples can significantly improve the performance of the ZSRE model.

[0050] Next, this application proposes a false positive detection method based on OOD boundaries. Specifically, for an instance in a cluster, its distance to the center point is calculated, and a simple threshold criterion is used to determine whether it is an OOD. For the i-th cluster... , ,..., ,in Cluster centers are represented as the average of the normalized instance embedding vectors: (12); The Euclidean distance from an instance to its centroid can be calculated as follows: (13); The decision function for OOD samples is: (14); In the formula, δ is the OOD boundary threshold, and 1 / {·} is the index function. δ is set manually. By detecting OOD samples through formula (14), the relational reasoning module proposed in this application can reduce the risk of false alarms.

[0051] Based on the extracted entity relationships, a knowledge graph is constructed, and the steps are as follows: Knowledge fusion: Merging identical entities and relationships from different sources to ensure data consistency; handling ambiguities in entities and relationships, such as distinguishing between entities with the same name but different identities; cleaning and normalizing data to ensure data quality; Knowledge representation: Entities and relations are represented as a graph structure, where nodes represent entities and edges represent relations. RDF (Resource Description Framework) or graph databases are used to store and represent knowledge. Knowledge reasoning: Using techniques such as logical reasoning and graph reasoning to infer new knowledge and relationships. This helps to improve knowledge graphs and enhance their practicality.

[0052] The constructed knowledge graph provides query and application capabilities: it offers a query interface that allows users to search the knowledge graph. The knowledge graph can be applied to real-world business scenarios, such as recommendation, question answering, and semantic search.

[0053] The methods for constructing knowledge graphs described above are existing technologies in this field, and will not be elaborated upon further in this application.

[0054] The experimental setup is described below, followed by the experimental results and an in-depth analysis.

[0055] This application was tested on the following benchmark relation extraction datasets: The Human Metabolome Database (HMDB) is a comprehensive database of human metabolomics established by the The Metabolomics Innovation Centre (TMIC) of Canada. HMDB 5.0 currently contains 217,920 compounds. In addition to these annotated metabolites, HMDB also maintains a set of 1,581,537 unannotated derived compounds and 220,945 metabolite entries, including water-soluble and lipid-soluble metabolites. We randomly selected 40 relations as known relations for training, and simultaneously selected m relations as unknown relations for testing from the remaining 40 relations, where m varies in the range {20, 30, 35, 40}. When using 20 relations for testing, the remaining 20 relations can be used as a validation set.

[0056] DrugBank is a bioinformatics and cheminformatics database provided by the University of Alberta. The latest version of DrugBank Online (version 5.1.10, released on January 4, 2023) contains 16,558 drug entries, including 2,758 approved small molecule drugs, 1,607 approved biologics (proteins, peptides, vaccines, and allergens), 135 or more nutritional supplements, and 6,723 investigational (discovery phase) drugs. In addition, 5,301 non-redundant protein sequences (i.e., drug targets / enzymes / transporters / carriers) are associated with these drug entries. To assist ZSRE, this application randomly selected all samples of 73 relations as training data and used the remaining 40 relations for testing. This application evaluated the ZSRE model under different conditions by varying the number of unknown relations in {20, 30, 35, 40}.

[0057] CTD is a publicly available database for studying associations between chemicals, genes, phenotypes, diseases, and the environment. The CTD database contains a wide range of data, including over 2.3 million chemical drugs, 46,689 genes, 4,340 phenotypes, and 7,212 diseases, encompassing gene and chemical phenotypes, drug-related diseases, gene-related diseases, and drug interactions. To reduce the impact of sample imbalance on test performance, we limit the number of instances per relation to 1,000. We use 20 randomized relations for training and the remaining 20 for testing. For CTD, this application modifies the number of unknown relations m in {15,20}.

[0058] In this study, we utilized data-driven ZSRE to help discover new relationships and obtain annotation data. To convert pseudo-labels predicted by clustering into true relationship labels, we selected representative samples near the cluster centroids and generated relationship names using manual or chatgpt-based annotation methods. By setting the number of representative samples to 1, we reduced the overall workload to a level similar to or even less than previous methods.

[0059] For evaluation metrics, we use the widely adopted F1 score and accuracy to assess the performance of the relation extraction model. Furthermore, since this application is based on a clustering approach, we also use the B3F1 score and normalized mutual information (NMI) to evaluate the effectiveness of the model's clustering.

[0060] In the experiments, this application implements SCL and competing methods based on the Transformers package, using a base version of a pre-trained BERT model as their base encoder. For the HMDB and DrugBank datasets, the maximum sentence length is set to 120, while for the CTD dataset, it is set to 200. The training period is set to 4, utilizing the Adam optimizer with a batch size of 64. The learning rate is set to 1e-5, and the weight decay is 0.1. The scaling temperature τ of the contrastive loss is set to 0.05, and λ1 and λ2 are set to 1 and 0.2 for all datasets.

[0061] In our experiments, we compared our proposed method with various strong baselines. Several methods transformed the Zero-Shot Relation Extraction (ZSRE) task into other task formats. These methods include the CIM model based on text meaning and the QARE model based on reading comprehension. Furthermore, two representation-based methods, MTB and RCL, were also included in the comparative analysis. These methods employ a clustering strategy to predict relations, obtaining discriminative representations of relations. Meanwhile, ZS-BERT, which predicts target relations by identifying the description closest to a given instance, was also selected as a competitor. Finally, we introduced prompt-based techniques as competing methods that fully utilize the internal knowledge within the ZSRE pre-trained language model. Specifically, these techniques include RelationPrompt and MultiPrompt.

[0062] Research question: Does SCL outperform other competing methods in zero-sample relation extraction? To answer this question, the overall results of SCL and the competing method on three datasets are presented in Tables 1, 2, and 3, where the models are tested with different numbers of unknown relations. Higher evaluation metrics correspond to better model performance. The best model performance is indicated in bold, and the results of the model proposed in this application are indicated in gray shading. △ is considered statistically significant as p < 0.05.

[0063] The results in Tables 1, 2, and 3 show that: (1) CIM, QARE, and ZS-BERT perform poorly because the differences in task representation hinder the relation extraction model from obtaining semantic representations of information. The SCL method proposed in this application consistently outperforms all the compared methods under different conditions.

[0064] Table 1: Performance of the HMDB dataset (%). △ indicates a statistically significant improvement;

[0065] Table 2: Performance (%) on the DrugBank dataset. △ indicates a significant statistical improvement;

[0066] Table 3: Performance of the CTD dataset (%). △ indicates a statistically significant improvement;

[0067] This application re-examines the task of zero-shot relation extraction and proposes a novel training method, SCL, to transfer relation knowledge learned from known relations to unknown relations. This application formally defines a three-step paradigm for data-driven relation extraction under zero-shot conditions, including encoder training, relation clustering, and summarization. The key to zero-shot relation extraction in this paradigm lies in training an encoder that can transform knowledge of known relations into the separation of unknown relations. To train a powerful encoder, this application designs a cue-based selective contrastive learning method, where the importance of different negative samples is emphasized by removing spurious negative samples and assigning different weights. In the testing phase, the encoded test samples are clustered. To convert the dummy labels of the clustering results into relation labels, relations are determined by selecting representative samples and summarizing relation descriptions from them.

[0068] As used herein, the term "preferred" is meant as an example, illustration, or illustration. Any aspect or design described herein as "preferred" need not be construed as being more advantageous than other aspects or designs. Rather, the use of the term "preferred" is intended to present the concept in a specific manner. As used in this application, the term "or" is intended to mean an inclusive "or" rather than an exclusionary "or." That is, unless otherwise specified or clear from the context, "X uses A or B" naturally includes either of the permutations. That is, if X uses A; X uses B; or X uses both A and B, then "X uses A or B" is satisfied in any of the foregoing examples.

[0069] Furthermore, although this disclosure has been shown and described with respect to one or more implementations, equivalent variations and modifications will occur to those skilled in the art based on a reading and understanding of this specification and the accompanying drawings. This disclosure includes all such modifications and variations and is limited only by the scope of the appended claims. In particular, with respect to the various functions performed by the aforementioned components (e.g., elements, etc.), the terminology used to describe such components is intended to correspond to any component (unless otherwise indicated) that performs the specified function of said component (e.g., is functionally equivalent to it), even if structurally not equivalent to the disclosed structure performing the functions in the exemplary implementations of this disclosure shown herein. Moreover, although specific features of this disclosure have been disclosed with respect to only one of several implementations, such features may be combined with one or more features of other implementations that may be desirable and advantageous for a given or particular application. Furthermore, with regard to the use of the terms “comprising,” “having,” “containing,” or variations thereof in the Detailed Description or claims, such terms are intended to be included in a manner similar to the term “including.”

[0070] The functional units in this invention embodiment can be integrated into a processing module, or each unit can exist physically separately, or multiple units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. The aforementioned devices or systems can execute the storage methods in the corresponding method embodiments.

[0071] In summary, the above embodiments are one implementation of the present invention, but the implementation of the present invention is not limited to the embodiments described above. Any changes, modifications, substitutions, combinations, or simplifications made that deviate from the spirit and principle of the present invention should be considered equivalent substitutions and are included within the protection scope of the present invention.

Claims

1. A data-driven zero-shot relation extraction method based on selective contrastive learning, characterized in that, Includes the following steps: Acquire medical drug data, which includes multiple entities, such as drug names or disease names. Label the relationships between drug names or disease names in the medical drug data to obtain a set of labeled relationship tags. During training, a pre-trained language model is used as the basic relation encoder; instances of entities are augmented with two different cue templates and then input into the language model to generate instance representations. The representation was then subjected to cue-based optimization and comparative learning; The prompt optimization is used to find the most suitable labels to fill the masking positions in the prompt template, and the contrastive learning is used to find the correspondence of each instance; during the prompt optimization process, a set of additional labels is used to expand the language model to represent the seen relations; in order to highlight the role of hard-to-contrast sample pairs, these hard-to-contrast sample pairs are dynamically emphasized to improve the ability to distinguish relations in the representation space. Further relational reasoning is performed by using a well-learned relational encoder to generate instance representations of unseen relations; The example illustrates the process of dividing the data into multiple clusters using the K-Means algorithm. For each cluster, the k samples closest to the center point are selected and aggregated to identify surfacing relationships that can be used as predicted labels. If false positives are distributed on the boundaries of each cluster, a new cluster boundary is assigned to the target relation to exclude false positives. Based on the extracted relationships between entities, a medical knowledge graph is constructed; semantic search is then performed based on the constructed medical knowledge graph to obtain recommended medical encyclopedia knowledge. The zero-sample relation extraction task refers to: given a reference to a pair of entities (e.g., ... s e o The text of the relation, also known as instances s, aims to identify the main entity e based on contextual clues. s and customer entity e o The semantic relationship r between e is represented by the symbol t. s and e o The entity tuple; Add template T1(·) to the data sample to prompt instance s, where the prompt template contains a [MASK] tag. The prompt template defined in the model is: Among them, e s and e o These are the head and tail of the entity, respectively; 's' are loaded into the template sequentially, and then a prompt input is generated: ， Will (e) s e o Inputting a pre-trained language model, we obtain the hidden vector corresponding to the [MASK] position: ， Simplify the notation to r = h [MASK] It represents the instance relation representation used for cueing learning and contrastive learning.

2. The data-driven zero-shot relation extraction method based on selective contrastive learning according to claim 1, characterized in that, During the optimization process, the language model's task is to predict which word is suitable to fill the [MASK] position for relation extraction; in order to extract the relation based on the relation representation r = h [MASK] To predict the target relation, we extend Ԑ with a set of learnable dummy relation labels V to fully represent the corresponding known relation R. s That is, using each virtual tag v i ∈V describes the relation r i ∈R s The implicit semantics, rather than a conventional mapper that maps a relation label to a label word in the vocabulary; To fully utilize relation r i The semantic information in the relation name is used to initialize the virtual word vector; Using the MLM head layer in a language model, [MASK] tags are recovered from the relation tag set based on inner product similarity: The MLM head layer corresponds to the relation classifier g(·); Obtain the training set The language model is optimized by minimizing the cross-entropy loss. r i It is the label of the i-th relation in the relation set.

3. The data-driven zero-shot relation extraction method based on selective contrastive learning according to claim 2, characterized in that, In the comparative learning process, given a set of examples This builds an enhanced view for instance-level contrastive learning; constructs contrastive positive instances by concatenating different cue templates for each instance; for (e s , e o An instance s of ) has the following enhanced view: ， Among them, T´(e s e o ) is another defined template: ， By providing a set of instances Add different prompt templates to obtain two views representing the instance using the following formula. and 。 4. The data-driven zero-shot relation extraction method based on selective contrastive learning according to claim 3, characterized in that, For anchor characterization Only its corresponding enhanced view As a positive contrast example; All instances with different relations in the augmentation set are used as contrast negative samples, denoted as... From a mathematical perspective, the importance of a negative contrast instance is measured by calculating the distance between it and the anchor point; the smaller the distance, the more difficult it is to represent a negative contrast instance. Therefore, the weight of a contrast instance is defined as: in yes and The Euclidean distance between them.

5. The data-driven zero-shot relation extraction method based on selective contrastive learning according to claim 4, characterized in that, In contrastive learning, selective contrastive loss is defined as: τ is the adjustable temperature parameter, and N is the batch size. It is the cosine similarity, where r is the relational representation vector of the samples. It is the representation vector of the enhanced view; Optimize the encoder with joint loss: Here, λ1 and λ2 are two weights that constitute different losses.

6. The data-driven zero-shot relation extraction method based on selective contrastive learning according to claim 5, characterized in that, During the testing phase, the input will be an instance of an unknown relation. The data is fed into the trained encoder to generate their representations. The K-Means algorithm is used to divide the representation into m groups, where m is the number of unknown relations, and denoted as Cluster(m). → {C1,... , C m }; Each cluster C i Includes a set of test samples; Summarize the relationships from typical samples. and assign it to cluster C i middle; During implementation, the sample closest to each cluster centroid is selected as the typical sample; C i The test samples will be assigned as prediction relationships. .

7. The data-driven zero-shot relation extraction method based on selective contrastive learning according to claim 6, characterized in that, False positive samples exist and are distributed at the edges of clusters. For the target predicted relation, these false positive samples are considered out-of-place (OOD) samples. During relation inference, for an instance in the cluster, its distance to the center point is calculated, and a threshold criterion is used to detect whether it is an OOD sample. For the i-th cluster , ,..., ,in ||r|| represents the L2 norm value of the relation representation vector, and the cluster center is represented as the average of all normalized instance embedding vectors: The Euclidean distance from an instance to its centroid is calculated as follows: The decision function for OOD samples is: In the formula, δ is the OOD boundary threshold, 1{·} is the index function, and δ is the default setting.

Images