Small-sample relation extraction method based on difficult sample comparative learning
Through core word retention enhancement and difficult sample comparative learning, the problem of insufficient model generalization ability when training data is scarce is solved, and the model's relationship extraction performance under extremely few sample conditions is improved, especially the ability to recognize difficult samples.
Patent Information
- Application Number
- CN202510873920.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-27
- Publication Date
- 2025-10-17
AI Technical Summary
Existing technologies are unable to fully capture relational semantic features when training data is scarce, which limits the model's generalization ability. In addition, they lack the ability to mine difficult samples, resulting in limited fine-grained semantic discrimination capabilities of the model.
The core word retention enhancement method is used to generate new samples, and through contrastive learning of difficult samples, difficult samples are dynamically identified for contrastive learning. Combined with the injection of virtual label words and entity type words, the weighted average optimized cross entropy loss and contrastive loss function are used for joint training.
The model's generalization ability and fine-grained relationship discrimination performance under conditions of very few samples are significantly improved, and its ability to distinguish difficult samples is enhanced.
Smart Images
Figure CN120804286A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of information extraction, and particularly relates to a few-shot relation extraction method based on difficult sample contrast learning. BACKGROUND
[0002] Relation extraction (RE) is one of the core tasks in the field of natural language processing (NLP), and its core goal is to accurately identify and extract the specific semantic relationship between entity pairs through semantic analysis of unstructured text. As a key role in the information extraction system, this technology is widely used in knowledge graph construction, intelligent question answering systems, semantic search and other important scenarios. Traditional supervised learning-based RE models, such as convolutional neural networks (CNN), recurrent neural networks (RNN), and BERT deep neural network architectures, have shown superior performance in benchmark tests, but their performance is closely related to the size of the labeled data, and usually requires a large amount of labeled samples to achieve ideal results. However, in actual industrial application scenarios, especially in biomedical, financial regulations and other vertical fields, there are many difficulties in obtaining high-quality labeled data. In view of this, recent research has focused on the emerging field of few-shot relation extraction (FSRE), aiming to optimize few-shot learning mechanisms to achieve accurate modeling and extraction of relationship semantics under the constraint of limited labeled resources.
[0003] Current frontier research achieves FSRE through a variety of innovative methods: prompt learning-based methods effectively activate the relationship extraction capabilities of pre-trained language models (PLMs) by restructuring tasks; generative data augmentation techniques alleviate data sparsity problems through semantic space expansion strategies; and knowledge injection methods significantly improve the model's representation ability for low-frequency relationships by incorporating knowledge graphs, relationship labels and other semantic knowledge. The synergistic innovation of these technologies has driven the development of FSRE.
[0004] Although knowledge injection-based few-shot relation extraction methods have been widely used in this field, there are still several key problems to be solved. First, under the condition of scarce training data, existing models often have difficulty fully capturing relationship semantic features, resulting in significant constraints on their generalization performance. In addition, current research generally ignores the mining of difficult samples. Such samples are often distributed near the class decision boundary, and if their representation rules can be deeply mined, the model's ability to distinguish fine-grained semantic differences will be effectively improved. SUMMARY
[0005] The technical problems to be solved by the present application are: how to solve the problems that the prior art is difficult to fully capture the relationship semantic features when the training data is scarce, thereby limiting the model generalization ability, and the prior art lacks mining of difficult samples, thereby limiting the model fine-grained semantic discrimination ability, and provide a few-shot relationship extraction method based on difficult sample contrast learning, which can significantly improve the generalization ability of the model under the condition of very few samples, and effectively enhance the discrimination ability of the model to difficult sample relationship.
[0006] The present application solves the above technical problems by the following technical solutions, the present application comprises the following steps:
[0007] S1: input training data;
[0008] S2: evaluate the number of each relationship type in the training set of the training data, if any relationship type contains only one sample, perform core word retention enhancement processing, generate new samples and add them to the training set, if all relationship types contain multiple samples, directly enter step S3;
[0009] S3: combine the label information in the training data and the pre-trained language model to respectively construct virtual label words and virtual entity type words, and complete the initialization of the virtual label words and the virtual entity type words;
[0010] S4: inject the virtual label words and the virtual entity type words into the original text of the training set samples;
[0011] S5: take the samples in the current batch as anchor points, and take the training set as a sample candidate pool, according to the cosine similarity principle, screen the samples that are easy to be misjudged, respectively construct a difficult positive sample set and a difficult negative sample set, and perform contrast learning;
[0012] S6: use the cross-entropy loss L TaiLr and the contrast loss function L HardCon to jointly train the pre-trained language model, and then obtain the trained relationship extraction model;
[0013] S7: input the text to be extracted and the corresponding head and tail entity information into the trained relationship extraction model to obtain the relationship extraction result.
[0014] Further, in the step S1, the training data contains an independent relationship set R T , a training set D T , and no relationship type in the training set D T does not appear in the relationship set R T , each sample in the training set D T is saved in the form of a dictionary and contains text, head entity information, tail entity information and relationship label.
[0015] Further, in the step S2, the specific process of core word retention enhancement processing is as follows:
[0016] S21: Calculate the product of term frequency and inverse document frequency by using TF-IDF value, so as to extract the most important M core words from each sample in the training set D T and store the M core words in the core word set t Key , the formula is as follows:
[0017] t Key = Top M [TF-IDF(t)]
[0018] Wherein, TF-IDF(t) is used to measure the importance of word t in the current sample, the larger the value, the more important the word is in the current sample; TF(t) is the term frequency, which represents the frequency of the word t appearing in the current sample; IDF(t) is used to measure the rarity of the word t in the training set D T ;
[0019] S22: Take the core word set t Key as the stop word list, use WordNet dictionary as the synonym library, replace the non-stop words in the original sample of the training set with their synonyms, so as to generate new samples, add the new samples to the original training set D T , form a new training set D New .
[0020] Further, in the step S3, the specific processing process is as follows:
[0021] S31: Use the pre-trained language model to predict the artificially designed prompt to generate the initialization representation Split the relationship labels in the relationship set R T into semantic phrases, calculate the probability distribution of each phrase combined with term frequency statistics, apply weighted average function to the semantic phrases, obtain the initialization representation Combine and to complete the initialization of virtual relationship words ;
[0022] S32: Generate the abstract representation of the training set in the debiased field obtained in step S2 by using Gaussian mixture model and variational autoencoder generative adversarial network algorithm, then uniformly sample the abstract representation in the debiased field, extract the abstract representation ω, and take it as the initialization representation of the virtual entity type word .
[0023] Further, in the step S31, the specific processing process is as follows:
[0024] S311: For each relation label, use the artificially designed prompt template to predict the initialization representation by the pre-trained language model
[0025] S312: Split the label words in the data set into semantic phrases And through the word frequency calculation to obtain the probability distribution of S r Apply the weighted average function to each word in the semantic phrase S r , and obtain the initialization representation
[0026] S313: Take the average of And Complete the initialization of the virtual relation word .
[0027] Further, in the step S32, the specific processing process is as follows:
[0028] S321: Use Gaussian mixture model to fit the distribution of the training set, and take the distribution as the target distribution;
[0029] S322: Through the variational autoencoder generative adversarial network algorithm, the encoding of the original text of the sample in the training set is gradually migrated to the target distribution, so as to obtain the abstract representation of the de-biased field;
[0030] S323: Uniformly sample from the abstract representation of the de-biased field, extract the abstract representation ω, and take it as the initialization representation of the virtual entity type word .
[0031] Further, in the step S4, the specific processing process is as follows:
[0032] S41: Combine prompt learning, inject the initialized virtual entity type word into the prompt, and then concatenate it with the original text of the sample in the training set processed in step S2;
[0033] S42: Take the virtual label word As an additional learnable relation embedding in the MLM head layer of the pre-trained language model, and then realize knowledge injection.
[0034] Further, in the step S5, the specific processing process is as follows:
[0035] S51: Initialize the candidate sample pool, and store the samples in the training set processed in step S2 into the candidate sample pool;
[0036] S52: Take each batch of samples as an anchor point a, and filter out difficult positive samples p and difficult negative samples n prone to misjudgment from the candidate sample pool based on cosine similarity;
[0037] S53: Take the similarity between the difficult positive samples p, the difficult negative samples n and the anchor point into the contrast loss function L HardCon for calculation.
[0038] Further, in the step S52, the specific processing process is as follows:
[0039] S521: According to the relationship label of the sample, add the candidate samples with the same relationship label as the anchor point a to the candidate positive sample set p={p1, p2,..., p i}; and add the candidate samples with different relationship labels from the anchor point a to the candidate negative sample set n={n1, n2,..., n i};
[0040] S522: Calculate the cosine similarity between the anchor point a and all candidate positive and negative samples in the candidate sample pool;
[0041] S523: In the candidate positive sample set, compare the similarity of each candidate positive sample p={p1, p2,..., p i} with the anchor point a, select the sample with the smallest similarity as the difficult positive sample, and record its similarity value sim + ;
[0042] S524: In the candidate negative sample set, compare the similarity of each candidate negative sample set n={n1, n2,..., n i} with the anchor point a; select the top N candidate negative samples with the highest similarity as the difficult negative sample set of the anchor point a, and take the cosine similarity as the difficult negative sample similarity set sim - .
[0043] Further, in the step S6, the loss function L FSRE during joint training is as follows:
[0044] L FSRE = aL TaiLr + (1-a)L HardCon
[0045] Wherein, a e [0, 1], is the weight hyperparameter of joint training, and the cross-entropy loss L TaiLr optimized by weighted average is as follows:
[0046]
[0047] wherein, gamma is in [0, 1], when gamma = 0, the loss function is the traditional cross-entropy loss; when gamma = 1, the result is -p after limit calculation θ (y|x), -p θ (y|x) represents the probability of the pre-trained language model to predict the true relationship label y under the condition of the given input x, and theta represents the weight parameters of all layers of the pre-trained language model.
[0048] The calculation formula of the contrast loss function L HardCon is as follows:
[0049]
[0050] wherein, B is the total number of samples in a batch, and tau is a temperature hyperparameter.
[0051] Compared with the prior art, the present application has the following advantages:
[0052] 1. The present application proposes a core word reservation enhancement method applied to few-shot samples to solve the problem that the prior art is difficult to fully capture relationship semantic features when the training data is scarce, thereby limiting the model generalization ability. The method extracts the core words in the sample and randomly replaces the non-core words to generate new samples that reserve the core words, thereby effectively improving the generalization ability of the model under the condition of few-shot samples.
[0053] 2. The present application proposes a difficult sample contrast learning method to solve the problem that the prior art lacks mining of difficult samples, resulting in limited model fine-grained semantic discrimination ability. The method can dynamically identify difficult samples from limited training samples and construct challenging positive and negative sample pairs for contrast learning, thereby significantly enhancing the fine-grained relationship discrimination performance of the model in a low-resource scenario. BRIEF DESCRIPTION OF DRAWINGS
[0054] Figure 1 is the implementation flowchart of the few-shot relationship extraction method based on difficult sample contrast learning in the embodiment of the present application;
[0055] Figure 2 is the overall framework diagram of the few-shot relationship extraction method based on difficult sample contrast learning in the embodiment of the present application. DETAILED DESCRIPTION
[0056] The embodiments of the present application will be described in detail below. The present embodiments are implemented on the premise of the technical solutions of the present application, and detailed implementation methods and specific operation processes are given, but the protection scope of the present application is not limited to the following embodiments.
[0057] As Figure 1As shown, this embodiment provides a technical solution: a method for extracting relations from a small number of samples based on contrastive learning of difficult samples, comprising the following steps:
[0058] S1: input training data;
[0059] In this embodiment, in S1, the training data includes an independent relation set R T , training set D T . Requires training set D T Does not exist and does not appear in the relation set R T At the same time, the relationship set R T It should contain a mapping of each relationship label to a unique ID, such as: {"org:founded":0,"org:subsidiaries":1",per:date_of_birth":2,"per:cause_of_death":3}. In addition, the training set D T Each sample in the dictionary is saved in the form of a dictionary, which contains a text token, a head entity information h (entity name and position pos), a tail entity information t (entity name and position pos) and a relation label relation, such as: {"token":["Pandit","worked","at","the","brokerage","Morgan","Stanley","for","about","11","years","until","2005",",","when","he","and","some","Morgan","Stanley","colleagues","quit","and","later","founded","the","hedge","fund","Old","Lane","Partners","."],"h":{"name":"Old Lane Partners","pos":[28,31]},"t":{"name":"2005","pos":[12,13]},"relation":"org:founded"}.
[0060] S2: Evaluation on training set D T If a relation type contains only one sample, the core word retention enhancement method is started to generate a new sample and add it to the training set D T ;
[0061] S2-1: If Figure 2 As shown, in the core word retention enhancement stage, the training set D is evaluated Tthe number of training samples k corresponding to each relation type. If a relation type contains only one sample, the core word reservation enhancement method is enabled, a new sample is generated and added to the training set D T ; if the number of samples exceeds one, the method is not enabled, and the virtual word initialization phase is directly entered.
[0062] S2-2: The product of term frequency and inverse document frequency is calculated by using TF-IDF value, so that the most important M core words t T are extracted from each sample d in the training set D Key = {t1, t2, …, t M}, the formula is as follows:
[0063]
[0064] TF-IDF(t) = TF(t) x IDF(t)
[0065] t Key = Top M [TF-IDF(t)]
[0066] Wherein, TF-IDF(t) is used to measure the importance of word t in the current sample, the larger the value, the more important the word is in the current sample; TF(t) represents the term frequency, which indicates the frequency of the word t appearing in the current sample; IDF(t) measures the rarity of the word t in the entire training set; r num represents the total number of relation sets R T , and k represents the number of samples corresponding to each relation label in the training set D T .
[0067] S2-3: The core word set t Key determined before is used as a stop word list, the original sample is replaced with its synonyms by using WordNet dictionary as a synonym library, so as to generate new samples d' similar to the original sample but slightly different. These new samples are added to the original training set D T , forming a new training set D New , the formula is as follows:
[0068]
[0069] Wherein, d i ' represents a new sample obtained by synonym replacement of the i th original sample, represents all generated new samples, r num represents the total number of relation labels of the training set D T , and k represents the number of samples corresponding to each relation label in the training set (at this time, k = 1).
[0070] S3: Combine the label information in the training set with the knowledge contained in the pre-trained language model to construct virtual label words and virtual entity type words respectively;
[0071] S3-1: Use the pre-trained language model (PLM) to predict the artificially designed prompts to extract the potential knowledge contained in the PLM and generate the initial semantic representation split the relationship labels in the relationship set R T into semantic phrases Combine the word frequency statistics to calculate the probability distribution φ r of each phrase. On this basis, apply a weighted average function to these semantic phrases to obtain the initial representation containing label knowledge Combine the results of and to complete the initialization of the virtual relationship words .
[0072] S3-1-1: For each relationship label, design an artificial prompt template and use a pre-trained language model (PLM) to predict to obtain an initial representation containing the potential knowledge of the PLM
[0073]
[0074] where hand_prompt represents an artificially designed prompt template, which is formed by splitting the relationship label, for example, the relationship label “per:country_of_death” can be split into “person”, “country”, “die”, forming the prompt template “head entity died in the country tail entity.” By using [MASK] mask in the template, the PLM can generate a preliminary representation for the relationship label, thereby obtaining an initial representation containing the knowledge of the PLM
[0075] S3-1-2: In order to reduce irrelevant labels generated in the prediction process, split the relationship labels in the relationship set R T into semantic phrases and calculate the probability distribution of S r by word frequency Apply a weighted average function to each word in S r to obtain an initial representation containing label knowledge The formula is as follows:
[0076]
[0077] where S rS3-1-1: Split the relation label, for example: the relation label "per:country_of_death", "per:country_of_birth" and "per:date_of_death" can be split into S r = {"country", "of", "death", "birth", "date"} respectively, and the corresponding probability distribution φ r = {2 / 9, 3 / 9, 2 / 9, 1 / 9, 1 / 9}.
[0078] S3-1-3: Take the average of and to complete the initialization of the virtual relation word , the formula is as follows:
[0079]
[0080] wherein mean represents the average processing.
[0081] S3-2: Obtain the abstract representation of the training set in the unbiased field by Gaussian Mixture Model (GMM) and Variational Autoencoder Generative Adversarial Network (SVGD) algorithm. Then, uniformly sample the above abstract representation, extract the abstract representation ω, and take it as the initialization representation of the virtual entity type word .
[0082] S3-2-1: Perform simple data processing on each sample in the training set D T (if k = 1 is D New ), that is, extract and process the keys "token" and "relation" of the dictionary (training set) D T ; merge the values of the key "token" into text as the original text of the i-th sample of the training set The value of the key "relation" is the true label y i of the i-th sample of the training set.
[0083] S3-2-2: Encode and obtain the hidden layer representation feature matrix H, the formula is as follows:
[0084]
[0085] wherein E(·) represents the encoder.
[0086] S3-2-3: Initialize GMM, set the number of Gaussian components to r num , and use the K-means++ algorithm to initialize the component parameters; thus obtaining the formula is as follows:
[0087] Pμ ,P σ ,P π =GMM(H,r num )
[0088] where P μ ,P σ and P π represent the mean vector, covariance matrix and weight of each Gaussian component, respectively;
[0089] S3-2-4: input the Gaussian mixture distribution determined by P μ ,P σ ,P π into the SVGD algorithm, taking the distribution as the target distribution, and the encoding h i of the original text is regarded as the initial particle θ i ;
[0090] S3-2-5: using the gradient descent method of minimizing the KL divergence, the SVGD algorithm will iteratively adjust the initial particle θ i to match the target distribution, and thus obtain the final particle θ i ';
[0091] S3-2-6: uniformly sample from the set of final particles, extract the abstract representation ω, and take it as the initialized representation of the virtual entity type word , as follows:
[0092]
[0093] S4: as shown in Figure 2 , in the knowledge injection stage, the virtual label word and the virtual entity type word containing the PLM knowledge and the label knowledge are injected into the original text (which is the token part in the sample processed by S2);
[0094] S4-1: combine prompt learning, inject the initialized virtual entity type word into the prompt, and then concatenate it with the original text to obtain the knowledge-containing input x i , as follows:
[0095]
[0096] prompt=[sub]head entity[sub][MASK][obj]tail entity[obj]
[0097] x=[CLS]x ori [SEP]prompt[SEP]
[0098] Wherein, the head entity and the tail entity represent the head entity and the tail entity of the sample respectively.
[0099] S4-2: the virtual label word As an additional learnable relationship embedding in the MLM head layer of the PLM, to fully represent the corresponding relationship label.
[0100] S5: taking the samples in the current batch as anchor points, taking the entire training set D T As a sample candidate pool, according to the cosine similarity principle, filter the samples prone to misjudgment, respectively construct the difficult positive sample set and the difficult negative sample set, and use these difficult samples for contrast learning;
[0101] S5-1: initialize the candidate sample pool, take the entire training set D T (If k = 1 is D New ) into it, to provide a wider range of choices for difficult sample screening.
[0102] S5-2: taking the samples in each batch as anchor points a, based on the cosine similarity, filter the difficult positive samples p and the difficult negative samples n prone to misjudgment from the candidate sample pool.
[0103] S5-2-1: initialize the candidate positive sample set and the candidate negative sample set. According to the sample label, add the candidate samples with the same relationship label as the anchor point a to the candidate positive sample set p = {p1, p2,..., p i}; add the candidate samples with different relationship labels from the anchor point a to the candidate negative sample set n = {n1, n2,..., n i}.
[0104] S5-2-2: calculate the cosine similarity between the anchor point a and all candidate positive and negative samples in the candidate pool, the formula is as follows:
[0105]
[0106] Wherein, b is the candidate positive and negative sample.
[0107] S5-2-3: in the candidate positive sample set, compare the similarity of each candidate positive sample p = {p1, p2,..., p i} and the anchor point a. Select the sample with the smallest similarity as the difficult positive sample, and record its similarity value sim + . Considering the special case that there may be some relationship types in the training set only one training sample, at this time only one difficult positive sample is selected. The formula is as follows:
[0108] sim + = min[sim(a, p i )]
[0109] where p i represents the i-th candidate positive sample of anchor point a, sim(x, p i ) represents the cosine similarity between anchor point a and the i-th candidate positive sample.
[0110] S5-2-4: In the candidate negative sample set, compare the similarity of each candidate negative sample set n = {n1, n2,..., n i} and anchor point a. Select the top N candidate negative samples with the highest similarity as the difficult negative sample set of anchor point x, and their cosine similarity as the difficult negative sample similarity set sim - , the formula is as follows:
[0111]
[0112] where n i represents the i-th candidate negative sample of anchor point a, sim(a, n i ) represents the cosine similarity between anchor point a and the i-th candidate negative sample.
[0113] S5-3: The similarity between the difficult samples determined in the previous step and the anchor points is included in the contrast loss function L HardCon to calculate, so as to optimize the relationship distribution in the semantic space, the formula is as follows:
[0114]
[0115] where B is the total number of samples in the batch, sim + is the similarity between the j-th anchor point a j and its difficult positive sample, sim i - is the similarity between the j-th anchor point a j and its i-th difficult negative sample, τ is a temperature hyperparameter.
[0116] S6: Through joint training, further improve the quality of virtual prompt words, and enhance the accuracy of relationship prediction.
[0117] S6-1: Use the optimized cross-entropy loss L TaiLr to alleviate the "overconfidence" phenomenon of the model caused by traditional cross-entropy loss, as follows:
[0118]
[0119] where γ ∈ [0, 1], when γ = 0, the loss function is the traditional cross-entropy loss; when γ = 1, the result is -p θ (y|x) after limit calculation; -pθ (y|x) represents the probability that the pre-trained language model predicts the true relation label y given the input x, and θ represents the weight parameters of all layers of the pre-trained language model;
[0120] S6-2: In the joint training stage, the optimized cross-entropy loss L TaiLr and the difficult sample contrastive learning loss L HardCon are jointly optimized to predict the virtual prompt word and the relation, and the formula is as follows:
[0121] L FSRE = aL TaiLr +(1-a)L HardCon
[0122] Wherein, a e [0, 1], is the weight hyperparameter of joint training. When a = 0, the loss function is the difficult sample contrastive learning loss L HardCon ; when a = 1, the loss function is the optimized cross-entropy loss L TaiLr .
[0123] S7: input the text to be extracted and the corresponding head and tail entities into the trained relation extraction model to obtain the relation extraction result.
[0124] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limiting the present application, and those skilled in the art can make changes, modifications, replacements and variations to the above embodiments within the scope of the present application.
Claims
1. A few-sample relation extraction method based on contrastive learning of difficult samples, characterized by: The following steps are involved: S1: input training data; S2: Evaluate the number of each relationship type in the training set of the training data. If any relationship type contains only one sample, perform core word retention enhancement processing, generate new samples and add them to the training set. If all relationship types contain multiple samples, directly proceed to step S3; S3: Combining the label information in the training data with the pre-trained language model, construct virtual label words and virtual entity type words respectively, and complete the initialization of virtual label words and virtual entity type words; S4: Inject virtual label words and virtual entity type words into the original text of the training set samples; S5: Use the samples in the current batch as anchor points and the training set as the sample candidate pool. According to the cosine similarity principle, construct a difficult positive sample set and a difficult negative sample set, and perform comparative learning. S6: Cross entropy loss L optimized using weighted average TaiLr And the contrast loss function L HardCon Jointly train the pre-trained language model to obtain a trained relation extraction model; S7: Input the text of the relationship to be extracted and the corresponding head and tail entity information into the trained relationship extraction model to obtain the relationship extraction result.
2. The method for extracting relations from a small number of samples based on contrastive learning of difficult samples according to claim 1, characterized in that: In step S1, the training data includes an independent relation set R T , training set D T , training set D T does not exist in the relation set R T The relationship type in the training set D T Each sample is saved in the form of a dictionary, which contains text, head entity information, tail entity information and relationship labels.
3. The method for extracting relations from a small number of samples based on contrastive learning of difficult samples according to claim 2, characterized in that: In step S2, the specific process of the core word retention enhancement process is as follows: S21: Use TF-IDF value to calculate the product of word frequency and inverse document frequency, so as to obtain the T Extract the most important M core words from each sample and store them in the core word set t Key In the formula, the following is: t Key =Top M [TF-IDF(t)] Among them, TF-IDF(t) is used to measure the importance of word t in the current sample. The larger the value, the more important the word is in the current sample. TF(t) is the word frequency, which indicates the frequency of word t appearing in the current sample. IDF(t) is used to measure the frequency of word t in the training set D. T Rarity in S22: The core word set t Key As a stop word list, use the WordNet dictionary as a synonym library, replace the non-stop words in the original samples of the training set with their synonyms, thereby generating new samples, and adding the new samples to the original training set D T In the new training set D New .
4. The method for extracting relations from a small number of samples based on contrastive learning of difficult samples according to claim 3, characterized in that: In step S3, the specific processing process is as follows: S31: Use pre-trained language models to predict manually designed prompts and generate initialization representations The relation set R T The relation tags in the sentence are split into semantic phrases, and the probability distribution of each phrase is calculated by combining the word frequency statistics. The weighted average function is applied to the semantic phrases to obtain the initialization representation. Combine and Complete the virtual relative word Initialization; S32: Generate an adversarial network algorithm using a Gaussian mixture model and a variational autoencoder to obtain the abstract representation of the training set obtained in step S2 in the debiased domain, then uniformly sample the abstract representation in the debiased domain, extract the abstract representation ω, and use it as the virtual entity type word Initialization representation of .
5. The method for extracting relations from a small number of samples based on contrastive learning of difficult samples according to claim 4, characterized in that: In step S31, the specific processing process is as follows: S311: For each relationship tag, use the manually designed prompt template and then use the pre-trained language model to predict it and obtain the initialization representation S312: Split the label words in the dataset into semantic phrases And get S by word frequency calculation r The probability distribution of Semantic phrase S r Apply the weighted average function to each word in to obtain the initialization representation S313: Yes and Take the average and complete the virtual relative words Initialization.
6. The method for extracting relations from a small number of samples based on contrastive learning of difficult samples according to claim 5, characterized in that: In step S32, the specific processing process is as follows: S321: Use the Gaussian mixture model to fit the distribution of the training set and use this distribution as the target distribution; S322: Generate an adversarial network algorithm using a variational autoencoder to gradually transfer the encoding of the original text of the samples in the training set to the target distribution, thereby obtaining an abstract representation of the debiased domain; S323: Uniformly sample from the abstract representation of the debiased domain, extract the abstract representation ω, and use it as the virtual entity type word Initialization representation of .
7. The method for extracting relations from a small number of samples based on contrastive learning of difficult samples according to claim 4, characterized in that: In step S4, the specific processing process is as follows: S41: Combined with prompt learning, the initialized virtual entity type words are injected into the prompt, and then concatenated with the original text of the samples in the training set after processing in step S2; S42: Virtual label words As an additional learnable relation embedding in the MLM head layer of the pre-trained language model, knowledge injection is achieved.
8. The method for extracting relations from a small number of samples based on contrastive learning of difficult samples according to claim 7, characterized in that: In step S5, the specific processing process is as follows: S51: Initialize the candidate sample pool and store the samples in the training set processed in step S2 into it; S52: Consider the samples in each batch as anchor points a, and screen out difficult positive samples p and difficult negative samples n from the candidate sample pool based on cosine similarity; S53: Incorporate the similarity between the difficult positive sample p, the difficult negative sample n and the anchor point into the contrast loss function L HardCon Calculation is performed in .
9. The method for extracting relations from a small number of samples based on contrastive learning of difficult samples according to claim 8, characterized in that: In step S52, the specific processing process is as follows: S521: According to the relationship labels of the samples, the candidate samples with the same relationship labels as the anchor point a are added to the candidate positive sample set p = {p1, p2, ..., p i }; Add candidate samples with different relationship labels to the anchor point a into the candidate negative sample set n={n1,n2,...,n i }; S522: Calculate the cosine similarity between the anchor point a and all candidate positive and negative samples in the candidate sample pool; S523: Compare the candidate positive samples p={p1, p2, ..., p i } and the similarity with anchor point a, select the sample with the smallest similarity as the difficult positive sample, and record its similarity value sim + ; S524: Compare the candidate negative sample sets n={n1,n2,...,n i } similarity with anchor point a; select the top N candidate negative samples with the highest similarity as the difficult negative sample set of anchor point a, and use their cosine similarity as the difficult negative sample similarity set sim - .
10. The method for extracting relations from a small number of samples based on contrastive learning of difficult samples according to claim 9, characterized in that: In step S6, the loss function L during joint training is FSRE The details are as follows: L FSRE =αL TaiLr +(1-α)L HardCon Among them, α∈[0,1] is the weight hyperparameter of joint training, and the weighted average optimized cross entropy loss L TaiLr The calculation formula is as follows: Among them, γ∈[0,1], when γ=0, the loss function is the traditional cross entropy loss; when γ=1, after limit calculation, the result is -p θ (y|x), -p θ (y|x) represents the probability of the pre-trained language model predicting the true relation label y given the input x, and θ represents the weight parameters of all layers of the pre-trained language model; Contrastive loss function L HardCon The calculation formula is as follows: Where B is the total number of samples in the batch and τ is a temperature hyperparameter.