Relationship extraction method and device based on comparative learning and differentiated data enhancement

Abstract

The invention discloses a relation extraction network method and device based on comparative learning and differentiated data enhancement, and the method aims at a long tail distribution problem in remote supervision relation extraction, and combines a differentiated enhancement sample generated by a large language model with a multi-level comparative learning mechanism. The extraction effect of the tail relation is improved through four main steps: firstly, dividing a sample based on relation frequency, and generating an enhanced sample by adopting layered prompt and differential sampling; secondly, constructing a sentence-level encoder and a packet-level encoder to carry out multi-level semantic modeling; thirdly, constructing positive and negative sample pairs, and introducing a contrast learning target to improve the representation discrimination capability; and finally, jointly optimizing package-level loss and sentence-level comparison loss, and realizing unified learning of global and local features. According to the method, the overall performance is kept stable while the tail relation recognition capability is improved, the problem of class imbalance in remote supervision relation extraction can be effectively relieved, and the method has high practical value and popularization prospects.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of natural language processing (NLP), in particular to a relation extraction method and device based on contrastive learning and differential data enhancement, and belongs to the technical field of information extraction and semantic understanding. BACKGROUND

[0002] Current distant supervision methods mainly focus on the problem of noisy labels and pay insufficient attention to the class imbalance caused by the long-tail distribution. Although existing research attempts to alleviate the imbalance through resampling or loss weighting, the effect is limited; a few works explore few-shot relation extraction, but still do not fully utilize the large amount of unlabeled data under distant supervision.

[0003] In recent years, large language models such as GPT-4o have shown excellent performance in natural language generation, and if they are used to generate synthetic training sentences for tail relations, it is expected to alleviate the problem of sample scarcity. Compared with traditional rules or word replacement methods, the sentences generated by large language models are more natural and diverse, which can help the model learn more rich relationship expressions.

[0004] However, this strategy also faces challenges: different relationships need to be enhanced differently, the quality and diversity of generated sentences need to be balanced, and semantic drift needs to be avoided. At the same time, simply increasing samples may not be enough to solve the long-tail problem, so the present application introduces contrastive learning, which constructs positive and negative sample pairs to explicitly optimize the representation space, so that the model can better distinguish between relationship categories, especially tail relationships. SUMMARY

[0005] The present application overcomes the above-mentioned shortcomings of the prior art and provides a relation extraction method and device based on contrastive learning and differential data enhancement. The present application studies the distribution characteristics of the long-tail data distribution for the long-tail of the distant supervision data, combines differential data enhancement generated by a large language model and a dual-view contrastive learning framework, and significantly improves the extraction effect of tail relationships without compromising the performance of head relationships.

[0006] In order to achieve the above-mentioned purposes, the technical solution provided by the present application is as follows:

[0007] The first aspect of the present application relates to a relation extraction method based on contrastive learning and differential data enhancement, comprising the following steps:

[0008] Step 1: Differential data enhancement: based on the distribution characteristics of the relationship categories in the training data, a large language model is used to perform high-diversity data enhancement on tail relationship instances and low-disturbance data expansion on main frequency relationship instances;

[0009] Step 2: Multi-level representation learning: a double-level encoding structure is constructed, including a sentence-level encoder and a bag-level encoder, to perform semantic representation on the original and enhanced samples;

[0010] Step 3: Contrastive learning: Form positive sample pairs with original and augmented samples, negative sample pairs with samples from different relations, and difficult negative sample mining mechanism to guide model optimization of semantic representation space.

[0011] Step 4: Training target definition: Joint bag-level relation classification loss and sentence-level contrastive learning loss, combined with language model pre-training target to form a unified optimization target.

[0012] Step 5: Including difficult positive sample priority training mechanism and dynamic negative sample sampling strategy based on semantic distance.

[0013] Step 1 specifically includes:

[0014] For tail relation samples, use high temperature parameters (τ≥1.2) and low core sampling threshold (p≤0.88) to generate diverse augmented samples; For main relation samples, use low temperature (τ≤0.6) and high core sampling threshold (p≥0.98) to generate structure equivalent samples.

[0015] Step 2 specifically includes:

[0016] The sentence-level encoder uses the BERT model for context embedding and concatenates the representations of the head and tail entities in the sentence to form entity-aware semantic vectors. The bag-level encoder weights and fuses the sentence representations in the bag based on the attention mechanism, and obtains the relation classification probability distribution through Softmax.

[0017] Step 3 specifically includes:

[0018] Positive sample pairs are composed of the same original sentence and its augmented sentence, and negative sample pairs are composed of bag representations of different entities or relations.

[0019] Step 4 specifically includes:

[0020] The training target includes three parts: sentence-level supervised contrastive loss, bag-level prototype contrastive loss, and language modeling auxiliary target; The sentence-level loss is a supervised contrastive loss that maximizes the similarity between samples within the same relation category and distinguishes between different relation categories, effectively enhancing intra-class consistency; The bag-level loss is a prototype contrastive loss that defines a learnable prototype vector for each relation, making the bag-level semantic representation closer to the true relation prototype and further away from other relation prototypes; The language modeling auxiliary target includes entity-aware mask language modeling and relation prompt generation, where the model randomly masks entities or relation words in the input sentence and predicts their content to enhance the understanding of semantic roles and relation structures;

[0021] The final training target function is:

[0022]

[0023] wherein, is a weight regulation function varying with the number of training steps, N B represents the number of samples in the current batch, and β and γ are the loss weight hyperparameters of the bag level and language modeling target respectively.

[0024] wherein, step 5 specifically comprises:

[0025] The difficult positive sample mining and dynamic negative sample mechanism are introduced, the sample difficulty distribution function γ(d) is introduced, and the number of difficult samples and the training stability are balanced.

[0026] The second aspect of the present application relates to a relationship extraction device for contrastive learning and differential data enhancement, comprising a memory and one or more processors, the memory has executable code stored therein, and the one or more processors execute the executable code to implement the relationship extraction method for contrastive learning and differential data enhancement of the present application.

[0027] The third aspect of the present application relates to a computer readable storage medium having a program stored thereon, which is executed by a processor to implement the relationship extraction method for contrastive learning and differential data enhancement of the present application.

[0028] The present application aims at the long-tail distribution problem existing in remote supervision relationship extraction, combines the differential enhancement samples generated by the large language model and the multi-level contrastive learning mechanism, and improves the extraction effect of the tail relationship through four main steps: first, the samples are divided based on the relationship frequency, and the hierarchical prompt and differential sampling are used to generate enhanced samples; second, the sentence-level and bag-level encoders are constructed for multi-level semantic modeling; third, the positive and negative sample pairs are constructed and the contrastive learning target is introduced to improve the representation discrimination ability; finally, the bag-level loss and sentence-level contrast loss are jointly optimized to realize the unified learning of global and local features. The present application improves the recognition ability of the tail relationship while keeping the overall performance stable, can effectively alleviate the class imbalance problem in remote supervision relationship extraction, and has strong practical value and popularization prospect.

[0029] The application has the advantages that: for the long-tail distribution problem of remote supervision data in natural language processing, a relationship extraction network based on contrast learning and differential data enhancement is proposed. The method models the relationship frequency, divides the main relationship and the tail relationship, and adopts the combination of differential enhancement strategy and contrast learning mechanism, which significantly improves the learning ability of the model for low-frequency relationships. The core innovation is to systematically integrate the relationship frequency information into data enhancement, representation learning and training target, and construct an end-to-end long-tail relationship extraction framework. The experimental results show that in the face of serious class imbalance, the application significantly improves the extraction effect of tail relationships while keeping the overall performance stable, providing an effective solution to the long-tail problem in natural language processing. BRIEF DESCRIPTION OF DRAWINGS

[0030] Figure 1 is a relationship extraction network diagram of the application.

[0031] Figure 2 is a schematic diagram of the construction method of the positive sample pair of the application.

[0032] Figure 3 is a device diagram of the application. DETAILED DESCRIPTION

[0033] The technical solutions of the application will be further described below with reference to the drawings.

[0034] Embodiment 1:

[0035] This embodiment relates to a relationship extraction method based on contrast learning and differential data enhancement, including the following steps:

[0036] Step 1: Given a dataset containing a set of text samples Adopting the multi-instance learning paradigm, instances sharing the same relationship triple <e1, e2, r1> are regarded as a data packet B. In the training phase of the model, the number of instances of each relationship in the dataset is counted one by one Then calculate the maximum sample number r m =max(N r1 ,N r2 ,…,N ri ), take r m as the denominator, calculate the ratio of each relationship relative to the maximum value, and then divide the relationship based on the threshold:

[0037]

[0038] Where i=1,…,N ri , B Major represents the packet set of the main relationship, and B Tail represents the packet set of the tail relationship

[0039] In the prompt function design, this module defines two types of demonstration templates, respectively suitable for main relations and tail relations.

[0040] For main relation samples, the "basic transformation prompt" strategy is adopted to guide the language model to generate semantically equivalent but structurally different sentences. The instruction form is: "Please rewrite '[original sentence]' while keeping the [relation type] relationship between [entity 1] and [entity 2] unchanged, making it semantically equivalent but structurally different." This prompt encourages the language model to adjust the structure of the original sentence, such as changing the order or replacing some words, while maintaining the core semantics.

[0041] In contrast, the "enhanced transformation prompt" strategy is adopted for tail relations, guiding the model to generate diversified alternative sentences with significantly different styles and vocabulary. The template is: "Given a sentence'[original sentence]'expressing the [relation type] between [entity 1] and [entity 2], please generate 3 semantically equivalent, significantly different in vocabulary and syntax structure. These sentences can add reasonable context information to enhance language diversity and naturalness while maintaining the explicitness of the [relation type] and ensuring no changes occur." By introducing context expansion and content restructuring, this type of prompt can guide the model to explore more rich language expression forms, effectively improving the representation ability of tail relation samples.

[0042] In the process of generating enhanced samples, the system introduces a differentiated sampling strategy. Specifically, for main relation samples, a lower generation temperature parameter (τ = 0.6) is set and a higher core sampling threshold (p = 0.98) is used to generate 1 to 2 semantically close samples, ensuring the stability of the generated content. For tail relation samples, a higher temperature parameter (τ = 1.2) and a slightly lower core sampling threshold (p = 0.88) are used to increase the randomness and diversity of sampling, generating 5 to 10 significantly different enhanced sentences for each original sample. To improve the modeling ability of different relation types, the model adjusts the temperature parameter τ and the core sampling threshold p to effectively generate data that fits the characteristics of each relation type, thereby enhancing the diversity of the corpus in structure and expression and alleviating the training difficulties caused by the scarcity of long-tail relation samples.

[0043] Step 2: Build a double-level encoder, which includes the following steps:

[0044] (1) Sentence-level encoder: Use the BERT encoder to obtain sentence-level representation for each sentence. Specifically, given an input sentence S i , use the BERT tokenizer to get the token sequence where N wFor the number of words in a sentence, i represents the i-th sentence in the input instance, e i1 and e i2 correspond to the head entity and tail entity in the sentence, respectively. Following the design of encoder-only Transformer, special symbols [CLS] and [SEP] are added at the beginning and end of each sentence, respectively. Then the word embeddings and position embeddings are input into the hidden layer of BERT to obtain a series of context representations In order to highlight the semantics related to the entity, the hidden vectors corresponding to the entity e i1 and e i2 are concatenated to obtain the entity-aware sentence representation h i The formula is as follows:

[0045]

[0046] Where : represents the concatenation operation.

[0047] (2) Package-level encoder:

[0048] For a package B containing shared fact triples <e1, e2, r c (where rc is the correct relationship), its weighted representation is The formula is as follows:

[0049]

[0050] Where N b is the size of the package, h i follows the definition in the sentence encoder, and a ki is the attention score between the k-th relationship and the i-th instance. The score is calculated as Where a ki represents the matching degree between the k-th relationship query and the i-th sentence in the package.

[0051] Then, a fully connected layer with a Softmax function is used to obtain the probability distribution of the relationship, and the formula is as follows:

[0052]

[0053] Where θ represents the learnable parameters, represents the relative score of all relationship labels, N r is the total number of relationships, M is the relationship weight matrix, and b is the bias term.

[0054] For the package encoder, our model uses cross-entropy loss as the task loss, and the loss function is as follows:

[0055]

[0056] Where, N s It is the set of all training samples

[0057] Step 3: Constructing the Contrastive Learning Module: This module builds sample pairs to optimize the discriminative ability of the representation space: Positive sample pairs consist of the original sample and its corresponding augmented sample, using the same entity pair and relation type to maximize representational similarity; Negative sample pairs consist of packet representations from different entity pairs or relations, using sentence semantic distance to filter difficult negative samples and enhance discriminative power. The contrastive loss uses the InfoNCE loss function to bring positive samples closer and distance negative samples further away, thereby significantly improving the model's discriminative ability under tail relations.

[0058] Positive sample pairs (x) i ,x i + The construction method of ) is as follows Figure 2 As shown. Given a primary relation instance x in a batch. m Example x of tail relationship t , where {x i =x m +x t |x m ∈B Major ,x t ∈B Tail}, representing the blended view as

[0059] x * =z major (x m )+z tail (x t (7)

[0060] Where z(·) represents selective enhancement. Finally, the feature representation is obtained using the BERT-based encoder f(·).

[0061] negative sample pairs (x) i ,x t - The construction of ) obtains a batch of packets (B1, B2, ..., B) through the packet encoder. G ) and its weighted representation, where G is the batch size. Follows CIL

[56] The method for constructing negative sample pairs uses other weighted packets to represent B in the same batch. t As x i - , where x i - ∈B s Furthermore, t≠s, ensuring that they come from different packages (different entity pairs or different relations), which helps to form a rich set of negative sample pairs to widen the distance between different relations.

[0062] Step 4: To further enhance the model's discriminative ability and generation diversity on long-tail relationships, the method improves and replaces the original training objective, introducing a more expressive supervision mechanism. At the sentence level, a supervised contrastive loss (Supervised Contrastive Loss) is used instead of the traditional InfoNCE loss. This loss function uses the similarity between multiple positive samples within the same relationship category to enhance intra-class consistency, while distinguishing between different categories. Its specific definition is as follows:

[0063]

[0064] where z i is the sentence representation of sample i, P(i) represents the set of positive samples belonging to the same relationship category as it does, A(i) is all the contrastive samples in the current batch, and τ is the temperature coefficient.

[0065] In the package-level modeling aspect, a prototype contrastive loss is introduced to replace the traditional package-level classification loss. A learnable prototype vector p r is maintained for each relationship r B , so that the package-level representation z B is closer to the correct relationship prototype and further away from other categories, and its loss function is defined as:

[0066]

[0067] where y B represents the true relationship label of package B.

[0068] To improve the model's generation ability and guide it to learn the underlying patterns between entity semantics, we introduce an auxiliary training objective L M , which is the entity-aware masked language modeling (Entity-aware MLM) and prompt-based generation (Prompt-based Generation) in the language modeling objective. For example, deliberately masking entity or relationship words in the input sentence and requiring the model to predict the masked content helps it understand semantic roles and relationship structures.

[0069] The final training objective function integrates the above three loss forms and is constructed as:

[0070]

[0071] where, is the weight control function that changes with the training step number, N B represents the number of samples in the current batch, and β and γ are the loss weight hyperparameters of the package-level and language modeling objectives, respectively.

[0072] Step 5: Difficult sample mining mechanism and dynamic contrastive sampling strategy. This strategy mainly includes the following two aspects:

[0073] Difficult positive sample mining: In each training epoch, identify the positive samples with low current prediction confidence of the model, and regard them as "difficult positive examples". These samples will be preferentially sampled in the next round of training to strengthen the model's learning ability for complex expressions.

[0074] Dynamic negative sampling mechanism: When constructing negative samples, instead of using randomly selected negative examples, the cosine similarity based on sentence semantic representation is calculated to dynamically select "difficult negative samples" closest to the positive sample representation. This method helps to improve the training gradient of contrastive learning and strengthen the model's discriminative ability for relationship boundaries.

[0075] At the same time, in order to balance the number of difficult samples and training stability, a sample difficulty distribution function γ(d) is introduced to dynamically adjust the sampling probability of samples according to their semantic distance:

[0076]

[0077] where d i represents the semantic distance of the sample from its same-class center, μ is the dynamic center distance, and β is the hyperparameter controlling the sampling tilt degree. This function ensures that the training process focuses on both extreme difficult samples and retains a certain proportion of easy-to-learn samples to stabilize convergence.

[0078] Example 2:

[0079] This embodiment relates to a disease diagnosis and treatment method applying the relationship extraction method based on contrastive learning and differential data augmentation of the present application.

[0080] This embodiment demonstrates the specific application effect of the method of the present application in the medical field, especially in the entity relationship extraction task of drug-disease, disease-symptom, etc. in electronic medical record (EMR) text, including the following steps:

[0081] Step 1: Differential data augmentation: based on the distribution characteristics of relationship categories in training data, use large language models to perform high diversity data augmentation on tail relationship instances and low disturbance data expansion on main frequency relationship instances;

[0082] Select the public medical relationship extraction dataset VSCMeD, which contains 595 real vascular surgery medical records, annotated entities and their relationships, for "drug-disease" and "disease-symptom" relationship construction. After counting the frequency of each type of relationship, it is found that "treatment", "causes" and other main relationships, and "prevention", "influence" and other tail relationships.

[0083] Accordingly, the prompt template is designed as follows:

[0084] Main relation template example:

[0085] "Rewrite the following medical sentence, keeping the 'treatment' relationship between '[Entity 1]' and '[Entity 2]' unchanged: '[Original sentence]'."

[0086] Tail relation template example:

[0087] "Based on '[Original sentence]', generate 3 new sentences expressing the 'prevention' relationship between '[Entity 1]' and '[Entity 2]' in a semantically equivalent manner, ensuring diversity in expression."

[0088] For main relations, use a low temperature and high core sampling strategy to generate 1-2 semantically close samples; for tail relations, use a high temperature and low core sampling strategy to generate 5-10 samples with rich syntactic transformations.

[0089] Step 2: Multi-level representation learning: Construct a double-level encoding structure, including a sentence-level encoder and a package-level encoder, to perform semantic representation on original and enhanced samples; use the BERT model to encode each enhanced EMR sentence, extracting entity-aware sentence representations; organize sentences into "relation packages" according to their attribution in the case record, constructing package-level semantic representations; apply attention mechanisms to weight and aggregate sentences in the package to obtain a relation probability distribution.

[0090] Step 3: Contrastive learning to optimize representation discriminability: Form positive sample pairs with original and enhanced samples, and negative sample pairs with samples from different relations, combined with a difficult negative sample mining mechanism to guide the model to optimize the semantic representation space.

[0091] Original medical text and its enhanced sentence form a positive sample pair, such as:

[0092] Original sentence: "Insulin is commonly used to treat diabetes."

[0093] Enhanced sentence: "Diabetes is usually controlled by using insulin."

[0094] Negative samples come from the semantic representations of other relations (such as "concurrent" and "trigger") in the package, constructing difficult negative contrast instances.

[0095] Step 4: Define training objectives: Combine package-level relation classification loss and sentence-level contrastive learning loss, integrate language model pre-training objectives, and form a unified optimization target. Introduce supervised contrastive loss and prototype contrastive learning to model "treatment" and "prevention" prototypes, making the model have stronger discriminability for intra-class consistency and inter-class differences of medical relations.

[0096] Step 5: Design a difficult positive sample priority training mechanism and a dynamic negative sample sampling strategy based on semantic distance.

[0097] During the training process, it is found that the "drug-side effect" relationship is uncertain in some samples. The system automatically identifies these as difficult positive samples and prioritizes them for the next round of training. It also dynamically selects "drug-use" samples with similar semantics but different relationships as difficult negative samples.

[0098] Step 6. Apply the dataset obtained in step 5 for disease diagnosis and treatment. Based on the relationship extraction model obtained through training, the system can automatically identify "drug-disease", "disease-symptom", and other key medical entity relationships in electronic medical record texts, assist in constructing patient symptom atlases and medication information, and further support disease diagnosis reasoning and personalized treatment recommendations. At the same time, combined with side effect and use relationship modeling, it improves the safety and accuracy of drug use, and provides reliable basis for clinical decision-making.

[0099] Embodiment 3

[0100] This embodiment relates to a network commodity recommendation method applying the relationship extraction method based on contrastive learning and differential data augmentation of the present application.

[0101] This embodiment illustrates the specific application of the present application in the network commodity recommendation scenario, mainly used for extracting "user-behavior", "commodity-attribute", "user-preference", "recommendation reason-commodity characteristics" and other semantic relationships from user comments, commodity descriptions, and behavior logs, to improve the semantic understanding ability and recommendation accuracy of the recommendation system. It includes the following steps:

[0102] Step 1: Differential data augmentation: based on the distribution characteristics of relationship categories in the training data, use a large language model to perform high-variability data augmentation on tail relationship instances and low-disturbance data expansion on main frequency relationship instances;

[0103] Select public commodity recommendation corpus, including Amazon Reviews Dataset, Jingdong user comment data (JDFull), AliExpress commodity text and image comment set, containing commodity information, user comments, behavior records (click, browse, purchase) and recommendation labels ("recommended for you", "guess you like") and other information.

[0104] Statistical results show that "user-behavior (click, purchase)", "commodity-attribute (price, brand)" are main relationships, while "user-preference", "recommendation reason-commodity characteristics" and other relationships are tail relationships, with few samples and complex semantics.

[0105] Design differential prompt templates for different relationships:

[0106] Main relationship template examples:

[0107] "Please rewrite the following sentence without changing the relationship between 'user' and 'purchase behavior': '[Original sentence]'."

[0108] Tail relationship template example:

[0109] "Please generate three different sentence patterns that express the reasons why the product is recommended based on the original sentence '[Original sentence]'. Keep features such as 'high cost performance' and 'popular brand', and enhance the diversity of expression."

[0110] Use a differentiated sampling strategy to generate 1-2 structural variant sentences for the main relationship, and 5-10 new samples with content differences for the tail relationship, significantly improving sample richness and tail modeling capabilities.

[0111] Step 2: Multi-level representation learning: Construct a double-level encoding structure, including a sentence-level encoder and a package-level encoder, to perform semantic representation on original and enhanced samples.

[0112] Use the BERT encoder to extract sentence representations for each product review or recommendation description sentence; organize sentences describing the same "user-product" pair or the same recommendation label into "relationship packages" to build package-level semantic representations; use attention mechanism to weight and aggregate sentence representations in the package to obtain the classification distribution of the recommendation relationship.

[0113] Step 3: Contrastive learning, optimize representation discriminability: Form positive sample pairs with original and enhanced samples, and negative sample pairs with samples from different relationships, combined with difficult negative sample mining mechanisms, to guide the model to optimize the semantic representation space.

[0114] Construct positive sample pairs:

[0115] Original sentence: "This earphone has good sound quality, and I often buy it"

[0116] Enhanced sentence: "I like this earphone with excellent sound quality"

[0117] Construct negative sample pairs: such as sentence pairs from different users, different products or different recommendation reasons, and use semantic similarity to select difficult negative examples.

[0118] Through contrastive loss, strengthen the model's ability to distinguish similar product attributes and user preference expressions, and better understand the fine-grained semantic differences in the recommendation context.

[0119] Step 4: Training target definition: Joint package-level relationship classification loss and sentence-level contrastive learning loss, integrate language model pre-training target, form unified optimization target.

[0120] At the sentence level, supervised contrastive loss is used to improve the semantic consistency of implicit relationships such as "user-preference"; at the package level, prototype contrastive learning is used to make samples of the same recommendation logic or recommendation reason aggregated in the same prototype; auxiliary tasks introduce "recommendation target prediction" and "commodity characteristic mask generation" to improve the modeling ability of the model for user intent and commodity features.

[0121] Step 5: Design includes difficult positive sample priority training mechanism and dynamic negative sample sampling strategy based on semantic distance.

[0122] Identify samples expressing vague but containing potential preferences (such as "this is not bad" and "consider it") as difficult positive samples, and prioritize them for training;

[0123] By calculating the semantic distance between user or commodity representations, difficult negative samples are dynamically selected (such as "cost-effective" vs. "good brand");

[0124] Use sample difficulty function γ(d) to adjust the training sample sampling probability to improve training stability and learning ability for difficult relationships.

[0125] Step 6. Apply the strategy of step 5 to recommend network goods. Based on the obtained relationship extraction and representation learning model, the system can automatically identify key relationships in user reviews and commodity descriptions, combine user behavior history and commodity attributes, and realize personalized recommendation. By modeling the tail relationships with rich semantic expression but complex structure, the system can accurately mine user's implicit demand and recommendation logic, improve the relevance and persuasiveness of the recommended content, and generate a more targeted commodity recommendation list for the user.

[0126] Embodiment 4

[0127] Reference Figure 3 The embodiment relates to a relationship extraction device for contrastive learning and differential data enhancement, comprising a memory and one or more processors, wherein the memory stores executable code, and the one or more processors execute the executable code to implement the relationship extraction method for contrastive learning and differential data enhancement in the embodiment 1.

[0128] Embodiment 5

[0129] The embodiment relates to a computer readable storage medium, which stores a program, and the program is executed by a processor to implement the relationship extraction method for contrastive learning and differential data enhancement in the embodiment 1.

[0130] The application has been described by the above examples, but it is obvious that the examples here are only for the purpose of explanation and illustration, and do not limit the application to the scope of the examples. The researchers in the technical field of the application can make non-essential changes in form and content according to the above steps without deviating from the scope of the application actually protected, and therefore the application is not limited to the above specific forms and details.

Claims

1. A relation extraction method based on contrastive learning and differential data augmentation, comprising the following steps: Step 1: Differential data augmentation: based on the distribution characteristics of the relation categories in the training data, a large language model is used to perform high diversity data augmentation on tail relation instances and low disturbance data expansion on main frequency relation instances; Step 2: Multi-level representation learning: a double-level encoding structure is constructed, including a sentence-level encoder and a package-level encoder, to perform semantic representation on original and augmented samples; Step 3: Contrastive learning: original samples and augmented samples form positive sample pairs, and samples from different relations form negative sample pairs. Combined with a difficult negative sample mining mechanism, the model is guided to optimize the semantic representation space; Step 4: Training target definition: combine the package-level relation classification loss and the sentence-level contrastive learning loss, integrate the language model pre-training target, and form a unified optimization target. Step 5: Design includes difficult positive sample priority training mechanism and dynamic negative sample sampling strategy based on semantic distance.

2. The relation extraction method for contrastive learning and differential data augmentation according to claim 1, wherein the step 1 specifically comprises: For tail relation samples, use a high temperature parameter (τ≥1.2) and a low core sampling threshold (p≤0.88) to generate diversity augmented samples; For main relation samples, use a low temperature (τ≤0.6) and a high core sampling threshold (p≥0.98) to generate structure equivalent samples.

3. The relation extraction method for contrastive learning and differential data augmentation according to claim 1, wherein the step 2 specifically comprises: The sentence-level encoder uses the BERT model for context embedding, and concatenates the representations of the head and tail entities in the sentence to form entity-aware semantic vectors. The package-level encoder uses an attention mechanism to weight and fuse the representations of the sentences in the package, and obtains the probability distribution of relation classification through Softmax.

4. The relation extraction method for contrastive learning and differential data augmentation according to claim 1, wherein the step 3 specifically comprises: The positive sample pair is composed of the same original sentence and its augmented sentence, and the negative sample pair is composed of the package representations of different entities or relations.

5. The relation extraction method for contrastive learning and differential data augmentation according to claim 1, wherein the step 4 specifically comprises: The training target includes three parts: sentence-level supervised contrastive loss, package-level prototype contrastive loss, and language modeling auxiliary target; the sentence-level loss is a supervised contrastive loss, which maximizes the similarity between samples within the same relation category and distinguishes the representations between different relation categories, effectively enhancing the intra-class consistency; the package-level loss is a prototype contrastive loss, which defines a learnable prototype vector for each relation, so that the package-level semantic representation is closer to the true relation prototype and farther away from other relation prototypes; the language modeling auxiliary target includes entity-aware mask language modeling and relation prompt generation, the model randomly masks the entity or relation words in the input sentence and predicts their contents to enhance the understanding ability of semantic roles and relation structures; The final training target function is: wherein, is a weight regulation function varying with the number of training steps, N B denotes the number of samples in the current batch, and β and γ are the loss weight hyperparameters for the batch level and language modeling objectives, respectively.

6. The relation extraction method for contrastive learning and differential data augmentation according to claim 1, wherein the step 5 specifically comprises: A difficult positive sample mining and dynamic negative sample mechanism is introduced, and a sample difficulty distribution function γ(d) is introduced to balance the number of difficult samples and training stability.

7. A relation extraction apparatus for contrastive learning and differential data augmentation, characterized in that, A memory and one or more processors are included, the memory has executable code stored therein, and the one or more processors execute the executable code to implement the relation extraction method for contrastive learning and differential data augmentation of any one of claims 1-6.

8. A computer-readable storage medium, characterized in that, A program is stored thereon, and the program is executed by a processor to implement the relation extraction method for contrastive learning and differential data augmentation of any one of claims 1-6.

Images