A low-resource relation extraction method based on semantic correction and adaptive contrast

Abstract

The application discloses a low-resource relation extraction method based on semantic correction and adaptive contrast, and belongs to the technical field of information extraction and artificial intelligence. The method of the application is as follows: 1, generating an enhanced sample; 2, fine-tuning an exogenous pseudo label scoring device; fine-tuning a relation extraction model by using cross-entropy loss; obtaining double-source pseudo label scores by using exogenous pseudo label scoring and endogenous pseudo label scoring; 3, calculating relation cluster semantic correction contrast loss of relation mapping features and entity mapping features; 4, combining adaptive contrast, relation cluster semantic correction contrast and cross-entropy loss to form relation extraction model loss; 5, iteratively training the relation extraction model by using model loss and cross-entropy loss; 6, inputting unstructured text data into the iteratively trained relation extraction model to obtain a relation type. Compared with the prior art, the application solves the technical problem of improving the accuracy of unstructured text data semantic extraction in a low-resource relation extraction task.

Description

Technical Field

[0001] This invention relates to a low-resource relation extraction method based on semantic correction adaptive comparison, belonging to the fields of information extraction and artificial intelligence technology, and is applied to extracting semantic relations between entities from unstructured text. Background Technology

[0002] Relation extraction aims to automatically extract semantic relationships between entities in unstructured text. It is a crucial subtask of information extraction and is widely applied in downstream tasks such as knowledge graph construction, intelligent question answering, search engines, and recommendation systems. This technique typically relies on large amounts of labeled data for supervised model training. Low-resource relation extraction aims to improve traditional supervised learning paradigms based on methods such as meta-learning, transfer learning, and data augmentation, extracting semantic relationships between target entities in text using less labeled corpus. With low-resource relation extraction, relation extraction models can reduce their dependence on large-scale labeled data and improve generalization ability while maintaining extraction performance as much as possible.

[0003] Current research on low-resource relation extraction faces challenges primarily in data, training strategies, and semantic modeling. On one hand, data augmentation methods lack diversity, have limited information gain, or exhibit excessive flexibility and semantic drift. On the other hand, self-training methods often produce pseudo-labels with insufficient accuracy, particularly in the early stages of iteration, leading to error propagation. Furthermore, existing research underutilizes the semantic information of relation types, failing to fully explore their inherent meaning.

[0004] Therefore, improving the performance of extracting semantic relationships from unstructured text data in low-resource scenarios has become an urgent problem to be solved. Summary of the Invention

[0005] The purpose of this invention is to solve the technical problem of improving the extraction of semantic relations from unstructured text data in low-resource scenarios, and to propose a low-resource relation extraction method based on semantic correction adaptive comparison.

[0006] The innovations of this invention include: First, designing a multi-granularity augmentation method for relation constraints, using a large language model to perform generative data augmentation on unlabeled samples at multiple granularities to obtain augmented samples. Then, calculating the semantic correction contrastive loss for relation clusters for unlabeled samples with dual-source pseudo-label scores exceeding a threshold, and for relation clusters constructed using the large language model. Finally, calculating the adaptive contrastive loss for both unlabeled samples and augmented samples using dual-source pseudo-label scores. This invention improves information gain while avoiding semantic drift, reduces interference from pseudo-label noise, and fully utilizes the semantic information of relation types.

[0007] The objective of this invention is achieved through the following technical solution:

[0008] This invention discloses a low-resource relation extraction method based on semantic correction and adaptive comparison, comprising the following steps:

[0009] Step 1: Insert structured unlabeled samples into the prompt words In China, large language models are augmented with multi-granular data. The generated initial augmented samples are constrained to form constrained augmented samples; semantic similarity is used to filter the constrained augmented samples in the candidate augmented sample pool to generate augmented samples;

[0010] Step 1.1: Obtain unstructured text data as the original sample;

[0011] Step 1.2: Randomly select from the original samples and construct relation types. The mapping relationship with the original samples forms labeled samples; the original samples that were not randomly selected are treated as unlabeled samples.

[0012] Step 1.3: Use semantic similarity to filter the constrained augmentation samples placed in the candidate augmentation sample pool to generate augmentation samples;

[0013] Step 1.3.1: Construct a set of prompt words with relational constraints, consisting of token granularity enhancement, clause granularity enhancement, and sentence granularity enhancement. ;

[0014] Step 1.3.2: Insert the structured, unlabeled samples with the configured structure into the prompt words. In the process, sample format constraints are applied to form constraint-enhanced samples;

[0015] Step 1.3.2.1: Set the structure of the entity position of the unlabeled sample to form a structured unlabeled sample as shown in equation (1);

[0016] (1)

[0017] in, , Mark the start and end positions of the header entity respectively. , Mark the start and end positions of the tail entity respectively. For the sample Token;

[0018] Step 1.3.2.2: Insert structured unlabeled samples into the prompt words. In China, multi-granularity data is used to augment large language models. Generate the initial enhanced sample as shown in equation (2);

[0019] (2)

[0020] in, This is the initial augmentation sample for token granularity, which is about to be... Replace "Token" with a synonym; This is the initial enhancement sample at the clause granularity, i.e., modification. Clauses; This serves as the initial sample for sentence-level augmentation, i.e., rewriting sentences using different wording or sentence structures. ;

[0021] Step 1.3.2.3: Apply sample format constraints to the initial augmented samples to form constrained augmented samples;

[0022] Step 1.3.3: Use semantic similarity to filter the constrained augmentation samples placed in the candidate augmentation sample pool to generate augmentation samples;

[0023] Step 1.3.3.1: Construct a candidate augmentation sample pool using the constraint augmentation sample set generated by the iterative loop;

[0024] Step 1.3.3.2: Use the semantic similarity shown in Equation (3) to filter the constrained enhancement samples in the candidate enhancement sample pool and generate enhancement samples;

[0025] (3)

[0026] in, Indicates particle size, Indicates the candidate boosting sample pool, The original unlabeled sample express The indivual Granularity-enhanced samples This represents the semantic similarity based on a single pre-trained language model, BERT.

[0027] Step 2: Utilizing the LoRA parameter-efficient fine-tuning method and the loss of the external pseudo-label scorer for the external pseudo-label scorer. Fine-tune; utilize cross-entropy loss Relation extraction model Fine-tuning was performed; external pseudo-labels were used for scoring. and endogenous pseudo-label scoring Obtaining dual-source pseudo-label scores ;

[0028] Step 2.1: Construct an external pseudo-label scorer The LoRA parameter-efficient fine-tuning method and the loss of the exogenous pseudo-label scorer are used to evaluate the exogenous pseudo-label scorer. Make fine adjustments;

[0029] Step 2.1.1: Utilize labeled samples based on relation type Construct positive and negative samples;

[0030] Step 2.1.2: Input positive and negative samples into the external pseudo-label scorer in batches using a random combination. In the process, the loss of the external pseudo-label scorer is obtained in the manner shown in Equation (4);

[0031] (4)

[0032] in, The number of labeled samples. To fine-tune the sample size, including Positive samples and One negative sample, Indicates the first The length of the output sequence for each sample for The output probability of a large language model For the first The input sequence of a sample, Indicates the first The nth sample target output sequence Each Token Indicates the first The first sample target output sequence One Token;

[0033] Step 2.1.3: Utilizing the LoRA parameter-efficient fine-tuning method and the loss of the external pseudo-label scorer for the external pseudo-label scorer. Make fine adjustments;

[0034] Step 2.2: Construct a relation extraction model Using cross-entropy loss Relation extraction model Make fine adjustments;

[0035] Step 2.2.1: Construct a relation extraction model Relation extraction model The features extracted from the labeled samples are concatenated into binary tuples to form entity features;

[0036] Step 2.2.1.1: Utilizing the relation extraction model The labeled samples are used to extract features as shown in Equation (5);

[0037] (5)

[0038] in, Indicates the addition of a special token , , , The following are labeled samples. , Embed dimensions into the BERT model;

[0039] Step 2.2.1.2: Perform binary concatenation on the features of the labeled samples to form entity features as shown in equation (6);

[0040] (6)

[0041] in, and Special Token , Corresponding features This indicates a feature concatenation operation. Original sample Entity characteristics;

[0042] Step 2.2.2: Utilize cross-entropy loss Relation extraction model Make fine adjustments;

[0043] Step 2.2.2.1: Obtain the relation extraction probability distribution of entity features using the method shown in equation (7). ;

[0044] (7)

[0045] in, This is the classification layer weight matrix. For the classification layer bias term, For a set of relation types The size of this, i.e., the number of relation types. This indicates a random deactivation operation. For activation functions;

[0046] Step 2.2.2.2: Through Calculate the cross-entropy loss as shown in equation (8). ;

[0047] (8)

[0048] in, The number of labeled samples. Indicates the first The true relation type of each sample;

[0049] Step 2.2.2.3: Utilize cross-entropy loss Relation extraction model Make fine adjustments;

[0050] Step 2.3: Input the unlabeled samples into the fine-tuned relation extraction model. This generates pseudo-labels for unlabeled samples;

[0051] Step 2.4: Scoring using external pseudo-labels and endogenous pseudo-label scoring Obtaining dual-source pseudo-label scores ;

[0052] Step 2.4.1: Input the unlabeled samples and their pseudo-labels into the fine-tuned external pseudo-label scorer. Obtain the exogenous pseudo-label score as shown in equation (9). ;

[0053] (9)

[0054] in, and They represent Output the logarithmic odds for "Yes" and "No";

[0055] Step 2.4.2: Input the unlabeled samples into the fine-tuned relation extraction model. Obtain the endogenous pseudo-label score as shown in equation (10). ;

[0056] (10)

[0057] in, For pseudo-labels of unlabeled samples, the relation extraction model As an endogenous pseudo-label scorer;

[0058] Step 2.4.3: Scoring using external pseudo-labels and endogenous pseudo-label scoring Obtain the dual-source pseudo-label score as shown in equation (11). ;

[0059] (11)

[0060] in, To balance the weights of the two types of pseudo-label scores;

[0061] Step 3: Utilize large language models Constructing relational types with labeled samples Relation clusters are generated by using a relation encoder to form relation mapping features for the relation clusters. The dual-source pseudo-label score should be no less than [amount missing]. Unlabeled samples and augmented samples are mapped to form entity mapping features. ; Calculate the semantic correction contrast loss for relation clusters of relation mapping features and entity mapping features. ;

[0062] Step 3.1: Construct relation clusters and extract relation mapping features of the relation clusters using a relation encoder;

[0063] Step 3.1.1: Utilizing a large language model Relationship types for labeled samples Construct relation clusters;

[0064] Step 3.1.1.1: Utilizing a large language model Relationship types for labeled samples Generate single relation descriptions through context learning;

[0065] Step 3.1.1.2: Generate multiple single relation descriptions through iterative loops, grouping those belonging to the same relation type... A single relation describes a set of relations;

[0066] Step 3.1.2: Use the pre-trained language model BERT as a relation encoder The relation mapping features of the relation clusters shown in Equation (12) are extracted by the relation encoder. ;

[0067] (12)

[0068] in, for and The feature represents the result after splicing. The name of the relation type. For relation type The corresponding relation cluster in the first This relationship explanation The weight matrix of the mapping layer. For the mapping layer bias term, For L2 normalization;

[0069] Step 3.2: Ensure the dual-source pseudo-label score is not less than [amount missing]. Unlabeled samples and augmented samples are mapped to form entity mapping features. ;

[0070] Step 3.2.1: Define the threshold The dual-source pseudo-label score should be no less than Unlabeled samples are considered as reliable pseudo-labeled samples;

[0071] Step 3.2.2: Map the trusted pseudo-label samples and the enhanced samples to form entity mapping features using the method shown in Equation (13). ;

[0072] (13)

[0073] Step 3.3: Calculate the semantic correction contrast loss of relation clusters for relation mapping features and entity mapping features using Equation (14). ;

[0074] (14)

[0075] Among them, when hour ,when hour, , This represents the original number of unlabeled samples. Temperature for semantic correction contrast loss of relation clusters For the first Two-source pseudo-label scores for each sample. For the first Pseudo-labels for each sample For a set of relation types, For the size of the relation cluster, For the first Entity mapping features of each sample For relation type The corresponding relation cluster in the first The mapping characteristics described by each relation;

[0076] Step 4: Utilize entity contrast mapping features to obtain the adaptive contrast loss for positive and negative samples of unlabeled samples. Combined with adaptive contrastive loss Semantic correction contrast loss for relation clusters and cross-entropy loss Forming a relation extraction model loss ;

[0077] Step 4.1: Map the unlabeled samples and augmented samples to form entity contrast mapping features using the method shown in Equation (15). ;

[0078] (15)

[0079] in, For entity features, The weight matrix of the mapping layer. For mapping layer bias terms;

[0080] Step 4.2: Obtain the adaptive contrast loss for positive and negative samples of unlabeled samples using entity contrast mapping features. ;

[0081] Step 4.2.1: Obtain the positive sample contrast distance of the unlabeled sample using the method shown in Equation (16);

[0082]

[0083] (16)

[0084] in, Temperature for adaptive contrastive learning, For the first Two-source pseudo-label scores for each sample. For the first Entity mapping features of each sample For the first Each sample corresponds to Granularity enhances the entity mapping features of samples. , and The first The and the first The pseudo-labels of the samples;

[0085] Step 4.2.2: Obtain the negative sample contrast distance of the unlabeled sample using the method shown in Equation (17);

[0086]

[0087] (17)

[0088] Step 4.2.3: Obtain the adaptive contrastive loss for positive and negative samples of unlabeled samples using the method shown in Equation (18). ;

[0089] (18)

[0090] in, This represents the number of unlabeled original samples;

[0091] Step 4.3: Obtain the relation extraction model as shown in equation (19) Model loss ;

[0092] (19)

[0093] in, For use in balancing and The weight, To compare the learning loss, For the cross-entropy loss in formula (8), For use in balancing and The weights;

[0094] Step 5: Utilize model loss and cross-entropy loss Iterative training of relation extraction model ;

[0095] Step 5.1: Utilize model loss Fine-tuning the relation extraction model ;

[0096] Step 5.2: Scoring the dual-source pseudo-labels Not less than Trustworthy pseudo-labeled samples are added to the set of labeled samples, and the cross-entropy loss of the updated labeled samples is used. Fine-tuning the relation extraction model ;

[0097] Step 5.3: Execute steps 2.3 to 5.2 in an iterative loop until the loop completes. Until then;

[0098] Step 6: Input the unstructured text data into the iteratively trained relation extraction model. To obtain the relation types used for semantic extraction of unstructured text data in low-resource relation extraction tasks;

[0099] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0100] 1. A multi-granularity data augmentation method based on relation constraints is designed to fully leverage the advantages of large language models in semantic understanding and generation. This method augments data at multiple granularities, improving data flexibility and diversity, and addressing the limited information gain of existing methods. Relation constraints and semantic consistency filtering mechanisms effectively enhance the relevance of generated data to the relation extraction task, preventing semantic drift and ensuring the practicality of the augmented samples.

[0101] 2. An adaptive contrastive loss based on dual-source pseudo-label scoring is constructed, effectively utilizing the powerful semantic understanding capabilities and rich domain knowledge of large language models. Simultaneously, the stability and rationality of the scoring are enhanced by integrating dual-source scoring. Adaptively adjusting the weights of various components of the contrastive loss through dual-source pseudo-label scoring reduces interference from pseudo-label noise, solves the error propagation problem caused by erroneous pseudo-labels, and enhances the robustness of the model.

[0102] 3. By introducing relation specification into the relation cluster semantic correction contrastive loss, we prevent similar samples from deviating from the semantic center of the corresponding relation type, thereby enhancing inter-class discrimination and improving the model's representational ability during training. Furthermore, relation clusters composed of multiple relation specification sets can enhance the model's ability to characterize intra-class diversity, reduce the risk of overfitting, and improve the stability of the feature learning process. Attached Figure Description

[0103] Figure 1 is a schematic diagram of the process of the present invention; Detailed Implementation

[0104] To better illustrate the purpose and advantages of this invention, the invention will be further described below with reference to the accompanying drawings and examples. It should be noted that the implementation of this invention is not limited to the following embodiments, and any modifications or alterations made to this invention will fall within the scope of protection of this invention.

[0105] Example

[0106] like Figure 1 As shown in the figure, the specific implementation steps of the low-resource relation extraction method based on semantic correction adaptive comparison in this embodiment are as follows:

[0107] Step 1: Insert structured unlabeled samples into the prompt words In China, large language models are augmented with multi-granular data. The generated initial augmented samples are constrained to form constrained augmented samples; semantic similarity is used to filter the constrained augmented samples in the candidate augmented sample pool to generate augmented samples;

[0108] Step 1.1: Obtain unstructured text data as the original sample;

[0109] Step 1.2: Randomly select from the original samples and construct relation types. The mapping relationship with the original samples forms labeled samples; the original samples that were not randomly selected are treated as unlabeled samples.

[0110] In this embodiment, a labeled sample is illustrated with the sentence "Her firm, Pamela Martin & Associates, was in business from 1993 until last year." The head entity is "Pamela Martin & Associates," the tail entity is "1993," and the relation type is "org:founded," meaning the head entity's creation date is the tail entity's creation date. An unlabeled sample is illustrated with the sentence "Option One is not a significant Kansas City area employer, and most of the previously announced reductions involved California jobs." The head entity is "Option One," and the tail entity is "California."

[0111] Step 1.3: Use semantic similarity to filter the constrained augmentation samples placed in the candidate augmentation sample pool to generate augmentation samples;

[0112] Step 1.3.1: Construct a set of prompt words with relational constraints, consisting of token granularity enhancement, clause granularity enhancement, and sentence granularity enhancement. ;

[0113] Step 1.3.2: Insert the structured, unlabeled samples with the configured structure into the prompt words. In the process, sample format constraints are applied to form constraint-enhanced samples;

[0114] Step 1.3.2.1: Set the structure of the entity position of the unlabeled sample to form a structured unlabeled sample as shown in equation (1);

[0115] (1)

[0116] in, , Mark the start and end positions of the header entity respectively. , Mark the start and end positions of the tail entity respectively. For the sample Token;

[0117] In this embodiment, the structured unlabeled sample is " <e1>Option One< / e1> is not a significant Kansas City area employer, and most of the previously announced reductions involved <e2>California< / e2> jobs.";

[0118] Step 1.3.2.2: Insert structured unlabeled samples into the prompt words. In China, multi-granularity data is used to augment large language models. Generate the initial enhanced sample as shown in equation (2);

[0119] (2)

[0120] in, This is the initial augmentation sample for token granularity, which is about to be... Replace "Token" with a synonym; This is the initial enhancement sample at the clause granularity, i.e., modification. Clauses; This serves as the initial sample for sentence-level augmentation, i.e., rewriting sentences using different wording or sentence structures. ;

[0121] Step 1.3.2.3: Apply sample format constraints to the initial augmented samples to form constrained augmented samples;

[0122] Step 1.3.3: Use semantic similarity to filter the constrained augmentation samples placed in the candidate augmentation sample pool to generate augmentation samples;

[0123] Step 1.3.3.1: Construct a candidate augmentation sample pool using the constraint augmentation sample set generated by the iterative loop;

[0124] Step 1.3.3.2: Use the semantic similarity shown in Equation (3) to filter the constrained enhancement samples in the candidate enhancement sample pool and generate enhancement samples;

[0125] (3)

[0126] in, Indicates particle size, Indicates the candidate boosting sample pool, The original unlabeled sample express The indivual Granularity-enhanced samples This represents the semantic similarity based on a single pre-trained language model, BERT.

[0127] In this embodiment, the token granularity enhancement sample is " <e1>Option One< / e1> is not asubstantial Kansas City area employer, and most of the previously announced reductions involved <e2>California< / e2> jobs." (Option One is not a major employer in the Kansas City area, and most of the previously announced layoffs involved jobs in California); the clause granularity enhancement sample is " <e1>Option One< / e1> is not a significant Kansas City area employer, and the majority of the previously announced job reductions primarily affected <e2>California< / e2> "jobs." (Option One is not a major employer in the Kansas City region, and the previously announced layoffs mostly affected jobs in California); the sentence granularity enhancement sample is "In the Kansas City region, <e1>Option One< / e1> is not a major employer, and the majority of the previously announced layoffs affected employees based in <e2>California< / e2> (In the Kansas City area, Option One is not a major employer, and the previously announced layoffs mostly affected employees in California.)

[0128] Step 2: Utilizing the LoRA parameter-efficient fine-tuning method and the loss of the external pseudo-label scorer for the external pseudo-label scorer. Fine-tune; utilize cross-entropy loss Relation extraction model Fine-tuning was performed; external pseudo-labels were used for scoring. and endogenous pseudo-label scoring Obtaining dual-source pseudo-label scores ;

[0129] Step 2.1: Construct an external pseudo-label scorer The LoRA parameter-efficient fine-tuning method and the loss of the exogenous pseudo-label scorer are used to evaluate the exogenous pseudo-label scorer. Make fine adjustments;

[0130] Step 2.1.1: Utilize labeled samples based on relation type Construct positive and negative samples;

[0131] In this embodiment, the positive sample is "Is the relationship between the head entity "Pamela Martin & Associates" and the tail entity "1993" in the sentence "Herfirm, <e1>Pamela Martin & Associates< / e1> , was in business from <e2> 1993< / e2> "until last year." classified as "org:founded"?", negative examples are those where "org:founded" is changed to other relation types;

[0132] Step 2.1.2: Input positive and negative samples into the external pseudo-label scorer in batches using a random combination. In the process, the loss of the external pseudo-label scorer is obtained in the manner shown in Equation (4);

[0133] (4)

[0134] in, The number of labeled samples. To fine-tune the sample size, including Positive samples and One negative sample, Indicates the first The length of the output sequence for each sample for The output probability of a large language model For the first The input sequence of a sample, Indicates the first The nth sample target output sequence Each Token Indicates the first The first sample target output sequence One Token;

[0135] Step 2.1.3: Utilizing the LoRA parameter-efficient fine-tuning method and the loss of the external pseudo-label scorer for the external pseudo-label scorer. Make fine adjustments;

[0136] Step 2.2: Construct a relation extraction model Using cross-entropy loss Relation extraction model Make fine adjustments;

[0137] Step 2.2.1: Construct a relation extraction model Relation extraction model The features extracted from the labeled samples are concatenated into binary tuples to form entity features;

[0138] Step 2.2.1.1: Utilizing the relation extraction model The labeled samples are used to extract features as shown in Equation (5);

[0139] (5)

[0140] in, Indicates the addition of a special token , , , The following are labeled samples. , Embed dimensions into the BERT model;

[0141] Step 2.2.1.2: Perform binary concatenation on the features of the labeled samples to form entity features as shown in equation (6);

[0142] (6)

[0143] in, and Special Token , Corresponding features This indicates a feature concatenation operation. Original sample Entity characteristics;

[0144] Step 2.2.2: Utilize cross-entropy loss Relation extraction model Make fine adjustments;

[0145] Step 2.2.2.1: Obtain the relation extraction probability distribution of entity features using the method shown in equation (7). ;

[0146] (7)

[0147] in, This is the classification layer weight matrix. For the classification layer bias term, For a set of relation types The size of this, i.e., the number of relation types. This indicates a random deactivation operation. For activation functions;

[0148] Step 2.2.2.2: Through Calculate the cross-entropy loss as shown in equation (8). ;

[0149] (8)

[0150] in, The number of labeled samples. Indicates the first The true relation type of each sample;

[0151] Step 2.2.2.3: Utilize cross-entropy loss Relation extraction model Make fine adjustments;

[0152] In this embodiment, the real relationship type is "org:founded";

[0153] Step 2.3: Input the unlabeled samples into the fine-tuned relation extraction model. This generates pseudo-labels for unlabeled samples;

[0154] In this embodiment, the pseudo-label for the unlabeled sample is "org:stateorprovince_of_headquarters", which means "the tail entity is the state or province where the head entity's headquarters is located";

[0155] Step 2.4: Scoring using external pseudo-labels and endogenous pseudo-label scoring Obtaining dual-source pseudo-label scores ;

[0156] Step 2.4.1: Input the unlabeled samples and their pseudo-labels into the fine-tuned external pseudo-label scorer. Obtain the exogenous pseudo-label score as shown in equation (9). ;

[0157] (9)

[0158] in, and They represent Output the logarithmic odds for "Yes" and "No";

[0159] Step 2.4.2: Input the unlabeled samples into the fine-tuned relation extraction model. Obtain the endogenous pseudo-label score as shown in equation (10). ;

[0160] (10)

[0161] in, For pseudo-labels of unlabeled samples, the relation extraction model As an endogenous pseudo-label scorer;

[0162] Step 2.4.3: Scoring using external pseudo-labels and endogenous pseudo-label scoring Obtain the dual-source pseudo-label score as shown in equation (11). ;

[0163] (11)

[0164] in, To balance the weights of the two types of pseudo-label scores;

[0165] In this embodiment, ;

[0166] Step 3: Utilize large language models Constructing relational types with labeled samples Relation clusters are generated by using a relation encoder to form relation mapping features for the relation clusters. The dual-source pseudo-label score should be no less than [amount missing]. Unlabeled samples and augmented samples are mapped to form entity mapping features. ; Calculate the semantic correction contrast loss for relation clusters of relation mapping features and entity mapping features. ;

[0167] Step 3.1: Construct relation clusters and extract relation mapping features of the relation clusters using a relation encoder;

[0168] Step 3.1.1: Utilizing a large language model Relationship types for labeled samples Construct relation clusters;

[0169] Step 3.1.1.1: Utilizing a large language model Relationship types for labeled samples Generate single relation descriptions through context learning;

[0170] Step 3.1.1.2: Generate multiple single relation descriptions through iterative loops, grouping those belonging to the same relation type... A single relation describes a set of relations;

[0171] In this embodiment, for the relation type "org:stateorprovince_of_headquarters", which means "the tail entity is the state or province where the head entity's headquarters are located", the relation description in its relation cluster includes: "The <e1>head entity< / e1> is located in or associated with the <e2>tail entity< / e2> The head entity typically represents a place, organization, or entity with a geographical or institutional connection. <e1>head entity< / e1> is located in or associated with a specific place or region represented by the <e2>tail entity< / e2> "." (The head entity is located in or associated with a specific location or area represented by the tail entity); "The <e1>head entity< / e1> is located in or associated with the <e2>tail entity< / e2> "indicating a geographical, institutional, or organizational connection." (The head entity is located in the position represented by the tail entity or is associated with the tail entity, indicating a geographical, institutional, or organizational connection.) <e1>head entity< / e1> is an organization, institution, or entity based in or associated with a specific location, which is identified as the <e2>tail entity< / e2> "(The head entity is an organization, institution, or entity established in or associated with a specific location, which is identified as the tail entity); "The <e1>head entity< / e1> is associated with a specific location,represented by <e2>tail entity< / e2>The head entity is associated with a specific location represented by the tail entity, which often indicates the place where the head entity is based, operates, or is relevant to.

[0172] Step 3.1.2: Use the pre-trained language model BERT as a relation encoder The relation mapping features of the relation clusters shown in Equation (12) are extracted by the relation encoder. ;

[0173] (12)

[0174] in, for and The feature represents the result after splicing. The name of the relation type. For relation type The corresponding relation cluster in the first This relationship explanation The weight matrix of the mapping layer. For the mapping layer bias term, For L2 normalization;

[0175] In this embodiment, , ;

[0176] Step 3.2: Ensure the dual-source pseudo-label score is not less than [amount missing]. Unlabeled samples and augmented samples are mapped to form entity mapping features. ;

[0177] Step 3.2.1: Define the threshold The dual-source pseudo-label score should be no less than Unlabeled samples are considered as reliable pseudo-labeled samples;

[0178] In this embodiment, ;

[0179] Step 3.2.2: Map the trusted pseudo-label samples and the enhanced samples to form entity mapping features using the method shown in Equation (13). ;

[0180] (13)

[0181] Step 3.3: Calculate the semantic correction contrast loss of relation clusters for relation mapping features and entity mapping features using Equation (14). ;

[0182] (14)

[0183] Among them, when hour ,when hour, , This represents the original number of unlabeled samples. Temperature for semantic correction contrast loss of relation clusters For the first Two-source pseudo-label scores for each sample. For the first Pseudo-labels for each sample For a set of relation types, For the size of the relation cluster, For the first Entity mapping features of each sample For relation type The corresponding relation cluster in the first The mapping characteristics described by each relation;

[0184] In this embodiment, ;

[0185] Step 4: Utilize entity contrast mapping features to obtain the adaptive contrast loss for positive and negative samples of unlabeled samples. Combined with adaptive contrastive loss Semantic correction contrast loss for relation clusters and cross-entropy loss Forming a relation extraction model loss ;

[0186] Step 4.1: Map the unlabeled samples and augmented samples to form entity contrast mapping features using the method shown in Equation (15). ;

[0187] (15)

[0188] in, For entity features, The weight matrix of the mapping layer. For mapping layer bias terms;

[0189] Step 4.2: Obtain the adaptive contrast loss for positive and negative samples of unlabeled samples using entity contrast mapping features. ;

[0190] Step 4.2.1: Obtain the positive sample contrast distance of the unlabeled sample using the method shown in Equation (16);

[0191]

[0192] (16)

[0193] in, Temperature for adaptive contrastive learning, For the first Two-source pseudo-label scores for each sample. For the first Entity mapping features of each sample For the first Each sample corresponds to Granularity enhances the entity mapping features of samples. , and The first The and the first The pseudo-labels of the samples;

[0194] Step 4.2.2: Obtain the negative sample contrast distance of the unlabeled sample using the method shown in Equation (17);

[0195]

[0196] (17)

[0197] Step 4.2.3: Obtain the adaptive contrastive loss for positive and negative samples of unlabeled samples using the method shown in Equation (18). ;

[0198] (18)

[0199] in, This represents the number of unlabeled original samples;

[0200] In this embodiment, for unlabeled samples " <e1>Option One< / e1> is not a significant Kansas City area employer, and most of the previously announced reductions involved <e2>California< / e2>"jobs.", positive samples include augmented samples of the unlabeled sample, as well as all unlabeled samples and their corresponding augmented samples with the same pseudo-label "org:stateorprovince_of_headquarters".

[0201] Step 4.3: Obtain the relation extraction model as shown in equation (19) Model loss ;

[0202] (19)

[0203] in, For use in balancing and The weight, To compare the learning loss, For the cross-entropy loss in formula (8), For use in balancing and The weights;

[0204] In this embodiment, , ;

[0205] Step 5: Utilize model loss and cross-entropy loss Iterative training of relation extraction model ;

[0206] Step 5.1: Utilize model loss Fine-tuning the relation extraction model ;

[0207] Step 5.2: Scoring the dual-source pseudo-labels Not less than Trustworthy pseudo-labeled samples are added to the set of labeled samples, and the cross-entropy loss of the updated labeled samples is used. Fine-tuning the relation extraction model ;

[0208] Step 5.3: Execute steps 2.3 to 5.2 in an iterative loop until the loop completes. Until then;

[0209] In this embodiment, ;

[0210] Step 6: Input the unstructured text data into the iteratively trained relation extraction model. To obtain the relation types used for semantic extraction of unstructured text data in low-resource relation extraction tasks;

[0211] To illustrate the effectiveness of the low-resource relation extraction method of this invention, this experiment compares three methods under the same conditions, using the same training set, validation set, and test set.

[0212] The first low-resource relation extraction method utilizes constrained back-translation and latent space interpolation strategies to achieve data augmentation and performs consistency training. The second method is an iterative feedback method based on feature mapping similarity calculation. The third method is the low-resource relation extraction method based on semantic correction and adaptive contrast, as proposed in this invention. The evaluation metrics used are: precision, recall, and F1 score.

[0213] The low-resource relation extraction results are as follows: 50% of the samples in the training set are taken as unlabeled data, and 3%, 10%, and 15% of the samples in the remaining training set are taken as labeled data, respectively. When the labeled data accounts for 3%, 10%, and 15%, respectively, the F1 scores of the first method are 55.80%, 61.30%, and 63.07%; the F1 scores of the second method are 50.83%, 58.94%, and 60.62%; and the F1 scores of our proposed method are 59.74%, 62.48%, and 64.49%. Experiments demonstrate the effectiveness of the proposed low-resource relation extraction method based on semantic correction and adaptive contrast.

[0214] This method maximizes information gain while avoiding semantic drift and effectively reduces the interference of pseudo-label noise on the training process. It can also further regulate the model training process through relation clusters to improve class discrimination and model representation ability. This solves the problems of limited information gain or easy semantic drift in existing data augmentation methods, error propagation caused by erroneous pseudo-labels in self-training methods, and insufficient relation semantic mining.

[0215] The above description is merely a preferred embodiment of the present invention, and the present invention should not be limited to the content disclosed in this embodiment and the accompanying drawings. Any equivalent or modified embodiments made without departing from the spirit of the present invention fall within the scope of protection of the present invention.

Claims

1. A low-resource relation extraction method based on semantic correction adaptive comparison, characterized in that: Includes the following steps, Step 1: Insert structured unlabeled samples into the prompt words In China, large language models are augmented with multi-granular data. The generated initial augmented samples are constrained to form constrained augmented samples; semantic similarity is used to filter the constrained augmented samples in the candidate augmented sample pool to generate augmented samples; Step 2: Utilizing the LoRA parameter-efficient fine-tuning method and the loss of the external pseudo-label scorer for the external pseudo-label scorer. Fine-tune; utilize cross-entropy loss Relation extraction model Fine-tuning was performed; external pseudo-labels were used for scoring. and endogenous pseudo-label scoring Obtaining dual-source pseudo-label scores ; Step 3: Utilize large language models Constructing relational types with labeled samples Relation clusters are generated by using a relation encoder to form relation mapping features for the relation clusters. The dual-source pseudo-label score should be no less than [amount missing]. Unlabeled samples and augmented samples are mapped to form entity mapping features. ; Calculate the semantic correction contrast loss for relation clusters of relation mapping features and entity mapping features. ; Step 4: Utilize entity contrast mapping features to obtain the adaptive contrast loss for positive and negative samples of unlabeled samples. Combined with adaptive contrastive loss Semantic correction contrast loss for relation clusters and cross-entropy loss Forming a relation extraction model loss ; Step 5: Utilize model loss and cross-entropy loss Iterative training of relation extraction model ; Step 6: Input the unstructured text data into the iteratively trained relation extraction model. This allows us to obtain relation types for semantic extraction of unstructured text data in low-resource relation extraction tasks.

2. The low-resource relation extraction method based on semantic correction adaptive comparison as described in claim 1, characterized in that: Step 1 is implemented as follows: Step 1.1: Obtain unstructured text data as the original sample; Step 1.2: Randomly select from the original samples and construct relation types. The mapping relationship with the original samples forms labeled samples; the original samples that were not randomly selected are treated as unlabeled samples. Step 1.3: Use semantic similarity to filter the constrained augmentation samples placed in the candidate augmentation sample pool to generate augmentation samples.

3. The low-resource relation extraction method based on semantic correction adaptive comparison as described in claim 2, characterized in that: Step 1.3 is implemented as follows: Step 1.3.1: Construct a set of prompt words with relational constraints, consisting of token granularity enhancement, clause granularity enhancement, and sentence granularity enhancement. ; Step 1.3.2: Insert the structured, unlabeled samples with the configured structure into the prompt words. In the process, sample format constraints are applied to form constraint-enhanced samples; Step 1.3.3: Use semantic similarity to filter the constrained augmentation samples placed in the candidate augmentation sample pool to generate augmentation samples.

4. The low-resource relation extraction method based on semantic correction adaptive comparison as described in claim 3, characterized in that: The implementation method for step 1.3.2 is as follows: Step 1.3.2.1: Set the structure of the entity position of the unlabeled sample to form a structured unlabeled sample as shown in equation (1); （1） in, , Mark the start and end positions of the header entity respectively. , Mark the start and end positions of the tail entity respectively. For the sample Token; Step 1.3.2.2: Insert structured unlabeled samples into the prompt words. In China, multi-granularity data is used to augment large language models. Generate the initial enhanced sample as shown in equation (2); （2） in, This is the initial augmentation sample for token granularity, which is about to be... Replace "Token" with a synonym; This is the initial enhancement sample at the clause granularity, i.e., modification. Clauses; This serves as the initial sample for sentence-level augmentation, i.e., rewriting sentences using different wording or sentence structures. ; Step 1.3.2.3: Apply sample format constraints to the initial augmented samples to form constrained augmented samples.

5. The low-resource relation extraction method based on semantic correction adaptive comparison as described in claim 3, characterized in that: The implementation method for step 1.3.3 is as follows: Step 1.3.3.1: Construct a candidate augmentation sample pool using the constraint augmentation sample set generated by the iterative loop; Step 1.3.3.2: Use the semantic similarity shown in Equation (3) to filter the constrained enhancement samples in the candidate enhancement sample pool and generate enhancement samples; （3） in, Indicates particle size, Indicates the candidate boosting sample pool, The original unlabeled sample express The indivual Granularity-enhanced samples This represents the semantic similarity based on an independent pre-trained language model, BERT.

6. The low-resource relation extraction method based on semantic correction adaptive comparison as described in claim 1, characterized in that: Step 2 is implemented as follows: Step 2.1: Construct an external pseudo-label scorer The LoRA parameter-efficient fine-tuning method and the loss of the exogenous pseudo-label scorer are used to evaluate the exogenous pseudo-label scorer. Make fine adjustments; Step 2.1.1: Utilize labeled samples based on relation type Construct positive and negative samples; Step 2.1.2: Input positive and negative samples into the external pseudo-label scorer in batches using a random combination. In the process, the loss of the external pseudo-label scorer is obtained in the manner shown in Equation (4); （4） in, The number of labeled samples. To fine-tune the sample size, including Positive samples and One negative sample, Indicates the first The length of the output sequence for each sample for The output probability of a large language model For the first The input sequence of a sample, Indicates the first The nth sample target output sequence Each Token Indicates the first The first sample target output sequence One Token; Step 2.1.3: Utilizing the LoRA parameter-efficient fine-tuning method and the loss of the external pseudo-label scorer for the external pseudo-label scorer. Make fine adjustments; Step 2.2: Construct a relation extraction model Using cross-entropy loss Relation extraction model Make fine adjustments; Step 2.2.1: Construct a relation extraction model Relation extraction model The features extracted from the labeled samples are concatenated into binary tuples to form entity features; Step 2.2.1.1: Utilizing the relation extraction model The labeled samples are used to extract features as shown in Equation (5); （5） in, Indicates the addition of a special token , , , The following are labeled samples. , Embed dimensions into the BERT model; Step 2.2.1.2: Perform binary concatenation on the features of the labeled samples to form entity features as shown in equation (6); （6） in, and Special Token , Corresponding features This indicates a feature concatenation operation. Original sample Entity characteristics; Step 2.2.2: Utilize cross-entropy loss Relation extraction model Make fine adjustments; Step 2.2.2.1: Obtain the relation extraction probability distribution of entity features using the method shown in equation (7). ; （7） in, This is the classification layer weight matrix. For the classification layer bias term, For a set of relation types The size of this, i.e., the number of relation types. This indicates a random deactivation operation. For activation functions; Step 2.2.2.2: Through Calculate the cross-entropy loss as shown in equation (8). ; （8） in, The number of labeled samples. Indicates the first The true relation type of each sample; Step 2.2.2.3: Utilize cross-entropy loss Relation extraction model Make fine adjustments; Step 2.3: Input the unlabeled samples into the fine-tuned relation extraction model. This generates pseudo-labels for unlabeled samples; Step 2.4: Scoring using external pseudo-labels and endogenous pseudo-label scoring Obtaining dual-source pseudo-label scores ; Step 2.4.1: Input the unlabeled samples and their pseudo-labels into the fine-tuned external pseudo-label scorer. Obtain the exogenous pseudo-label score as shown in equation (9). ; （9） in, and They represent Output the logarithmic odds for "Yes" and "No"; Step 2.4.2: Input the unlabeled samples into the fine-tuned relation extraction model. Obtain the endogenous pseudo-label score as shown in equation (10). ; （10） in, For pseudo-labels of unlabeled samples, the relation extraction model As an endogenous pseudo-label scorer; Step 2.4.3: Scoring using external pseudo-labels and endogenous pseudo-label scoring Obtain the dual-source pseudo-label score as shown in equation (11). ; （11） in, To balance the weights of the two types of pseudo-label scores.

7. The low-resource relation extraction method based on semantic correction adaptive comparison as described in claim 1, characterized in that: Step 3 is implemented as follows: Step 3.1: Construct relation clusters and extract relation mapping features of the relation clusters using a relation encoder; Step 3.1.1: Utilizing a large language model Relationship types for labeled samples Construct relation clusters; Step 3.1.1.1: Utilizing a large language model Relationship types for labeled samples Generate single relation descriptions through context learning; Step 3.1.1.2: Generate multiple single relation descriptions through iterative loops, grouping those belonging to the same relation type... A single relation describes a set of relations; Step 3.1.2: Use the pre-trained language model BERT as a relation encoder The relation mapping features of the relation clusters shown in Equation (12) are extracted by the relation encoder. ; （12） in, for and The feature represents the result after splicing. The name of the relation type. For relation type The corresponding relation cluster in the first This relationship explanation The weight matrix of the mapping layer. For the mapping layer bias term, For L2 normalization; Step 3.2: Ensure the dual-source pseudo-label score is not less than [amount missing]. Unlabeled samples and augmented samples are mapped to form entity mapping features. ; Step 3.2.1: Define the threshold The dual-source pseudo-label score should be no less than Unlabeled samples are considered as reliable pseudo-labeled samples; Step 3.2.2: Map the trusted pseudo-label samples and the enhanced samples to form entity mapping features using the method shown in Equation (13). ; （13） Step 3.3: Calculate the semantic correction contrast loss of relation clusters for relation mapping features and entity mapping features using Equation (14). ; （14） Among them, when hour ,when hour, , This represents the original number of unlabeled samples. Temperature for semantic correction contrast loss of relation clusters For the first Two-source pseudo-label scores for each sample. For the first Pseudo-labels for each sample For a set of relation types, For the size of the relation cluster, For the first Entity mapping features of each sample For relation type The corresponding relation cluster in the first The mapping characteristics described by each relation.

8. The low-resource relation extraction method based on semantic correction adaptive comparison as described in claim 1, characterized in that: Step 4 is implemented as follows: Step 4.1: Map the unlabeled samples and augmented samples to form entity contrast mapping features using the method shown in Equation (15). ; （15） in, For entity features, The weight matrix of the mapping layer. For mapping layer bias terms; Step 4.2: Obtain the adaptive contrast loss for positive and negative samples of unlabeled samples using entity contrast mapping features. ; Step 4.3: Obtain the relation extraction model as shown in equation (19) Model loss ; （19） in, For use in balancing and The weight, To compare the learning loss, For the cross-entropy loss in formula (8), For use in balancing and The weight.

9. The low-resource relation extraction method based on semantic correction adaptive comparison as described in claim 8, characterized in that: Step 4.2 is implemented as follows: Step 4.2.1: Obtain the positive sample contrast distance of the unlabeled sample using the method shown in Equation (16); （16） in, Temperature for adaptive contrastive learning, For the first Two-source pseudo-label scores for each sample. For the first Entity mapping features of each sample For the first Each sample corresponds to Granularity enhances the entity mapping features of samples. , and The first The and the first The pseudo-labels of the samples; Step 4.2.2: Obtain the negative sample contrast distance of the unlabeled sample using the method shown in Equation (17); （17） Step 4.2.3: Obtain the adaptive contrastive loss for positive and negative samples of unlabeled samples using the method shown in Equation (18). ; （18） in, This represents the number of unlabeled original samples.

10. The low-resource relation extraction method based on semantic correction adaptive comparison as described in claim 1, characterized in that: Step 5 is implemented as follows: Step 5.1: Utilize model loss Fine-tuning the relation extraction model ; Step 5.2: Scoring the dual-source pseudo-labels Not less than Trustworthy pseudo-labeled samples are added to the set of labeled samples, and the cross-entropy loss of the updated labeled samples is used. Fine-tuning the relation extraction model ; Step 5.3: Execute steps 2.3 to 5.2 in an iterative loop until the loop completes. That's it, until the next time.

Images