A method for extracting few-sample relations based on memory enhancement and hierarchical decision-making

Abstract

This invention relates to the fields of natural language processing and information extraction technology, specifically to a few-sample relation extraction method based on memory enhancement and hierarchical decision-making. The method integrates the original text, structured hints, and learnable vectors into a unified input text sequence. It then determines whether a relationship exists between entities. If a relationship exists, it proceeds to the type enhancement stage, generating a relation type representation. A two-layer memory bank is then constructed to generate context-aware soft-hint vectors. During the training phase, uncertainty is assessed based on the information entropy of the sample prediction probability distribution, and learning weights are dynamically adjusted to optimize the model's ability to distinguish similar relations. During the inference phase, when the decision weights exceed a threshold, a large language model is introduced for joint decision-making, ultimately outputting the relation category. This method effectively solves the problems of low accuracy in identifying unrelated samples and difficulty in distinguishing semantically similar relations in few-sample relation extraction, significantly improving the model's generalization ability and robustness.

Description

Technical Field

[0001] This invention relates to the fields of natural language processing and information extraction technology, specifically to a few-sample relation extraction method based on memory enhancement and hierarchical decision-making. Background Technology

[0002] Relation extraction (RE), a core task in natural language processing, aims to identify semantic relationships between entity pairs from unstructured text. It is a key technology for downstream applications such as knowledge graph construction, information retrieval, and intelligent question answering. Traditional supervised learning methods rely on a large amount of labeled data. However, in real-world application scenarios (such as vertical domains, emergency response, or low-resource languages), acquiring a large amount of labeled data is often costly and time-consuming. To address this, Few-Shot Relation Extraction (FSRE) is dedicated to training models with high generalization ability under conditions of a very small number of labeled samples.

[0003] Although methods based on prompt learning and large language models (LLM) have made initial progress in the FSRE task, existing technologies still face two major challenges: First, the accuracy of identifying unrelated instances is low. In real-world texts, the vast majority of entity pairs do not have semantic relationships, resulting in an imbalance where unrelated categories account for a larger proportion of samples than other relation categories. Existing large language models tend to misclassify unrelated instances as specific relations, reducing their accuracy. Second, distinguishing semantically similar relations is difficult. Some relation types have high semantic similarity (e.g., "per:parent" and "per:spouse", or "org:subsidiary" and "org:member_of"). With limited samples, large language models struggle to capture their subtle semantic boundaries, leading to confusion. To address this, a method for extracting few-sample relations based on memory enhancement and hierarchical decision-making is proposed. Summary of the Invention

[0004] To address the technical problems existing in the prior art, the present invention provides a few-sample relation extraction method based on memory enhancement and hierarchical decision-making.

[0005] To solve the above-mentioned technical problems, the present invention provides the following technical solution: a few-sample relation extraction method based on memory enhancement and hierarchical decision-making, wherein the few-sample relation extraction method includes the following steps:

[0006] S1 integrates the original text, structured hints, and learnable vectors into a unified representation of the input text sequence. ;

[0007] S2, uses a two-stage prompting module to determine and enhance relationships;

[0008] The dual-stage prompting module includes two sub-task stages: relation existence judgment and specific relation type enhancement. In the relation existence judgment stage, it is determined whether there is a relation between entities in the input text sequence. If there is, it enters the specific relation type enhancement stage, which performs specific type enhancement operations on the text sequence after relation existence judgment, incorporates prior knowledge of relation types, and generates relation type representations.

[0009] S3 employs a memory-enhanced soft cue generation module to construct a two-layer memory bank. It retrieves the most relevant information to the current input through memory and uses this information to generate context-aware soft cue vectors.

[0010] S4. Based on the calculation results of steps S1-S3, the small language model is trained. During the model training phase, the entropy-guided loss function is used to measure the uncertainty of the model prediction, dynamically focusing on samples with higher prediction uncertainty, and adaptively adjusting the learning weights.

[0011] S5 uses a trained Small Language Model (SLM) and Large Language Model (LLM) for joint inference. During the inference phase, a hierarchical decision-making strategy is adopted: first, the prediction confidence of the Small Language Model is calculated. If the confidence is low, the Large Language Model is introduced for joint decision-making, and finally the relation category is output.

[0012] Preferably, in step S1, the unified representation Contains original text Hard prompt template and soft prompt placeholder , the original text Hard prompt template and soft prompt placeholder By concatenating functions Concatenate into the input text sequence The calculation formula is:

[0013] ;

[0014] In the formula, This is a vector concatenation operation.

[0015] Preferably, step S2 includes the following steps:

[0016] S21, The encoder is guided to focus on potential semantic associations between entities through a carefully designed input template in order to assess the existence of relationships between entities; the input template is a hard cue template. With soft prompt template Combined sequence templates;

[0017] S22, after determining that relationships exist between entities, an adaptive type enhancement network is used to generate a representation that is highly sensitive to the relationship type, thereby distinguishing different relationship types; the adaptive type enhancement network consists of the normalization operation LayerNorm, the forward propagation operation FFN, and the relationship embedding. The dot product is used to construct the relational type representation after type enhancement. The calculation formula is:

[0018] ;

[0019] In the formula, It is a feedforward neural network; For layer normalization operation; An embedding matrix for relation types is provided, containing prior semantic information for different relation types; For context-sensitive basic representation; This is the feature representation after type enhancement;

[0020] S23, through a conditional feature fusion strategy, dynamically combines features from different stages to generate a relation type representation. ;

[0021] The conditional feature fusion strategy is based on the relationship score predicted by the model and a preset threshold. The size comparison determines whether a relationship exists between entities, and then decides whether to incorporate the feature vectors output by the two-stage prompting module into the type enhancement features. The calculation formula is:

[0022] ;

[0023] In the formula, The feature fusion coefficient; A threshold exists for the relation to determine whether to activate relation type enhancement. The probability that the relation exists;

[0024] During training, the rate of change of loss and the entropy of the predicted probability distribution for each sample are closely monitored. The difficulty of the sample is dynamically evaluated and adjusted accordingly through the feature fusion coefficient λ.

[0025] Preferably, step S21 includes the following steps:

[0026] S211, Sequence from input text Generate context-sensitive underlying representations To capture the subtle semantic features of entities and their context. The calculation formula is:

[0027] ;

[0028] In the formula, For pre-trained language model encoders; For entity mask matrix;

[0029] S212, based on the sensitive underlying representation below The probability of the existence of a relationship is calculated using a nonlinear mapping network. ;when When, it indicates that a relationship exists between entities, where, The relationship threshold is calculated using the following formula:

[0030] ;

[0031] In the formula, For Hadamard product; It is an adaptive attention weight matrix that can dynamically highlight the most relevant features in relation judgment, thereby enhancing the model's ability to capture key information; This is the Sigmoid activation function.

[0032] Preferably, step S3 includes the following steps:

[0033] S31, Construct a two-layer memory: including a static relation prototype memory that stores abstract representations of relation categories. and a dynamic instance memory for storing typical sample features ;

[0034] S32, perform memory retrieval enhancement, generate context-aware soft cue vectors, and inject the generated soft cue vectors into the input sequence. This provides explicit, learnable memory enhancement signals for small language models to improve their ability to discriminate similar relationships.

[0035] Preferably, step S31 includes the following steps:

[0036] S311, Constructing a Static Relation Prototype Memory Calculate the average of all sample features for each relation category in the training set, and store it as an abstract representation of that relation category. The training set is derived from manually labeled instances given in the few-shot relation extraction task. The calculation formula is as follows:

[0037] ;

[0038] In the formula, Let i be the set of all samples in the training set that belong to relation category i; This is the i-th memory item in the static relation prototype memory; Let i be a training sample of relation category i; for The encoded feature vector representation;

[0039] S312, Constructing a dynamic instance memory : Used to preserve the feature representation of typical instances for each relation category, and its initialization process is achieved by sampling representative instances from the training set;

[0040] In the few-shot learning setting, the definition of representative instances has special characteristics: since each relation category contains only K labeled samples, the total number of samples is extremely small. Therefore, the K samples are directly regarded as typical representatives of the category without additional screening or clustering operations. The calculation formula is as follows:

[0041] ;

[0042] In the formula, The embedding vector for a representative instance; For feature enhancement vectors; Context adaptation factor; This is the j-th memory item in the dynamic instance memory.

[0043] Preferably, step S32 includes the following steps:

[0044] S321, the output of step S2 Through linear projection layer Transform into query vector , The calculation formula is:

[0045] ;

[0046] In the formula, The projective weight matrix is ​​a learnable matrix. For projection bias term;

[0047] S322 employs a difference-aware memory retrieval mechanism to retrieve relevant information from a two-layer memory bank. The calculation formula is:

[0048] ;

[0049] ;

[0050] ;

[0051] ;

[0052] In the formula, To query the similarity between the vector and the memory item; For memory items from static relation prototype memory or dynamic instance memory; Functions for relational type compatibility; For the relation type identifier of the memory item; The weights of the static relation prototype memory; Weights for the dynamic instance memory; Temperature factor; This is the index set of the Top-k relation prototypes selected from the static relation prototype memory; This is the index set of the Top-k instances selected from the dynamic instance memory; The index of the selected item in the static relation prototype set; The index of the selected item in the dynamic instance collection; The similarity between the query vector and any memory item belonging to the static relation prototype; The similarity between the query vector and any memory item belonging to a dynamic instance;

[0053] S323, during training, uses the update rate The instance characteristics are updated in real time, thereby dynamically updating the dynamic instance memory to ensure its timeliness. The calculation formula is as follows:

[0054] ;

[0055] In the formula, The updated instance feature vector; The instance feature vector before the update; For update functions;

[0056] S324, representing the retrieved relevant information. Through memory projection matrix Transform to soft suggestion space and match with query vector Add them together to generate a soft cue vector. , The calculation formula is:

[0057] ;

[0058] Soft hint vector Injected into the input sequence .

[0059] Preferably, step S4 includes the following steps:

[0060] S41, using information entropy To measure the uncertainty of a model's predictions, the higher the entropy value, the more uncertain the model's prediction for that sample. The calculation formula is:

[0061] ;

[0062] In the formula, The number of relation categories; Let be the probability that the sample belongs to the i-th relation category; The information entropy for predicting the probability distribution p is used to measure the uncertainty of the model's prediction.

[0063] Predicting normalized entropy values ​​using an auxiliary entropy prediction network. , The calculation formula is:

[0064] ;

[0065] In the formula, Multilayer perceptrons are used to capture the intrinsic relationship between sample representation and prediction uncertainty; The pooled representation of the input text sequence in the last layer of the encoder;

[0066] S42, Dynamically adjusts the focusing parameters to reduce focus loss. This allows the small language model to assign greater weight to samples with high uncertainty during training, and the adjusted focusing parameters... for:

[0067] ;

[0068] In the formula, For hyperparameters;

[0069] For high-entropy samples, Increase the size of the sample, thereby increasing its weight in the total loss.

[0070] S43, Calculate entropy-guided focus loss ,Will Standard cross-entropy loss We perform a weighted summation to obtain the total loss. The calculation formula is:

[0071] ;

[0072] ;

[0073] In the formula, This represents the proportion of standard cross-entropy loss in the total loss. Entropy guides the proportion of focus loss in the total loss, which is used to dynamically adjust the importance of the two loss functions during training; This is a category balancing factor used to assign different weights to categories of similarity. This represents the model's predicted probability of the true label category of the sample.

[0074] Preferably, step S5 includes the following steps:

[0075] S51, using a small language model to generate a preliminary predicted probability distribution. The calculation formula is:

[0076] ;

[0077] In the formula, This is the weight matrix for the classification layer; It is the bias vector; This is a normalized exponential function that converts the output into a probability distribution. For relationship categories; For input samples;

[0078] S52 utilizes LoRA technology for task-adaptive fine-tuning of the Large Language Model (LLM), and introduces a low-rank adaptation matrix into the attention layer of the LLM. The calculation formula is:

[0079] ;

[0080] In the formula, Θ is the frozen parameter of the large language model; It is a low-rank adaptive matrix; The regularization coefficient is used. Functions for large language models; For input samples; Input the category label corresponding to the sample; The Frobenius norm is used for regularization constraints.

[0081] S53, settings defined by task Entity Context Enhancement and decision guidance Multi-level prompt pattern template, multi-level prompt pattern template The calculation formula is:

[0082] ;

[0083] S54, Calculate decision weights based on the uncertainty of SLM prediction. The calculation formula is:

[0084] ;

[0085] In the formula, For balance parameters; It is the entropy function; The maximum probability value in the probability distribution predicted by the SLM model;

[0086] S54, Joint Decision-Making: [This appears to be a fragment of a larger text, possibly related to decision-making or With decision threshold In comparison, if Greater than the threshold If the prediction is positive, then the large language model is activated for prediction; otherwise, the prediction result of the SLM is directly adopted. The calculation formula is as follows:

[0087] .

[0088] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0089] 1. This invention employs a two-stage prompting and hierarchical decision-making approach to accurately identify unrelated samples. By using memory-enhanced soft prompting generation and an entropy-guided loss function, it decouples the judgment of the existence of relationships from the classification of specific types, significantly improving the model's ability to discriminate semantically similar relationships. Furthermore, through a two-layer memory architecture, it integrates abstract relationship prototypes and specific instance features, and with the help of memory retrieval enhancement, it achieves a dual improvement in the recognition rate of unrelated samples and the accuracy of similar relationship classification, providing a more efficient and accurate solution for practical applications.

[0090] 2. This invention addresses the core problem of low accuracy in identifying the "no_relation" category in a few-sample relation extraction task by using a two-stage prompting module. The first stage filters out unrelated samples, and the second stage performs fine-grained classification on the related samples, avoiding interference from a large number of unrelated samples to the relation classifier and significantly improving the recognition rate of unrelated samples.

[0091] 3. This invention introduces a two-layer memory bank architecture. Through a difference-aware memory retrieval mechanism, it retrieves the most relevant historical typical instances and relationship prototypes from the memory bank and converts them into a soft cue vector injection model. This provides explicit prior knowledge assistance for the current decision, effectively solving the problem of difficulty in distinguishing fine-grained similarity relationships (such as "per:parents" and "per:spouse"), and significantly improving the classification accuracy of similarity relationships.

[0092] 4. In this invention, the entropy-guided loss function can adaptively adjust the learning weights according to the prediction uncertainty of the samples, thus optimizing the model's performance on difficult samples. Through the joint decision-making mechanism, it fully combines the efficient computing power of SLM (Small Language Model) and the deep semantic understanding capability of LLM, significantly improving the relation extraction performance in low-sample scenarios while ensuring efficiency. It balances inference efficiency and prediction accuracy, avoids the computational overhead caused by frequent LLM calls, and has significant practical value in resource-constrained scenarios. Attached Figure Description

[0093] Figure 1 This is a flowchart illustrating the few-sample relation extraction method of the present invention;

[0094] Figure 2 This is a schematic diagram of the overall framework of the few-sample relation extraction method of the present invention. Detailed Implementation

[0095] The present invention will be further described below with reference to the accompanying drawings and embodiments, which illustrate the above and other technical features and advantages of the present invention. However, the following embodiments are merely preferred embodiments of the present invention and are not exhaustive.

[0096] Example 1:

[0097] like Figure 1-2 As shown, this invention provides a few-shot relation extraction method based on memory enhancement and hierarchical decision-making. The few-shot relation extraction method includes the following steps:

[0098] S1 integrates the original text, structured hints, and learnable vectors into a unified representation of the input text sequence. ;

[0099] S2, uses a two-stage prompting module to determine and enhance relationships;

[0100] The two-stage prompting module includes two sub-task stages: relation existence judgment and specific relation type enhancement. In the relation existence judgment stage, it is determined whether there is a relation between the entities in the input text sequence. If there is, it enters the specific relation type enhancement stage, which performs specific type enhancement operations on the text sequence after relation existence judgment, incorporates prior knowledge of relation types, and generates relation type representations.

[0101] S3 employs a memory-enhanced soft cue generation module to construct a two-layer memory bank. It retrieves the most relevant information to the current input through memory and uses this information to generate context-aware soft cue vectors.

[0102] S4. Based on the calculation results of steps S1-S3, train the small language model (such as Roberta), that is, calculate the loss based on the forward calculation results of steps S1-S3 and update the parameters according to the loss; during the model training stage, use the entropy-guided loss function to measure the uncertainty of the model prediction, dynamically focus on samples with high prediction uncertainty, and adaptively adjust the learning weights.

[0103] S5 uses a trained small language model and a large language model (such as LLaMA-3) for joint reasoning. During the reasoning stage, a hierarchical decision-making strategy is adopted: first, the prediction confidence of the small language model is calculated. If the confidence is low, the large language model is introduced for joint decision-making, and finally the relation category is output.

[0104] In this embodiment, in step S1, a unified representation is used. Contains original text Hard prompt template and soft prompt placeholder , the original text Hard prompt template and soft prompt placeholder By concatenating functions Concatenate into the input text sequence The calculation formula is:

[0105] ;

[0106] In the formula, This represents a vector concatenation operation.

[0107] In this embodiment, step S2 includes the following steps:

[0108] S21 guides the encoder to focus on potential semantic relationships between entities through a carefully designed input template to assess the existence of relationships between entities; the input template is a hard-suggestion template. With soft prompt template Combined sequence templates;

[0109] S22, after determining that relationships exist between entities, an adaptive type enhancement network is used to generate representations highly sensitive to relationship types, thereby distinguishing different relationship types; the adaptive type enhancement network consists of the normalization operation LayerNorm, the forward propagation operation FFN, and relation embedding. The dot product is used to construct the relation; the relation type depends on the specific dataset used, and the TACRED dataset, which contains 42 relation types (such as per:parents, org:founded, no_relation, etc.), is used. The relation type representation is enhanced by type enhancement. The calculation formula is:

[0110] ;

[0111] In the formula, It is a feedforward neural network; For layer normalization operation; An embedding matrix for relation types is provided, containing prior semantic information for different relation types; For context-sensitive basic representation; This is the feature representation after type enhancement;

[0112] S23, through a conditional feature fusion strategy, dynamically combines features from different stages to generate a relation type representation. ;

[0113] In the first stage, the contextual representation is obtained by encoding the sequence in the input text. The type-sensitive augmented representation generated in the second stage ;

[0114] The conditional feature fusion strategy specifically refers to comparing the relationship score predicted by the model with the preset threshold θ to determine whether there is a relationship between entities, and then deciding whether to integrate the feature vector finally output by the two-stage prompting module into the type enhancement feature.

[0115] The calculation formula is:

[0116] ;

[0117] In the formula, The feature fusion coefficient; A threshold exists for the relation to determine whether to activate relation type enhancement. The probability that the relation exists;

[0118] During training, the rate of change of loss and the entropy of the predicted probability distribution for each sample are closely monitored. The difficulty of the sample is dynamically evaluated and adjusted accordingly through the feature fusion coefficient λ.

[0119] In this embodiment, step S21 includes the following steps:

[0120] S211, Sequence from input text Generate context-sensitive underlying representations To capture the subtle semantic features of entities and their context. The calculation formula is:

[0121] ;

[0122] In the formula, For pre-trained language model encoders (e.g., Roberta); For entity mask matrix;

[0123] S212, based on the sensitive underlying representation below The probability of the existence of a relationship is calculated using a nonlinear mapping network. ,when When, it indicates that a relationship exists between entities, where, The relationship threshold can be set to 0.5 in the experiment, and the calculation formula is as follows:

[0124] ;

[0125] In the formula, For Hadamard product; It is an adaptive attention weight matrix that can dynamically highlight the most relevant features in relation judgment, thereby enhancing the model's ability to capture key information; This is the Sigmoid activation function.

[0126] In this embodiment, step S3 includes the following steps:

[0127] S31, Construct a two-layer memory: including a static relation prototype memory that stores abstract representations of relation categories. and a dynamic instance memory for storing typical sample features ;

[0128] S32, perform memory retrieval enhancement, generate context-aware soft cue vectors, and inject the generated soft cue vectors into the input sequence. This provides explicit, learnable memory enhancement signals for small language models to improve their ability to discriminate similar relationships.

[0129] In this embodiment, step S31 includes the following steps:

[0130] S311, Constructing a Static Relation Prototype Memory Calculate the average of all sample features for each relation category in the training set, and store it as an abstract representation of that relation category. The training set is derived from manually labeled instances given in the few-shot relation extraction task, such as K instances for each relation category (K=8, 16, 32). The calculation formula is as follows:

[0131] ;

[0132] In the formula, Let i be the set of all samples in the training set that belong to relation category i; This is the i-th memory item in the static relation prototype memory; Let i be a training sample of relation category i; for The encoded feature vector representation;

[0133] S312, Constructing a dynamic instance memory : Used to preserve the feature representation of typical instances for each relation category, the initialization process of the dynamic instance memory is achieved by sampling representative instances from the training set;

[0134] In the few-shot learning setting, the definition of representative instances has special characteristics: since each relation category contains only K labeled samples, the total number of samples is extremely small. Therefore, the K samples are directly regarded as typical representatives of the category without additional screening or clustering operations. The calculation formula is as follows:

[0135] ;

[0136] In the formula, The embedding vector for a representative instance; For feature enhancement vectors; Context adaptation factor; This is the j-th memory item in the dynamic instance memory.

[0137] In this embodiment, step S32 includes the following steps:

[0138] S321, the output of step S2 Through linear projection layer Transform into query vector , The calculation formula is:

[0139] ;

[0140] In the formula, The projective weight matrix is ​​a learnable matrix. For projection bias term;

[0141] S322 employs a difference-aware memory retrieval mechanism to retrieve relevant information from a two-layer memory bank. The calculation formula is:

[0142] ;

[0143] ;

[0144] ;

[0145] ;

[0146] In the formula, To query the similarity between the vector and the memory item; For memory items from static relation prototype memory or dynamic instance memory; Functions for relational type compatibility; For the relation type identifier of the memory item; The weights of the static relation prototype memory; Weights for the dynamic instance memory; Temperature factor; This is the index set of the Top-k relation prototypes selected from the static relation prototype memory; This is the index set of the Top-k instances selected from the dynamic instance memory; The index of the selected item in the static relation prototype set; The index of the selected item in the dynamic instance collection; The similarity between the query vector and any memory item belonging to the static relation prototype; The similarity between the query vector and any memory item belonging to a dynamic instance;

[0147] S323, during training, uses the update rate The instance characteristics are updated in real time, thereby dynamically updating the dynamic instance memory to ensure its timeliness. The calculation formula is as follows:

[0148] ;

[0149] In the formula, The updated instance feature vector; The instance feature vector before the update; For update functions;

[0150] S324, representing the retrieved relevant information. Through memory projection matrix Transform to soft suggestion space and match with query vector Add them together to generate a soft cue vector. , The calculation formula is:

[0151] ;

[0152] Soft hint vector Injected into the input sequence ;

[0153] In this embodiment, step S4 includes the following steps:

[0154] S41 uses information entropy to measure the uncertainty of the model's prediction. The higher the entropy value, the more uncertain the model's prediction for that sample. The calculation formula is as follows:

[0155] ;

[0156] In the formula, The number of relation categories; Let be the probability that the sample belongs to the i-th relation category; The information entropy for predicting the probability distribution p is used to measure the uncertainty of the model's prediction.

[0157] Predicting normalized entropy values ​​using an auxiliary entropy prediction network. , The calculation formula is:

[0158] ;

[0159] In the formula, A multilayer perceptron for capturing the intrinsic relationship between sample representation and prediction uncertainty; The pooled representation of the input text sequence in the last layer of the encoder;

[0160] S42, Dynamically adjusts the focusing parameters to reduce focus loss. This allows the small language model to assign greater weight to samples with high uncertainty during training, and the adjusted focusing parameters... for:

[0161] ;

[0162] In the formula, For hyperparameters;

[0163] For high-entropy samples, Increase the weight of the sample in the total loss, thus forcing the model to pay more attention to those samples that are difficult to distinguish or whose prediction is uncertain when updating the gradient, thereby improving the model's ability to overcome difficult samples;

[0164] S43, Calculate entropy-guided focus loss ,Will Standard cross-entropy loss We perform a weighted summation to obtain the total loss. This is used to guide the backpropagation and parameter updates of the model, enhancing the learning of difficult samples while ensuring the basic classification accuracy. The calculation formula is:

[0165] ;

[0166] ;

[0167] In the formula, This represents the proportion of standard cross-entropy loss in the total loss. Entropy guides the proportion of focus loss in the total loss, which is used to dynamically adjust the importance of the two loss functions during training; This is a category balancing factor used to assign different weights to categories of similarity. This represents the model's predicted probability of the true label category of the sample.

[0168] In this embodiment, step S5 includes the following steps:

[0169] S51, using a small language model to generate a preliminary predicted probability distribution. The calculation formula is:

[0170] ;

[0171] In the formula, This is the weight matrix for the classification layer; It is the bias vector; This is a normalized exponential function that converts the output into a probability distribution. For relationship categories; For input samples;

[0172] S52 uses LoRA technology to perform task-adaptive fine-tuning of the large language model LLM (Llama-3), and introduces a low-rank adaptation matrix into the attention layer of the large language model LLM (Llama-3). The calculation formula is:

[0173] ;

[0174] In the formula, Θ is the frozen parameter of the large language model; It is a low-rank adaptive matrix; The regularization coefficient is used. Functions for large language models; For input samples; Input the category label corresponding to the sample; For Frobenius norm (used for regularization constraints);

[0175] S53, settings defined by task Entity Context Enhancement and decision guidance Multi-level prompt pattern template, multi-level prompt pattern template The calculation formula is:

[0176] ;

[0177] S54, Calculate decision weights based on the uncertainty of SLM prediction. The calculation formula is:

[0178] ;

[0179] In the formula, For balance parameters; It is the entropy function; The maximum probability value in the probability distribution predicted by the SLM model;

[0180] S54, Joint Decision-Making: [This appears to be a fragment of a larger text, possibly related to decision-making or With decision threshold In comparison, if Greater than the threshold If the prediction is positive, then the large language model is activated for prediction; otherwise, the prediction result of SLM is directly adopted. The specific calculation formula is as follows:

[0181] .

[0182] Example 2:

[0183] Sample input: "She was in her early teens when her mom told her dad hecouldn't see his daughters if he continued taking drugs."

[0184] Heading entity: her (referring to the daughter);

[0185] Tail entity: his (referring to father);

[0186] SLM preliminary prediction: It may predict "per:parents" (parents 0.4) or "per:children" (children 0.31), with a confidence level lower than the experimental setting hierarchical decision threshold of 0.6, triggering LLM joint decision-making;

[0187] LLM analysis: Based on the context "mom told her dad" and "see his daughters", LLM can accurately infer that her is his daughter;

[0188] Final output: per:parents (parental relationships); confidence level 0.82.

[0189] The above are merely preferred embodiments of the present invention and are illustrative in nature, not restrictive. Those skilled in the art will understand that many changes, modifications, and even equivalents can be made within the spirit and scope defined by the claims of the present invention, all of which will fall within the protection scope of the present invention.

Claims

1. A few-sample relation extraction method based on memory enhancement and hierarchical decision-making, characterized in that, The few-sample relation extraction method includes the following steps: S1 integrates the original text, structured hints, and learnable vectors into a unified representation of the input text sequence. ; S2, uses a two-stage prompting module to determine and enhance relationships; The dual-stage prompting module includes two sub-task stages: relation existence judgment and specific relation type enhancement. In the relation existence judgment stage, it is determined whether there is a relation between entities in the input text sequence. If there is, it enters the specific relation type enhancement stage, which performs specific type enhancement operations on the text sequence after relation existence judgment, incorporates prior knowledge of relation types, and generates relation type representations. S3 employs a memory-enhanced soft cue generation module to construct a two-layer memory bank. It retrieves the most relevant information to the current input through memory and uses this information to generate context-aware soft cue vectors. S4. Based on the calculation results of steps S1-S3, the small language model is trained. During the model training phase, the entropy-guided loss function is used to measure the uncertainty of the model prediction, dynamically focusing on samples with higher prediction uncertainty, and adaptively adjusting the learning weights. S5 uses a trained small language model and a large language model for joint reasoning. During the reasoning stage, a hierarchical decision-making strategy is adopted: first, the prediction confidence of the small language model is calculated. If the confidence is low, the large language model is introduced for joint decision-making, and finally the relation category is output.

2. The few-sample relation extraction method based on memory enhancement and hierarchical decision-making as described in claim 1, characterized in that, In step S1, the unified representation Contains original text Hard prompt template and soft prompt placeholder , the original text Hard prompt template and soft prompt placeholder By concatenating functions Concatenate into the input text sequence The calculation formula is: ； In the formula, This is a vector concatenation operation.

3. The few-sample relation extraction method based on memory enhancement and hierarchical decision-making as described in claim 1, characterized in that, Step S2 includes the following steps: S21, The encoder is guided to focus on potential semantic associations between entities through a carefully designed input template in order to assess the existence of relationships between entities; the input template is a hard cue template. With soft prompt template Combined sequence templates; S22, after determining that relationships exist between entities, an adaptive type enhancement network is used to generate a representation that is highly sensitive to the relationship type, thereby distinguishing different relationship types; the adaptive type enhancement network consists of the normalization operation LayerNorm, the forward propagation operation FFN, and the relationship embedding. The dot product is used to construct the relational type representation after type enhancement. The calculation formula is: ； In the formula, It is a feedforward neural network; For layer normalization operation; An embedding matrix for relation types is provided, containing prior semantic information for different relation types; For context-sensitive basic representation; This is the feature representation after type enhancement; S23, through a conditional feature fusion strategy, dynamically combines features from different stages to generate a relation type representation. ; The conditional feature fusion strategy is based on the relationship score predicted by the model and a preset threshold. The size comparison determines whether a relationship exists between entities, and then decides whether to incorporate the feature vectors output by the two-stage prompting module into the type enhancement features. The calculation formula is: ； In the formula, The feature fusion coefficient; A threshold exists for the relation to determine whether to activate relation type enhancement. The probability that the relation exists; During training, the rate of change of loss and the entropy of the predicted probability distribution for each sample are closely monitored. The difficulty of the sample is dynamically evaluated and adjusted accordingly through the feature fusion coefficient λ.

4. The few-sample relation extraction method based on memory enhancement and hierarchical decision-making as described in claim 1, characterized in that, Step S21 includes the following steps: S211, Sequence from input text Generate context-sensitive underlying representations To capture the subtle semantic features of entities and their context. The calculation formula is: ； In the formula, For pre-trained language model encoders; For entity mask matrix; S212, based on the sensitive underlying representation below The probability of the existence of a relationship is calculated using a nonlinear mapping network. ;when When, it indicates that a relationship exists between entities, where, The relationship threshold is calculated using the following formula: ； In the formula, For Hadamard product; It is an adaptive attention weight matrix that can dynamically highlight the most relevant features in relation judgment, thereby enhancing the model's ability to capture key information; This is the Sigmoid activation function.

5. The few-sample relation extraction method based on memory enhancement and hierarchical decision-making as described in claim 1, characterized in that, Step S3 includes the following steps: S31, Construct a two-layer memory: including a static relation prototype memory that stores abstract representations of relation categories. and a dynamic instance memory for storing typical sample features ; S32, perform memory retrieval enhancement, generate context-aware soft cue vectors, and inject the generated soft cue vectors into the input sequence. This provides explicit, learnable memory enhancement signals for small language models to improve their ability to discriminate similar relationships.

6. The few-sample relation extraction method based on memory enhancement and hierarchical decision-making as described in claim 1, characterized in that, Step S31 includes the following steps: S311, Constructing a Static Relation Prototype Memory Calculate the average of all sample features for each relation category in the training set, and store it as an abstract representation of that relation category. The training set is derived from manually labeled instances given in the few-shot relation extraction task. The calculation formula is as follows: ； In the formula, Let i be the set of all samples in the training set that belong to relation category i; This is the i-th memory item in the static relation prototype memory; Let i be a training sample of relation category i; for The encoded feature vector representation; S312, Constructing a dynamic instance memory : Used to preserve the feature representation of typical instances for each relation category, and its initialization process is achieved by sampling representative instances from the training set; In the few-shot learning setting, the definition of representative instances has special characteristics: since each relation category contains only K labeled samples, the total number of samples is extremely small. Therefore, the K samples are directly regarded as typical representatives of the category without additional screening or clustering operations. The calculation formula is as follows: ； In the formula, The embedding vector for a representative instance; For feature enhancement vectors; Context adaptation factor; This is the j-th memory item in the dynamic instance memory.

7. The few-sample relation extraction method based on memory enhancement and hierarchical decision-making as described in claim 1, characterized in that, Step S32 includes the following steps: S321, the output of step S2 Through linear projection layer Transform into query vector , The calculation formula is: ； In the formula, The projective weight matrix is ​​a learnable matrix. For projection bias term; S322 employs a difference-aware memory retrieval mechanism to retrieve relevant information from a two-layer memory bank. The calculation formula is: ； ； ； ； In the formula, To query the similarity between the vector and the memory item; For memory items from static relation prototype memory or dynamic instance memory; Functions for relational type compatibility; For the relation type identifier of the memory item; The weights of the static relation prototype memory; Weights for the dynamic instance memory; Temperature factor; This is the index set of the Top-k relation prototypes selected from the static relation prototype memory; This is the index set of the Top-k instances selected from the dynamic instance memory; The index of the selected item in the static relation prototype set; The index of the selected item in the dynamic instance collection; The similarity between the query vector and any memory item belonging to the static relation prototype; The similarity between the query vector and any memory item belonging to a dynamic instance; S323, during training, uses the update rate The instance characteristics are updated in real time, thereby dynamically updating the dynamic instance memory to ensure its timeliness. The calculation formula is as follows: ； In the formula, The updated instance feature vector; The instance feature vector before the update; For update functions; S324, representing the retrieved relevant information. Through memory projection matrix Transform to soft suggestion space and match with query vector Add them together to generate a soft cue vector. , The calculation formula is: ； Soft hint vector Injected into the input sequence .

8. The few-sample relation extraction method based on memory enhancement and hierarchical decision-making as described in claim 1, characterized in that, Step S4 includes the following steps: S41, using information entropy To measure the uncertainty of a model's predictions, the higher the entropy value, the more uncertain the model's prediction for that sample. The calculation formula is: ； In the formula, The number of relation categories; Let be the probability that the sample belongs to the i-th relation category; The information entropy for predicting the probability distribution p is used to measure the uncertainty of the model's prediction. Predicting normalized entropy values ​​using an auxiliary entropy prediction network. , The calculation formula is: ； In the formula, Multilayer perceptrons are used to capture the intrinsic relationship between sample representation and prediction uncertainty; The pooled representation of the input text sequence in the last layer of the encoder; S42, Dynamically adjusts the focusing parameters to reduce focus loss. This allows the small language model to assign greater weight to samples with high uncertainty during training, and the adjusted focusing parameters... for: ； In the formula, For hyperparameters; For high-entropy samples, Increase the size of the sample, thereby increasing its weight in the total loss. S43, Calculate entropy-guided focus loss ,Will Standard cross-entropy loss We perform a weighted summation to obtain the total loss. The calculation formula is: ； ； In the formula, This represents the proportion of standard cross-entropy loss in the total loss. Entropy guides the proportion of focus loss in the total loss, which is used to dynamically adjust the importance of the two loss functions during training; This is a category balancing factor used to assign different weights to categories of similarity. This represents the model's predicted probability of the true label category of the sample.

9. The few-sample relation extraction method based on memory enhancement and hierarchical decision-making as described in claim 1, characterized in that, Step S5 includes the following steps: S51, using a small language model to generate a preliminary predicted probability distribution. The calculation formula is: ； In the formula, This is the weight matrix for the classification layer; It is the bias vector; This is a normalized exponential function that converts the output into a probability distribution. For relationship categories; For input samples; S52 utilizes LoRA technology for task-adaptive fine-tuning of the Large Language Model (LLM), and introduces a low-rank adaptation matrix into the attention layer of the LLM. The calculation formula is: ； In the formula, Θ is the frozen parameter of the large language model; It is a low-rank adaptive matrix; The regularization coefficient is used. Functions for large language models; For input samples; Input the category label corresponding to the sample; The Frobenius norm is used for regularization constraints. S53, settings defined by task Entity Context Enhancement and decision guidance Multi-level prompt pattern template, multi-level prompt pattern template The calculation formula is: ； S54, Calculate decision weights based on the uncertainty of SLM prediction. The calculation formula is: ； In the formula, For balance parameters; It is the entropy function; The maximum probability value in the probability distribution predicted by the SLM model; S54, Joint Decision-Making: [This appears to be a fragment of a larger text, possibly related to decision-making or With decision threshold In comparison, if Greater than the threshold If the prediction is positive, then the large language model is activated for prediction; otherwise, the prediction result of the SLM is directly adopted.

Images