Dynamic prototype coding and prompt self-construction method for zero sample relation extraction

Abstract

The invention discloses a dynamic prototype coding and prompt self-construction method for zero sample relation extraction. The method comprises the following steps: S1, defining a task; s2, adaptive prompt words are generated through a dynamic prompt generator, and a dynamic prompt template is constructed; s3, dynamically generating and aggregating context-related prototype representations through a dynamic prototype aggregator; and S4, training and testing. While the sentences and prototype semantics are accurately aligned, the distinguishing ability of the model to complex relationships and similar relationships is improved.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, specifically to a dynamic prototype encoding and prompting self-construction method for zero-sample relation extraction. Background Technology

[0002] The goal of the ZeroSRE task is to identify new relationships without training instances. Existing ZeroSRE research is mainly divided into three categories: classification-based methods, generative methods, and prototype-based methods.

[0003] Among them, the classification-based approach models the ZeroSRE task as a multi-class classification problem, uses a pre-trained model to encode text and relation descriptions, and performs classification prediction by comparing the semantic similarity between sentences and relation descriptions. This method is highly dependent on relation descriptions and has limited generalization ability.

[0004] Generative methods directly generate relation labels using generative models without requiring an explicit classifier. However, these methods are susceptible to textual noise, resulting in insufficient accuracy and stability. Prototype-based methods, as one of the mainstream approaches, model the ZeroSRE task as a semantic matching problem. They align sentences with their corresponding relation descriptions in a vector space and predict invisible relations by minimizing the distance between the sentence embedding and the corresponding prototype embedding. Because these methods only focus on sentence-level representations, they perform poorly with similar and complex relations.

[0005] RE-Matching proposes a fine-grained semantic matching method that decomposes sentence-level similarity scores into entity matching scores and context matching scores, allowing the model to focus more on the differences between specific features in a sentence. However, it still has limitations in handling similarity relationships and complexities. AlignRE constructs more robust prototype representations using side information beyond relational descriptions and constructs a unified prompt template for sentences to reduce the encoding difference between the prompt and the prototype. However, its fixed prompt template is difficult to adapt to diverse language expressions, limiting the model's generalization ability. CE-DA improves the prototype representation ability to some extent by aggregating side information, but its aggregation strategy is simple and cannot fully model the semantic relationships and importance differences between different information. Its prototype representation is not well adapted to the ZSRE task.

[0006] Recently, ZeroSRE methods based on large language models have made significant progress. They provide a new approach to ZeroSRE tasks by leveraging the multi-turn dialogue capabilities of LLMs for feedback-driven sample augmentation and making predictions without labeled data. However, this approach also comes with high computational costs.

[0007] While existing prototype-based zero-shot relation extraction methods have achieved good results by utilizing edge information and constructing cue templates, these methods typically rely on static prototype encoding and fixed cue templates. This makes them difficult to handle complex or semantically similar relation types, and they are prone to issues with inaccurate alignment between the prototype and the context semantics. Summary of the Invention

[0008] This invention aims to solve the technical problems existing in the prior art. In particular, it innovatively proposes a dynamic prototype encoding and prompt self-construction method for zero-sample relation extraction, which improves the model's ability to distinguish between complex and similar relations while accurately aligning sentence and prototype semantics.

[0009] To achieve the above objectives, this invention provides a dynamic prototype encoding and prompt self-construction method for zero-sample relation extraction, comprising the following steps:

[0010] S1: Define the task;

[0011] S2: Generate adaptive prompt words through a dynamic prompt generator to build dynamic prompt templates;

[0012] S3: Dynamically generate and aggregate context-dependent prototype representations through a dynamic prototype aggregator;

[0013] S4: Training and Testing.

[0014] In the above scheme, step S1 includes:

[0015] S1-1: Define visible relationships, invisible relationships, and visible datasets;

[0016] set up Let be a visible relation, and represent the set of trained relations, where , Indicates the first One visible relationship;

[0017] set up Let be an invisible relation, representing the set of untrained relations, where , Indicates the first An invisible relationship;

[0018] set up The visible dataset is, This is an invisible dataset.

[0019] S1-2: Train model M using the visible dataset;

[0020] S1-3: Test model M using an invisible dataset.

[0021] In the above scheme, step S2 includes:

[0022] S2-1: Input a sentence and mark it;

[0023] S2-2: A prompt template for building a T5 model;

[0024] The dynamic prompt generator generates verb phrases based on entity tags, automatically constructing prompts with [MASK] tags;

[0025] S2-3: Input sentence Convert to input for the T5 model;

[0026] S2-4: Predict relational phrases based on the generated prompts and the input sentence using the T5 model;

[0027] S2-5: Optimize the relational phrases predicted by the T5 model using the Beam Search algorithm;

[0028] The formula is as follows:

[0029] ;

[0030] in, This indicates the decoding status at the current time step. Indicates based on the current decoding state Perform a beam search operation. Indicates the candidate sequence { Take the sequence that makes the following expression the largest. For the beam search width, It is the first in the candidate sequence Candidate phrases for each position, Indicates the first Word in front of arrive Probability product under given conditions Given all the preceding words, the candidate phrase is... The probability of;

[0031] S2-6: After obtaining the optimal pre-selected words, construct a personalized prompt template for each input sentence;

[0032] S2-7: The hidden state, calculated by averaging the three masks of the cue template, is used as the final sentence representation, with the following formula:

[0033] ;

[0034] in, This represents the features obtained from the final aggregation. This indicates a pooling operation. This indicates the feature marked with [MASK] corresponding to the head entity. This indicates the feature marked with [MASK] corresponding to the tail entity. The feature marked with [MASK] represents the relation.

[0035] In the above scheme, the format after conversion in step S2-3 is as follows:

[0036] ;

[0037] in, , Each represents a sentence The head and tail entities in the middle, Indicates splicing.

[0038] In the above scheme, step S2-1 includes the following:

[0039] Enter sentence ,in, This represents the input sentence. Sentence The first token in the middle, Sentence The nth token;

[0040] Through four special tokens , , and To mark sentences The positions of the head and tail entities are obtained. as follows:

[0041] ;

[0042] in, This indicates the processed sentence. , , These represent the 1st, 2nd, and nth tokens in the original sentence, respectively. The starting token representing the header entity. This represents the original word content corresponding to the head entity. This indicates the end token of the header entity. The starting token representing the tail entity. The original word content representing the tail entity. This represents the end token of the tail entity.

[0043] In the above scheme, step S3 includes the following steps:

[0044] S3-1: Relational prototype processing;

[0045] S3-2: Text Feature Extraction;

[0046] S3-3: Aggregate the features processed in steps S3-1 and S3-2 to obtain the final prototype embedding;

[0047] In the above scheme, step S3-1 includes the following steps:

[0048] S3-1-1: Input Relationship tags ,describe and three randomly selected tag names aliases , , The information will be encoded using the same BERT encoder for all five types of prototype information mentioned above.

[0049] S3-1-2: Stack the embedding vectors corresponding to the above 5 types of prototype information to form the prototype feature matrix as follows:

[0050] ;

[0051] in, It is a relationship The corresponding prototype feature matrix, For the hidden layer dimension, For encoder, This is a max pooling operation;

[0052] S3-1-2: Capturing key information within a message through a multi-head self-attention mechanism;

[0053] The prototype feature matrix The input unit of the self-attention mechanism is fed into a linear projection to obtain the Query, Key, and Value as follows:

[0054] ;

[0055] in, All are learnable parameter matrices. For querying the matrix, The key matrix, It is a value matrix;

[0056] Then, the scaled dot product attention is computed in parallel within the multi-head space to dynamically capture the importance of different edge information.

[0057] S3-1-3: Normalization is performed using the LayerNorm layer;

[0058] After attention-weighted aggregation and residual connections, the context-adaptive relation prototype representation is obtained as follows:

[0059] ;

[0060] in, For context-adaptive prototype representation, This indicates the number of heads in the multi-head attention mechanism, where each head independently calculates attention to capture semantic relationships in different subspaces. This is a normalized exponential function used to calculate the attention weights between nodes. The dimension of each head is used to scale the dot product to stabilize the gradient; For relationship The query matrix, For relationship The key matrix, Key matrix transpose, For relationship The value matrix, It is a relationship The corresponding prototype feature matrix.

[0061] In the above scheme, step S3-2 includes the following steps:

[0062] S3-2-1: Input a sentence containing the target entity;

[0063] S3-2-2: Encode the input sentence using the BERT encoder to extract a context-aware semantic vector representation;

[0064] S3-2-3: The information encoded in step S3-2 is processed through the Pooling layer to extract global text features;

[0065] In the above scheme, step S4 includes the following steps:

[0066] S4-1: Obtain sentence representation and prototype embedding;

[0067] S4-2: Determine the prototype of the positive sample relation corresponding to the current sample. ;

[0068] S4-3: Select a set of prototype relationships between several negative samples that are unrelated to positive samples. ;

[0069] S4-4: Setting Hyperparameters ;

[0070] S4-5: Calculate similarity;

[0071] S4-5-1: Calculate sentences Compared with positive sample prototype similarity ;

[0072] ;

[0073] in, Sentences indicating calculation Compared with positive sample prototype similarity

[0074] S4-5-2: Calculate sentences With each negative sample prototype The value with the highest similarity ;

[0075] ;

[0076] in, To obtain the maximum value, To ask for a sentence With negative sample prototype Similarity;

[0077] S4-6: Calculate marginal loss;

[0078] The calculation is performed using the following function:

[0079] ;

[0080] in, The loss value. This represents the currently input sentence. Embed the positive sample prototype corresponding to the current input sentence. For negative sample embedding, The negative samples representing random sampling are: indivual, The hyperparameter used to control the minimum distance between positive and negative samples; Indicates the number of samples;

[0081] S4-7: When L=0, the positive and negative sample discrimination is sufficient and there is no need to update the model; when L>0, the positive and negative sample discrimination is insufficient and the model parameters need to be updated through backpropagation.

[0082] In summary, the beneficial effects of this invention are as follows: 1. This technical solution uses a unified encoding structure for sentences and prototypes, and brings their semantics closer by sharing a representation space; the dynamic prototype encoding method enables the prototype to adaptively adjust according to the context, thereby effectively enhancing the model's ability to model complex relationships and adapt to unseen relationships; 2. The automated prompt generator can automatically generate verb-type prompt phrases through the T5 model, and refine the semantic characterization granularity of sentences by utilizing the prior knowledge of the pre-trained model, providing rich semantic expressions for distinguishing similar relationships; 3. The dynamic prototype aggregator can dynamically generate and aggregate context-related prototype representations, and perform context-aware interactive modeling of prototype features composed of tag names, descriptive statements, and multiple aliases through a self-attention structure, realizing the dynamic fusion of different semantic components, automatically capturing the association of various side information, and adaptively adjusting the prototype representation according to the input context, thereby effectively improving the model's semantic expression ability and robustness. Attached Figure Description

[0083] Figure 1 This is a structural diagram of the model of the present invention.

[0084] Figure 2 This represents the change in F1 score for each model on the FewRel dataset with varying numbers of unknown relations.

[0085] Figure 3 In Figure a, we see the embedding space comparison of our technical solution on unseen relations (t-SNE visualization), and in Figure b, we see the embedding space comparison of the baseline model on unseen relations (t-SNE visualization).

[0086] Figure 4 This represents the change in the matching F1 score of each model on the WikiZSL dataset under different numbers of unknown relations. Detailed Implementation

[0087] The present invention will be further described below with reference to the embodiments and accompanying drawings:

[0088] like Figures 1-4 As shown, a dynamic prototype encoding and prompt self-construction method for zero-sample relation extraction includes the following steps:

[0089] S1: Define the task;

[0090] S1-1: Define visible relationships, invisible relationships, and visible datasets;

[0091] set up Let be a visible relation, and represent the set of trained relations, where , Indicates the first One visible relationship;

[0092] set up Let be an invisible relation, representing the set of untrained relations, where , Indicates the first An invisible relationship;

[0093] set up The visible dataset is, This is an invisible dataset.

[0094] in,

[0095] The goal of zero-sample relation extraction is to extract from visible relations Learn knowledge and extend it to invisible relationships superior. and There is no overlap between them, that is .

[0096] S1-2: Train model M using the visible dataset;

[0097] Design model M, and use Training is performed, where the sample set ,in It is to input a sentence. It is a head entity. It is a tail entity. This represents the correspondence, where N is the number of samples;

[0098] Training is performed using the designed model M: ;

[0099] in, The model representing the design, Represents the first visible dataset One test sample, This represents the model's predicted output for the sample. Indicates the visible relationships predicted by the model;

[0100] S1-3: Test model M using an invisible dataset;

[0101] During the testing phase, given an input sentence x and entity pairs... , Model M is used to predict unseen relations expressed by entities in the input sentence. ,Right now ;

[0102] in, The first unseen dataset represents the... One test sample, This represents the model's predicted output for the sample. Indicates the visible relationships predicted by the model;

[0103] S2: Generate adaptive prompt words through a dynamic prompt generator to build dynamic prompt templates;

[0104] S2-1: Input a sentence and mark it;

[0105] Enter sentence ,in, This represents the input sentence. Sentence The first token in the middle, Sentence The nth token;

[0106] Through four special tokens , , and To mark sentences The positions of the head and tail entities are obtained. as follows:

[0107] ;

[0108] in, This indicates the processed sentence. , , These represent the 1st, 2nd, and nth tokens in the original sentence, respectively. The starting token representing the header entity. This represents the original word content corresponding to the head entity. This indicates the end token of the header entity. The starting token representing the tail entity. The original word content representing the tail entity. The end token representing the tail entity;

[0109] S2-2: A prompt template for building a T5 model;

[0110] The dynamic prompt generator generates verb phrases based on entity tags, automatically constructing prompts with [MASK] tags;

[0111] S2-3: Input sentence Convert to input for the T5 model;

[0112] The converted format is as follows:

[0113]

[0114] in, , Each represents a sentence The head and tail entities in the middle, Indicates splicing, These are placeholder markers used in sequence-to-sequence models such as T5 to indicate the positions of the output to be generated; the meaning of the sentence is: Generate only one that best describes and A verb phrase relating the two. This relationship is... ;

[0115] S2-4: Predict relational phrases based on the generated prompts and the input sentence using the T5 model;

[0116] The T5 model generates words based on pre-training capabilities for word completion tasks. Corresponding candidate phrases;

[0117] S2-5: Optimize the relational phrases predicted by the T5 model using the Beam Search algorithm;

[0118] The formula is as follows:

[0119] ;

[0120] in, This indicates the decoding status at the current time step. Indicates based on the current decoding state Perform a beam search operation. Indicates the candidate sequence { Take the sequence that makes the following expression the largest. The beam search width is 3 in this embodiment to balance generation quality and computational efficiency. It is the first in the candidate sequence Candidate phrases for each position, Indicates the first Word in front of arrive Probability product under given conditions Given all the preceding words, the candidate phrase is... The probability of;

[0121] S2-6: After obtaining the optimal pre-selected words, construct a personalized prompt template for each input sentence;

[0122] Verb phrases generated by the T5 model are used as context-sensitive semantic cues and filled into a predefined template framework to construct dynamic prompts. In this embodiment, the template format is as follows:

[0123] ;

[0124] The template construction process is fully automated. After obtaining the final sentence prompt template, we concatenate the sentence with the prompt template to obtain the final sentence representation, which is then fed into the encoder to obtain the final sentence embedding. We use BERT-base (Devlin et al., 2019) as a pre-trained encoder to generate the sentence representation and prototype representation.

[0125] S2-7: The hidden state, calculated by averaging the three masks in the prompt template, is used as the final sentence representation. The formula is as follows:

[0126] ;

[0127] in, This represents the features obtained from the final aggregation. This indicates a pooling operation. This indicates the feature marked with [MASK] corresponding to the head entity. This indicates the feature marked with [MASK] corresponding to the tail entity. The features marked with [MASK] represent the relationships;

[0128] S3: Dynamically generate and aggregate context-dependent prototype representations through a dynamic prototype aggregator;

[0129] S3-1: Relational prototype processing;

[0130] S3-1-1: Input Relationship tags ,describe and three randomly selected tag names aliases , , The five types of prototype information are encoded using the same BERT encoder. Sentences and prototypes share the same encoder, thus achieving alignment in the representation space and avoiding the distribution inconsistency caused by different encoders.

[0131] S3-1-2: Stack the embedding vectors corresponding to the above 5 types of prototype information to form the prototype feature matrix as follows:

[0132] ;

[0133] in, It is a relationship The corresponding prototype feature matrix, For the hidden layer dimension, For encoder, This is a max pooling operation;

[0134] S3-1-2: Capturing key information within a message through a multi-head self-attention mechanism;

[0135] To further explore the semantic relationships between various edge information in the prototype and achieve more effective representation fusion, a multi-head self-attention mechanism is used to model the interaction relationships between different edge information.

[0136] The prototype feature matrix The input unit of the self-attention mechanism is fed into a linear projection to obtain the Query, Key, and Value as follows:

[0137] ;

[0138] in, All are learnable parameter matrices. For querying the matrix, The key matrix, It is a value matrix;

[0139] Then, the scaled dot product attention is computed in parallel within the multi-head space to dynamically capture the importance of different edge information.

[0140] S3-1-3: Normalization is performed using the LayerNorm layer;

[0141] After attention-weighted aggregation and residual connections, the context-adaptive relation prototype representation is obtained as follows: ;

[0142] in, For context-adaptive prototype representation, This indicates the number of heads in the multi-head attention mechanism, where each head independently calculates attention to capture semantic relationships in different subspaces. This is a normalized exponential function used to calculate the attention weights between nodes. The dimension of each head is used to scale the dot product to stabilize the gradient; For relationship The query matrix, For relationship The key matrix, Key matrix transpose, For relationship The value matrix, It is a relationship The corresponding prototype feature matrix;

[0143] S3-2: Text Feature Extraction;

[0144] S3-2-1: Input a sentence containing the target entity;

[0145] S3-2-2: Encode the input sentence using the BERT encoder to extract a context-aware semantic vector representation;

[0146] S3-2-3: The information encoded in step S3-2 is processed through the Pooling layer to extract global text features;

[0147] S3-3: Aggregate the features processed in steps S3-1 and S3-2 to obtain the final prototype embedding;

[0148] S4: Training and Testing;

[0149] S4-1: Obtain sentence representation and prototype embedding;

[0150] S4-2: Determine the prototype of the positive sample relation corresponding to the current sample. ;

[0151] S4-3: Select a set of prototype relationships between several negative samples that are unrelated to positive samples. ;

[0152] S4-4: Setting Hyperparameters ;

[0153] S4-5: Calculate similarity;

[0154] S4-5-1: Calculate sentences Compared with positive sample prototype similarity ;

[0155] ;

[0156] in, Sentences indicating calculation Compared with positive sample prototype similarity

[0157] S4-5-2: Calculate sentences With each negative sample prototype The value with the highest similarity ;

[0158] ;

[0159] in, To obtain the maximum value, To ask for a sentence With negative sample prototype Similarity;

[0160] S4-6: Calculate marginal loss;

[0161] The calculation is performed using the following function:

[0162] ;

[0163] in, The loss value. This represents the currently input sentence. Embed the positive sample prototype corresponding to the current input sentence. For negative sample embedding, The negative samples representing random sampling are: In this embodiment, 7 negative samples were used. The hyperparameter controls the minimum distance between positive and negative samples. When L=0, the discrimination between positive and negative samples is sufficient, and no model update is needed. When L>0, the discrimination between positive and negative samples is insufficient, and the model parameters need to be updated through backpropagation to improve the similarity of positive samples and reduce the similarity of negative samples.

[0164] To better illustrate our technical solution, we use two commonly used datasets in this field, FewRel and Wiki-ZSL, to evaluate the performance of our model on the zero-shot relation extraction task. The FewRel dataset (Gao et al., 2019) contains a total of 56,000 instances, with the training set covering 64 relations and the validation set including 16 relations. Wiki-ZSL (Chen and Li, 2021) was generated from Wiki-KB (Sorokin and Gurevych, 2017) using a remote supervision method and contains rich textual information and category labels. This dataset contains a total of 93,383 sentences, covering 113 relation types. Compared to the FewRel dataset, Wiki-ZSL provides richer relation information, but due to the remote supervision method, the original data inevitably contains more noise. Relation descriptions can be obtained from Chen and Li (2021) or directly accessed through Wikidata. Additionally, other supplementary information, such as relation label names and aliases, can also be obtained directly from Wikidata.

[0165] Training and Evaluation: Following the experimental setup of (Chen and Li, 2021) et al., we randomly selected a seed relation as the invisible label and split the dataset into training and validation sets. We ensured no overlap between visible and invisible relations. To ensure the reliability of the experiment, we chose five random seeds consistent with those of Zhao et al. (2023) to split the dataset and reported the average results for different selections.

[0166] Experimental Setup: We used the same Bert-base-uncased encoder as the pre-trained encoder for both the input sentence and the relation description. We employed the AdamW optimizer with a learning rate of 2e-5 and 5 epochs. To achieve optimal performance on both datasets, we optimized other parameters: the number of heads in the multi-head attention mechanism during aggregation was set to 8, and dropout was set to 0.2. All experiments were conducted on an NVIDIA GeForce RTX 3090 system.

[0167] Evaluation Metrics: In our experiments, the F1 score was used as the primary evaluation metric. The F1 score is the harmonic mean of precision and recall, balancing the model's performance in both areas and effectively reflecting its performance on imbalanced tasks. Furthermore, to provide a more comprehensive evaluation of the model's performance, we also report precision and recall related to the F1 score. These metrics offer more detail and help us better understand the model's predictive quality and recognition capabilities.

[0168] For the baseline models, we compared three types of baseline models: prototype-based, classification-based, and generative-based.

[0169] For prototype-based methods:

[0170] RE-Matching transforms sentence-level semantic matching into fine-grained semantic matching and filters out irrelevant information in sentences.

[0171] AlignRE effectively bridges the gap between sentences and their corresponding prototypes using a method based on encoding alignment and semantic alignment.

[0172] SDZRE employs bidirectional semantic distillation, which not only filters out irrelevant information in sentences but also enhances relevant information.

[0173] For classification-based methods, GLiREL proposes an efficient zero-shot relation classification architecture and training paradigm that can simultaneously predict relation labels between multiple entities in a single forward propagation and demonstrates good performance on the FewRel and WikiZSL benchmarks.

[0174] For generative methods, SUMASK proposed a summarize-and-ask hint mechanism, which recursively transforms the relation extraction task into a question-and-answer format, thereby significantly improving the performance of large language models in zero-shot relation extraction.

[0175] RelationPrompt synthesizes relation examples using structured prompt templates and combines them with TripletSearch decoding to achieve multi-relation extraction, demonstrating significant performance in zero-sample relation triple extraction tasks.

[0176] The experimental results are shown in Table 1 below:

[0177] Table 1: Results (%) of the proposed DPPS solution and baseline method on the FewRel and Wiki-ZSL datasets, where m represents the number of unknown relations, and bold and underline represent the best and second-best results, respectively.

[0178]

[0179] Table 1 shows the results of the proposed DPPS solution on the FewRel and WikiZSL datasets. Experiments demonstrate that DPPS significantly outperforms previous state-of-the-art (SOTA) methods in F1 score on the WikiZSL dataset, and achieves SOTA results even in the complex case of the FewRel dataset (m=15). Particularly noteworthy is its excellent performance in the complex case of m=15, with F1 scores increasing by 4.47% and 5.85% on the FewRel and WikiZSL datasets, respectively. While previous methods showed a significant performance decline as the number of unknown relations increased (m=15), DPPS maintained strong stability. Figure 2 , Figure 4 As shown.

[0180] Furthermore, our method demonstrates a more significant improvement over the baseline model AlignRE. In predicting three different numbers of unknown relations, our F1 scores in FewRel increased by 2.03%, 4.77%, and 7.22%, respectively, while in Wiki-ZSL, they increased by 6.61%, 9.39%, and 7.54%, respectively. This indicates the effectiveness of DPPS in ZeroSRE tasks and its superior ability to generalize to predicting more unknown relations. In summary, DPPS not only performs well in the transfer from known relations to unknown relations but also maintains stable performance as the scale of unknown relations increases, demonstrating that our proposed method has a clear advantage in ZeroSRE tasks and is particularly suitable for complex and resource-constrained scenarios.

[0181] To further verify the effectiveness of our proposed method, we conducted ablation experiments on the FewRel dataset with an m=15 setting. The results are shown in Table 2. "w / o T5" indicates the removal of the T5-based auto-completion template building block and the use of a fixed prompt template. "w / o Mapa" means replacing dynamic prototype aggregation with a traditional static method. "w / o Pro" corresponds to removing the dynamic prototype representation and using a pre-coded static prototype.

[0182] Removing the auto-suggestion template building block and using a fixed suggestion template leads to a performance degradation in the model. This demonstrates that our proposed auto-suggestion generator provides effective external semantic cues for sentence representations, helping the model better align with relational prototypes. This result validates the importance of auto-suggestion templates in improving model performance.

[0183] When the dynamic prototype aggregation model is removed and the traditional static method is used to aggregate prototype edge information, the model performance decreases by 0.86%. We believe this is because traditional static aggregation cannot adequately model the semantic differences and importance between different edge information, thus limiting the expressive power of the prototype representation. In contrast, our proposed dynamic prototype aggregation method adaptively assigns weights to information from different sources, thereby obtaining a more accurate and robust prototype representation.

[0184] When we remove the dynamic prototype representation and use a pre-encoded static prototype representation, we observe a significant performance drop in the experimental results, confirming the crucial role of dynamic prototypes in achieving accurate semantic matching. This indicates that static prototypes struggle to flexibly adjust according to context, leading to insufficient semantic alignment between sentences and prototypes. In contrast, the dynamic prototype representation proposed in this paper adaptively integrates side information, resulting in a more discriminative prototype representation, thereby effectively improving model performance.

[0185] Table 2: Ablation experiments (%) of this technical solution, with experimental results uniformly selected when m=15.

[0186]

[0187] Table 3: Comparison of different prototype coding methods on the FewRel dataset (m=15)

[0188]

[0189] To further explore the impact of different encoding methods on prototype representation, we conducted experiments on the FewRel dataset, testing four different encoding strategies. The results are shown in Table 3. AlingRE represents the traditional static prototype encoding method, BERT_SEN represents dynamic encoding of the prototype using a sentence-transformer, BERT_BERT represents encoding the prototype and sentence separately using two BERT encoders, and Ours represents our proposed DPPS method, which uses the same encoder for dynamic encoding of both the prototype and sentence.

[0190] Experiments show that the unified encoder method proposed in this paper significantly outperforms other methods in F1 score, achieving 84.53%, which is 0.55 percentage points higher than the second-best method, BERT-BERT. It is worth noting that although BERT-BERT uses two independent BERT encoders, its performance is still lower than the encoding strategy proposed in this paper. We believe that independent encoders may introduce unnecessary representation space inconsistencies, leading to difficulties in aligning the prototype with the sentence representation.

[0191] The BERT-SEN method performed the worst, with an F1 score of only 76.50%, indicating that using Sentence-Transformer for prototype encoding cannot effectively capture the fine-grained semantic relationships between sentences and prototypes. In contrast, the unified encoder design of DPPS, through a shared representation space, allows prototypes to dynamically adjust according to sentence context, significantly improving the model's ability to understand relational semantics.

[0192] Furthermore, considering the balance between precision and recall, the DPPS method maintains high precision while achieving a recall of 83.93%, indicating that it effectively captures positive examples while reducing false positives. This is closely related to our proposed dynamic prototype representation mechanism. Dynamic prototype representation, through context-aware adaptive adjustment, effectively alleviates the problem of semantic ambiguity in traditional static encoding methods, enabling the model to more accurately distinguish semantically similar relation types.

[0193] Table 4: Comparison of the effects of different prompt templates

[0194]

[0195] To systematically evaluate the impact of prompt templates on model performance, we designed a comparative experiment. We used four representative prompt formats to guide the T5 model to generate verb phrase prompts and evaluated the relation extraction effect under the FewRel (m=15) setting. The results are shown in Table 4.

[0196] Experiments show that, under the premise of semantic coherence and clear task orientation, the impact of different prompt formats on model performance is relatively limited—the F1 score difference among the variants does not exceed 1.38%. This indicates that as long as the prompt template can clearly convey the core instruction of "generating verb phrases describing the relationship between two entities," the T5 model can stably output discriminative prompt words, thereby ensuring the basic performance of downstream relation extraction tasks. This result verifies that our T5-based automatic prompt generation framework has good robustness.

[0197] However, it is worth noting that our complete method (Ours) still significantly outperforms all contrastive prompt templates (improving F1 by 1.03–1.38 percentage points). This is primarily due to our additional introduction of relational semantic descriptions as conditional constraints into the prompts, enabling T5 to not only rely on sentence context during generation but also incorporate prior relational knowledge, thus producing more accurate and discriminative verb phrases. For example, for the "place of birth" relation, the standard prompt might generate the generalized "located in," while our method tends to generate the more specific "was born in." This semantic refinement capability is one of the key factors contributing to DPPS's leading performance in complex zero-shot scenarios.

[0198] In summary, while changes to the basic suggestion format have a limited impact on overall performance, carefully designed suggestions that incorporate external semantic information can still significantly improve model performance. This further validates the value of automated suggestion generation mechanisms: they not only avoid the subjectivity and limitations of manual templates but also provide a flexible interface for dynamically injecting structured prior knowledge.

[0199] Table 5: Comparison of different methods for abstract relations

[0200]

[0201] To further verify the model's generalization ability when dealing with complex relationships that are highly semantically abstract and structurally implicit, we selected five representative abstract relationships from the FewRel dataset for testing: P31 (instanceof), P361 (part of), P527 (has part), P460 (said to be the same as), and P921 (mainsubject). These types of relationships often have significant semantic abstraction and context dependence, and traditional methods are prone to semantic confusion and boundary ambiguity in the absence of clear semantic cues. The experimental results are shown in Table 5.

[0202] As can be seen, DPPS significantly outperforms the baseline method AlignRE in the abstract relation extraction task, with an F1 score improvement of up to 18.73%. This result demonstrates that DPPS's dynamic prototype encoding and self-constructed prompting mechanism exhibit stronger robustness and generalization ability in the face of abstract and complex semantic scenarios, maintaining high relation recognition accuracy under conditions of low explicit features. The main reason for this is that dynamic prototype encoding allows the model to adaptively adjust the relation prototype representation according to the contextual semantics, thereby enhancing the model's sensitivity to semantic ambiguity and implicit relations. Simultaneously, the self-constructed prompting mechanism guides the model to explicitly focus on semantic dependencies between entities by automatically generating context-related verb phrase prompts, effectively compensating for the lack of explicit semantic cues in abstract relations. Therefore, DPPS, based on this technical solution, can form more discriminative and consistent relation representations in the semantic space.

[0203] To further demonstrate DPPS's ability to learn better relation representations, we use the t-SNE (Van der Maaten and Hinton, 2008) method to reduce the high-dimensional representation of relation instances to a two-dimensional plane for visualization. We randomly selected five categories from the FewRel dataset as unseen relations and extracted their relation-level semantic representations from the test set instances of these relations. The results are as follows: Figure 3 As shown in the diagram. Each data point represents a relation instance, and the color corresponds to its actual label category. From... Figure 3 (b) As can be seen, in the baseline model AlignRE, the distribution of data points from different categories is quite mixed, the boundaries are blurred, and some relationships overlap significantly, indicating that the model has difficulty effectively distinguishing semantically similar relationships. In contrast, the relationship representations generated by DPPS exhibit stronger intra-class clustering and inter-class separability, indicating that DPPS can learn more discriminative relationship feature representations, thereby effectively alleviating the problem of confusion between similar relationships and improving the generalization ability in zero-shot scenarios.

[0204] Therefore, this technical solution proposes a zero-shot relation extraction model—DPPS—which improves the model's generalization and discriminative capabilities through dynamic prototype encoding and a prompt self-construction mechanism. The dynamic prototype mechanism enables the prototype representation to adaptively adjust with the context, achieving unified semantic alignment between sentences and prototypes; automatic prompt generation utilizes the T5 model to generate verb phrase-type prompts, strengthening semantic differentiation. Experimental results show that DPPS outperforms existing methods in complex scenarios using FewRel and Wiki-ZSL, especially demonstrating significant advantages in abstract relations and complex semantic scenarios, validating the effectiveness of this method in improving the robustness and generalization ability of zero-shot relation extraction.

Claims

1. A dynamic prototype encoding and prompt self-construction method for zero-sample relation extraction, characterized in that: Includes the following steps: S1: Define the task; S2: Generate adaptive prompt words through a dynamic prompt generator to build dynamic prompt templates; S3: Dynamically generate and aggregate context-dependent prototype representations through a dynamic prototype aggregator; S4: Training and Testing.

2. The dynamic prototype encoding and prompt self-construction method for zero-sample relation extraction according to claim 1, characterized in that: Step S1 includes: S1-1: Define visible relationships, invisible relationships, and visible datasets; set up Let be a visible relation, and represent the set of trained relations, where , Indicates the first One visible relationship; set up Let be an invisible relation, representing the set of untrained relations, where , Indicates the first An invisible relationship; set up For the visible dataset, This is an invisible dataset; S1-2: Train model M using the visible dataset; S1-3: Test model M using an invisible dataset.

3. The dynamic prototype encoding and prompting self-construction method for zero-sample relation extraction according to claim 1, characterized in that: Step S2 includes: S2-1: Input a sentence and mark it; S2-2: A prompt template for building a T5 model; The dynamic prompt generator generates verb phrases based on entity tags, automatically constructing prompts with [MASK] tags; S2-3: Input sentence Convert to input for the T5 model; S2-4: Predict relational phrases based on the generated prompts and the input sentence using the T5 model; S2-5: Optimize the relational phrases predicted by the T5 model using the Beam Search algorithm; The formula is as follows: ； in, This indicates the decoding status at the current time step. Indicates based on the current decoding state Perform a beam search operation. Indicates the candidate sequence { Take the sequence that makes the following expression the largest. For the beam search width, It is the first in the candidate sequence Candidate phrases for each position, Indicates the first Word in front of arrive Probability product under given conditions Given all the preceding words, the candidate phrase is... The probability of; S2-6: After obtaining the optimal pre-selected words, construct a personalized prompt template for each input sentence; S2-7: The hidden state, calculated by averaging the three masks of the cue template, is used as the final sentence representation, with the following formula: ； in, This represents the features obtained from the final aggregation. This indicates a pooling operation. This indicates the feature marked with [MASK] corresponding to the head entity. This indicates the feature marked with [MASK] corresponding to the tail entity. The feature marked with [MASK] represents the relation.

4. The dynamic prototype encoding and prompt self-construction method for zero-sample relation extraction according to claim 1, characterized in that: The converted format in step S2-3 is as follows: ;in, , Each represents a sentence The head and tail entities in the middle, Indicates splicing.

5. The dynamic prototype encoding and prompt self-construction method for zero-sample relation extraction according to claim 1, characterized in that: Step S2-1 includes the following: Enter sentence ,in, This represents the input sentence. Sentence The first token in the middle, Sentence The nth token; Through four special tokens , , and To mark sentences The positions of the head and tail entities are obtained. as follows: ； in, This indicates the processed sentence. , , These represent the 1st, 2nd, and nth tokens in the original sentence, respectively. The starting token representing the header entity. This represents the original word content corresponding to the head entity. This indicates the end token of the header entity. The starting token representing the tail entity. The original word content representing the tail entity. This represents the end token of the tail entity.

6. The dynamic prototype encoding and prompt self-construction method for zero-sample relation extraction according to claim 1, characterized in that: Step S3 includes the following steps: S3-1: Relational prototype processing; S3-2: Text Feature Extraction; S3-3: Aggregate the features processed in steps S3-1 and S3-2 to obtain the final prototype embedding.

7. The dynamic prototype encoding and prompt self-construction method for zero-sample relation extraction according to claim 6, characterized in that: Step S3-1 includes the following steps: S3-1-1: Input Relationship tags ,describe and three randomly selected tag names aliases , , The information will be encoded using the same BERT encoder for all five types of prototype information mentioned above. S3-1-2: Stack the embedding vectors corresponding to the above 5 types of prototype information to form the prototype feature matrix as follows: ； in, It is a relationship The corresponding prototype feature matrix, For the hidden layer dimension, For encoder, This is a max pooling operation; S3-1-2: Capturing key information within a message through a multi-head self-attention mechanism; The prototype feature matrix The input unit of the self-attention mechanism is fed into a linear projection to obtain the Query, Key, and Value as follows: ； in, All are learnable parameter matrices. For querying the matrix, The key matrix, It is a value matrix; Then, the scaled dot product attention is computed in parallel within the multi-head space to dynamically capture the importance of different edge information; S3-1-3: Normalization is performed using the LayerNorm layer; After attention-weighted aggregation and residual connections, the context-adaptive relation prototype representation is obtained as follows: ； in, For context-adaptive prototype representation, This indicates the number of heads in the multi-head attention mechanism, where each head independently calculates attention to capture semantic relationships in different subspaces. This is a normalized exponential function used to calculate the attention weights between nodes. The dimension of each head is used to scale the dot product to stabilize the gradient; For relationship The query matrix, For relationship The key matrix, Key matrix transpose, For relationship The value matrix, It is a relationship The corresponding prototype feature matrix.

8. The dynamic prototype encoding and prompt self-construction method for zero-sample relation extraction according to claim 6, characterized in that: Step S3-2 includes the following steps: S3-2-1: Input a sentence containing the target entity; S3-2-2: Encode the input sentence using the BERT encoder to extract a context-aware semantic vector representation; S3-2-3: The information encoded in step S3-2 is processed through the Pooling layer to extract global text features.

9. The dynamic prototype encoding and prompt self-construction method for zero-sample relation extraction according to claim 1, characterized in that: Step S4 includes the following steps: S4-1: Obtain sentence representation and prototype embedding; S4-2: Determine the prototype of the positive sample relation corresponding to the current sample. ; S4-3: Select a set of prototype relationships between several negative samples that are unrelated to positive samples. ; S4-4: Setting Hyperparameters ; S4-5: Calculate similarity; S4-5-1: Calculate sentences Compared with positive sample prototype similarity ; ； in, Sentences indicating calculation Compared with positive sample prototype Similarity; S4-5-2: Calculate sentences With each negative sample prototype The value with the highest similarity ; ； in, To obtain the maximum value, To ask for a sentence With negative sample prototype Similarity; S4-6: Calculate marginal loss; The calculation is performed using the following function: ； in, The loss value. This represents the currently input sentence. Embed the positive sample prototype corresponding to the current input sentence. For negative sample embedding, The negative samples representing random sampling are: indivual, The hyperparameter used to control the minimum distance between positive and negative samples; Indicates the number of samples; S4-7: When L=0, the positive and negative sample discrimination is sufficient and there is no need to update the model; when L>0, the positive and negative sample discrimination is insufficient and the model parameters need to be updated through backpropagation.

Images