A lightweight zero-shot relation extraction method based on multi-path semantic alignment strategy

Abstract

The application discloses a lightweight zero-shot relation extraction method based on a multi-path semantic alignment strategy, and comprises the following steps: S1, data and model preparation; S2, pre-training of the model; S3, training of the pre-trained model; and S4, result output by the trained model. The original representation directly output by the encoder can be used for training, and through the pre-training and training strategies based on contrast learning, fine-grained multi-path semantic matching between context representation and entity representation is realized.

Description

Technical Field

[0001] This invention relates to the field of zero-sample relation extraction technology, specifically to a lightweight zero-sample relation extraction method based on a multi-path semantic alignment strategy. Background Technology

[0002] Relation extraction (RE) is a branch of information extraction (IE) in Natural Language Processing (NLP), aiming to identify semantic relationships between entity pairs in unstructured text. However, traditional supervised RE models heavily rely on large-scale labeled data and often struggle when faced with newly emerging or rare relationships. To alleviate this limitation, zero-shot relation extraction (ZSRE) has attracted increasing attention. It requires models to generalize to never-before-seen relation types based solely on natural language descriptions or prior semantics, thus examining the model's ability to transfer relational knowledge beyond labeled data.

[0003] However, existing semantic matching methods, especially those based on Siamese dual-tower encoders, often lack fine-grained semantic interactions between the two sides. This results in the sentence and relation representation learning processes being independent of each other. In this model, entity information and contextual information are often fused into a single representation, lacking explicit interaction modeling, which limits the model's performance in fine-grained semantic alignment and generalization. To improve the quality of semantic modeling, existing fine-grained semantic matching methods often rely on additional neural network layers as refining modules to further map and compress the encoder's output representation. While this method can compensate for the semantic gap between the sentence and relation sides to some extent, it also introduces additional computational overhead and inference latency, making it unsuitable for deployment in real-world scenarios. Summary of the Invention

[0004] This invention aims to solve the technical problems existing in the prior art. In particular, it proposes a lightweight zero-sample relation extraction method based on a multi-path semantic alignment strategy. It can be trained using only the original representation directly output by the encoder. Through a pre-training and training strategy based on contrastive learning, it can achieve fine-grained multi-path semantic matching between context representation and entity representation.

[0005] To achieve the above objectives, this invention provides a lightweight zero-sample relation extraction method based on a multi-path semantic alignment strategy, comprising the following steps:

[0006] S1: Data and Model Preparation;

[0007] S2: Pre-train the model;

[0008] S2-1: Encode the input sentence;

[0009] The labeled input sentence is fed into BERT-base to extract three types of feature vectors: global context, head entity, and tail entity. The hidden state sequence of all tokens is obtained and the three types of representations are extracted as follows:

[0010] ;

[0011] ;

[0012] in, The vector representation of the location of the special marker [CLS]. Indicates the input sentence The context vector representation of the first token Indicates the input sentence No. The context vector representation of each token; For input sentence The special marker [CLS] is appended to the beginning of the sentence before inputting BERT-base. Indicates the input sentence The first token, Indicates the input sentence The second token, Indicates the input sentence The One token; A representation of contextual information; and These represent the information of the head entity and the tail entity, respectively; where, This is a vector representation of the location of the head entity's starting marker [Eh]. The vector representation of the location of the tail entity's starting marker [Et];

[0013] S2-2: Refined pre-training;

[0014] S2-2-1: Prepare the positive and negative sample pairs needed for contrastive learning;

[0015] Given a training batch, the batch contains a total of Each input sentence constitutes a batch sample set. Using the BERT-base model as the encoder and in the encoder Enable random Dropout masking in the Transformer network layer;

[0016] Each input sentence within the batch is input into the encoder. The data is encoded twice to generate positive and negative sample pairs.

[0017] Sentence Repeat the input to the encoder twice. In the middle, relying on different random Dropout masks in the two forward propagations, each type of semantic representation is... Two homologous enhancement vectors are generated respectively. , ,in, This indicates the first input to the encoder. The generated augmentation vector, This indicates the second input to the encoder. The generated augmented vector;

[0018] Two augmentation vectors with the same input sentence and the same representation , As positive sample pairs;

[0019] Use all the augmented vectors of all other sentences in the current mini-batch as negative sample pairs for the current sample;

[0020] S2-2-2: Based on bidirectional InfoNCE, calculate the bidirectional InfoNCE loss for the global context, head entity, and tail entity representations respectively;

[0021] The bidirectional InfoNCE loss for the global context, head entity, and tail entity representations is calculated using the following formula:

[0022] ;

[0023] in, Indicates the first The sentence, the first The corresponding individual loss value for the semantic representation of the class, Represents the cosine similarity between vectors. This indicates the first input to the encoder. The generated augmentation vector, This indicates the first input to the encoder. The generated augmented vector; Indicates temperature hyperparameter;

[0024] S2-2-3: Calculate the total global loss;

[0025] ;

[0026] in, The bidirectional InfoNCE loss represents the global context representation. The bidirectional InfoNCE loss represents the head entity representation. The bidirectional InfoNCE loss represents the tail entity representation; Indicates the semantic weight of the head entity. Indicates the semantic weight of the tail entity;

[0027] S2-2-4: Total Loss Minimize as the optimization objective, perform backpropagation and gradient descent, and update the encoder. All network parameters;

[0028] S2-2-5: Repeat steps S2-2-1 to S2-2-4 to iterate through all training data, completing the entire pre-training process, and finally outputting the encoder optimized through contrastive learning. ;

[0029] S3: Train the pre-trained model;

[0030] S3-1: Employs intermediate layer semantic representation distillation to directly align the vector feature spaces of the teacher model and the student model, while preserving underlying semantic knowledge;

[0031] S3-2: Training using a multi-directional semantic alignment strategy;

[0032] S3-3: Calculate the global final total loss;

[0033] S4: Output the results using the trained model.

[0034] In the above scheme, step S1 further includes:

[0035] S1-1: Data preparation;

[0036] S1-1-1: Process the input sentences in the training dataset;

[0037] training dataset Each input sentence is marked with special tags [CLS], [Eh], [\Eh], [Et], and [\Et] to indicate the positions of the first and last entities in the input sentence;

[0038] in, Indicates the first One input sentence, Indicates the first The header entity of an input sentence. Indicates the first The tail entity of the input sentence, Indicates the first Labels for each input sentence. Indicates the first A relational description, Indicates the size of the training dataset;

[0039] S1-1-2: Processing the relational descriptions of the training dataset;

[0040] S1-2: Initialize the model;

[0041] BERT-base is used as the encoder.

[0042] In the above scheme, step S3-1 further includes:

[0043] S3-1-1: Initialize the teacher and student models;

[0044] set up and These represent the student encoder and the teacher encoder, respectively, both of which are pre-trained encoders. Freeze the teacher encoder All network parameters are fixed and do not undergo gradient updates throughout the training process, making them a fixed and unchanging original pre-trained knowledge base.

[0045] S3-1-2: Enter a sentence By simultaneously processing sentences using both teacher encoders and student encoders... Encode the data and extract three types of representations;

[0046] Enter sentence Simultaneously input the teacher encoder With student encoder In each case, three complete sets of semantic vectors are extracted.

[0047] ;

[0048] ;

[0049] ;

[0050] ;

[0051] in, The contextual representation semantic vector extracted for the student encoder. The semantic vector of the head entity extracted for the student encoder. The semantic vector of the tail entity extracted for the student encoder. The contextual representation semantic vector extracted for the teacher encoder. The semantic vector of the head entity extracted by the teacher encoder. The semantic vector of the tail entity extracted by the teacher encoder.

[0052] S3-1-3: Calculate the cosine distance between the teacher encoder and the student encoder;

[0053] The cosine distance of a single set of representations is calculated using the following formula:

[0054] ;

[0055] in, Represents the cosine distance. Cosine similarity between vectors Semantic representation index ; This represents the student representation vector corresponding to the semantic components; The teacher representation vector representing the corresponding semantic component;

[0056] S3-1-4: Calculate the total characterization distillation loss;

[0057] ;

[0058] in, For the first in the current batch The total representation distillation loss for each input sentence Indicates the first The cosine distance between the semantic vectors of the teacher encoder and the student encoder of each input sentence's context representation. Indicates the first The cosine distance between the semantic vectors of the head entities of the teacher encoder and the student encoder for each input sentence. Indicates the first The cosine distance between the tail entity semantic vectors of the teacher encoder and student encoder for each input sentence. , This is the current batch size. It represents the set of positive integers.

[0059] In the above scheme, step S3-2 further includes:

[0060] S3-2-1: The pre-trained encoder As an encoder for relation description, the structured relation description text is input into the encoder for encoding;

[0061] Given relation description ,in and They are Hypersemantic description of a middle entity, wherein, For the first The original text sequence describing the relationship; For the first The first of the original text sequences describing the relationship One word;

[0062] The complete description of the original relation is encoded as follows:

[0063] ;

[0064] The text encoding of the head entity hypersemantic description is as follows:

[0065] ;

[0066] The text encoding of the tail entity hypersemantic description is as follows:

[0067] ;

[0068] in, In the relation description, the first The representation vector of each context part; In the relation description, the first The representation vector of the head entity part; In the relation description, the first The representation vector of each tail entity part.

[0069] S3-2-2: Extract the beginning of the sentence as follows:

[0070] ;

[0071] in, This represents the overall representation vector of the contextual semantics in the relation description. This represents the representation vector of the first word in the context portion of the relation description. This represents the overall semantic representation vector of the head entity in the relation description. Let represent the representation vector of the first word in the head entity part of the relation description. This represents the overall semantic representation vector of the tail entity in the relation description. This represents the representation vector of the first word in the tail entity part of the relation description;

[0072] Then for the first batch The overall representation vectors of the contextual semantics, head entity semantics, and tail entity semantics of all relation descriptions are extracted as follows: , , ;

[0073] S3-2-3: Encoder after pre-training For the first batch The following representations of the input sentence are extracted at the context, head entity, and tail entity levels: , , ;

[0074] S3-2-4: Calculate the contrastive loss of each part of the context, head entity, and tail entity representation;

[0075] ;

[0076] in, It is the contrastive learning loss function corresponding to the semantic part; Represents the cosine similarity between vectors; Indicates the first in the batch The representation vector of the corresponding semantic part in each input sentence; Indicates the first in the batch The representation vector describing the positive sample relation of the corresponding semantic part in each input sentence; Indicates the first in the batch A semantic representation vector describing a relation; This represents temperature, which is an adjustable hyperparameter. Batch size;

[0077] Based on the above formula, the contrast loss of each part of the context, head entity, and tail entity representation is calculated respectively. , , ;

[0078] S3-2-5: Calculate the total loss of the semantic alignment main task;

[0079] ;

[0080] in, Indicates the semantic weight of the head entity. Indicates the semantic weight of the tail entity. Indicates the first The total loss of the semantic alignment main task for each input sentence. Indicates the first The contrastive loss is derived from the contextual representation of each input sentence. Indicates the first The contrastive loss is calculated based on the head entity representation of each input sentence. Indicates the first Contrast loss for tail entity representations of each input sentence.

[0081] In the above scheme, step S3-3 also includes:

[0082] Introducing equilibrium hyperparameters We then weight and fuse the main task comparison loss and the distillation regularization loss to obtain the final optimization objective throughout the training process, as shown in the following formula:

[0083] ;

[0084] in, For the first The global final total loss for each input sentence; As a weighting factor, For the first The total representation distillation loss for each input sentence Indicates the first The total loss of the semantic alignment main task for each input sentence;

[0085] In the above scheme, step S4 also includes:

[0086] S4-1: Input the sentence to be predicted and all its candidate no-relationship tags gather ;

[0087] S4-2: The encoder, after training, processes the input sentences separately. Each candidate relation description is encoded, and its three semantic representations—context, head, and tail entities—are extracted.

[0088] S4-3: Calculate the matching score between the sentence input and each candidate relation using cosine similarity;

[0089] Use weights during the training phase , Calculate the current sentence With the A comprehensive matching score for each candidate relation description;

[0090] ;

[0091] in, Represents the cosine similarity between vectors. Indicates the current sentence With the A comprehensive matching score for each candidate relation description; and These are the semantic weights of the head entity and the semantic weights of the tail entity, respectively; where... express a sentence Contextual semantic feature vectors express a sentence The feature vector of the head entity. express a sentence The tail entity feature vector; where, Indicates the first The contextual semantic feature vectors describing the candidate relations Indicates the first The head entity feature vectors described by the candidate relations Indicates the first Tail entity feature vectors described by candidate relations;

[0092] S4-4: Traverse all candidate relations and select the category with the highest matching score as the final predicted relation for the sentence;

[0093] The formula is as follows:

[0094] ;

[0095] in, For the final predicted relation of the sentence, To find the index corresponding to the maximum value of the independent variable, Indicates the current sentence With the The overall matching score for each candidate relation description.

[0096] In summary, the beneficial effects of this invention are as follows: This technical solution constructs a fine-grained semantic matching framework through a two-stage contrastive learning strategy of pre-training and training, and a multi-path semantic alignment mechanism. It also explicitly models the multi-directional interactions between context, head entity, and tail entity, thereby effectively enhancing the feature alignment capability between the sentence end and the relation end. This not only alleviates the representation collapse problem that pre-trained language models are prone to in zero-shot scenarios, but also amplifies the subtle semantic differences between similar relations, enabling the model to cope more robustly with complex contextual interference.

[0097] Furthermore, inspired by existing work, a contrastive pre-training strategy based on dropout noise is introduced, combined with a simple yet effective feature distillation module. Without adding additional learnable parameters, this module improves the model's robustness to semantic perturbations by constraining intermediate representations and effectively mitigates the catastrophic forgetting problem during the fine-tuning stage, thereby further enhancing the model's generalization and discriminative ability on unseen relationships. Attached Figure Description

[0098] Figure 1 This is a structural comparison diagram of the present invention and the traditional relation alignment model, where (a) is the structural diagram of the traditional relation alignment model and (b) is the structural diagram of the present invention.

[0099] Figure 2 This is a structure diagram of the pre-training phase.

[0100] Figure 3 This is a structural diagram of the training and evaluation phases.

[0101] Figure 4(a) is a schematic diagram of the embedding vector projected from the FewRel dataset in the t-SNE space; (b) is a schematic diagram of the embedding vector projected from the FewRel dataset in the t-SNE space; (c) is a schematic diagram of the embedding vector projected from the Wiki-ZSL dataset in the t-SNE space; and (d) is a schematic diagram of the embedding vector projected from the Wiki-ZSL dataset in the t-SNE space.

[0102] Figure 5 This is a performance diagram of the present technical solution on the FewRel dataset with different dropout values.

[0103] Figure 6 This technical solution takes a different approach. A performance diagram of the values ​​on the FewRel dataset. Detailed Implementation

[0104] The present invention will be further described below with reference to the embodiments and accompanying drawings:

[0105] like Figures 1-3 As shown, a lightweight zero-shot relation extraction method based on a multi-path semantic alignment strategy includes the following steps:

[0106] S1: Data and Model Preparation;

[0107] S1-1: Data preparation;

[0108] The entire dataset is divided into training datasets. and test dataset These two datasets correspond to the seen relation sets, respectively. and Unseen Relationship Set , and These represent the number of seen relation labels and the number of unseen relation labels, respectively; specifically, the labels in the two relation sets do not overlap, and each relation label is essentially associated with a natural language relation description and hypernyms related to the head and tail entities of the relation. The goal is to... Learn on and The generalization ability of the test model is then tested.

[0109] Training dataset Contains n instances, where Indicates the first One input sentence, Indicates the first The header entity of each input sentence. Indicates the first The tail entity of the input sentence, Indicates the first The positive correlation labels of the input sentences express The relationship description and its superordinate terms.

[0110] In the first stage, sentences are used as input in the training dataset. The above is the initialization model Pre-training was performed, and then the pre-trained data was used in the second stage. Fine-grained semantic alignment is achieved through multi-directional contrastive learning, ultimately yielding the final model. .

[0111] During the testing phase, the test dataset is used. For the model An assessment was conducted, in which The cosine similarity between the sentence representation and each candidate relation description is calculated, and then the relation with the highest similarity score is selected as the final prediction.

[0112] S1-1-1: Process the input sentences in the training dataset;

[0113] training dataset Each input sentence is marked with special tags [CLS], [Eh], [\Eh], [Et], and [\Et] to indicate the positions of the first and last entities in the input sentence;

[0114] in, Indicates the first One input sentence, Indicates the first The header entity of an input sentence. Indicates the first The tail entity of the input sentence, Indicates the first Labels for each input sentence. Indicates the first A relational description, Indicates the size of the training dataset;

[0115] For example, the sentence entered in this embodiment is "Apple, situated at California", which means: Apple Inc. is located in California.

[0116] S1-1-2: Processing the relational descriptions of the training dataset;

[0117] Prepare relation description text and write a structured description for each zero-sample relation;

[0118] Given relation description ,in and They are Hypersemantic description of a middle entity, wherein, Represents the set of head entity supersemantics. Represents the original relational text. Represents the set of supersemantic words for tail entities. Indicates the first Size entity hypersemantic word, Indicates the first Individual suffix supersemantic words;

[0119] For example, in this embodiment, the given relation is described in natural language as "An organization is located in a city," which means that an organization is located in a city. The head entity is "item," and its hypersynonym is "item, object." The tail entity is "sovereign state," and its hypersynonym is "country, sovereign state."

[0120] The structured relation description text is constructed based on hypersemantics, and the static relation description template is as follows:

[0121] Static slot filling template: Fills the superclass of the entity into a predefined template, which is defined as: "The type of the head / tail entity, including supersynonym 1, supersynonym 2, ..., supersynonym n".

[0122] Keywords: For keyword templates, the superclasses of an entity are directly concatenated to form the relation description of that entity. For example, the relation description of an entity would become: "supersynonym 1, supersynonym 2".

[0123] In this embodiment, the first template is selected as the relation description, while the second type is used for comparative experiments.

[0124] S1-2: Initialize the model;

[0125] BERT-base is used as the encoder;

[0126] S2: Pre-train the model;

[0127] S2-1: Encode the input sentence;

[0128] The labeled input sentence is fed into BERT-base to extract three types of feature vectors: global context, head entity, and tail entity, to prepare for subsequent relation classification. The hidden state sequence of all tokens is obtained and the three types of representations are extracted as follows:

[0129] ;

[0130] ;

[0131] in, The vector representation of the location of the special marker [CLS]. Indicates the input sentence The context vector representation of the first token Indicates the input sentence No. The context vector representation of each token; For input sentence The special marker [CLS] is appended to the beginning of the sentence before inputting BERT-base. Indicates the input sentence The first token, Indicates the input sentence The second token, Indicates the input sentence The One token; A representation of contextual information; and These represent the information of the head entity and the tail entity, respectively; where, This is a vector representation of the location of the head entity's starting marker [Eh]. The vector representation of the location of the tail entity's starting marker [Et];

[0132] S2-2: Refined pre-training;

[0133] S2-2-1: Prepare the positive and negative sample pairs needed for contrastive learning;

[0134] Given a training batch, the batch contains a total of Each input sentence constitutes a batch sample set. Using the BERT-base model as the encoder and in the encoder Enable random Dropout masking in the Transformer network layer;

[0135] Each input sentence within the batch is input into the encoder. The data is encoded twice to generate positive and negative sample pairs.

[0136] Sentence Repeat the input to the encoder twice. In the middle, relying on different random Dropout masks in the two forward propagations, each type of semantic representation is... Two homologous enhancement vectors are generated respectively. , ,in, This indicates the first input to the encoder. The generated augmentation vector, This indicates the second input to the encoder. The generated augmented vector;

[0137] Two augmentation vectors with the same input sentence and the same representation , As positive sample pairs;

[0138] Use all the augmented vectors of all other sentences in the current mini-batch as negative sample pairs for the current sample;

[0139] S2-2-2: Based on bidirectional InfoNCE, calculate the bidirectional InfoNCE loss for the global context, head entity, and tail entity representations respectively;

[0140] The bidirectional InfoNCE loss for the global context, head entity, and tail entity representations is calculated using the following formula:

[0141] ;

[0142] in, Indicates the first The sentence, the first The corresponding individual loss value for the semantic representation of the class, Represents the cosine similarity between vectors. This indicates the first input to the encoder. The generated augmentation vector, This indicates the first input to the encoder. The generated augmented vector; Indicates hyperparameters;

[0143] S2-2-3: Calculate the total global loss;

[0144] ;

[0145] in, The bidirectional InfoNCE loss represents the global context representation. The bidirectional InfoNCE loss represents the head entity representation. The bidirectional InfoNCE loss represents the tail entity representation; Indicates the semantic weight of the head entity. Indicates the semantic weight of the tail entity;

[0146] S2-2-4: Total Loss Minimize as the optimization objective, perform backpropagation and gradient descent, and update the encoder. All network parameters;

[0147] S2-2-5: Repeat steps S2-2-1 to S2-2-4 to iterate through all training data, completing the entire pre-training process, and finally outputting the encoder optimized through contrastive learning. This is used for fine-tuning of downstream tasks during subsequent training phases;

[0148] S3: Train the pre-trained model;

[0149] S3-1: Employs intermediate layer semantic representation distillation to directly align the vector feature spaces of the teacher model and the student model, while preserving underlying semantic knowledge;

[0150] S3-1-1: Initialize the teacher and student models;

[0151] set up and These represent the student encoder and the teacher encoder, respectively, both of which are pre-trained encoders. ,in The pre-trained encoder is used at the beginning of the second phase. Initialize the weights; freeze the teacher encoder. All network parameters are fixed and do not undergo gradient updates throughout the training process, making them a fixed and unchanging original pre-trained knowledge base.

[0152] S3-1-2: Enter a sentence By simultaneously processing sentences using both teacher encoders and student encoders... Encode the data and extract three types of representations;

[0153] Enter sentence Simultaneously input the teacher encoder With student encoder In each case, three complete sets of semantic vectors are extracted.

[0154] ;

[0155] ;

[0156] , ;

[0157] in, The contextual representation semantic vector extracted for the student encoder. The semantic vector of the head entity extracted for the student encoder. The semantic vector of the tail entity extracted for the student encoder. The contextual representation semantic vector extracted for the teacher encoder. The semantic vector of the head entity extracted by the teacher encoder. The semantic vector of the tail entity extracted by the teacher encoder.

[0158] S3-1-3: Calculate the cosine distance between the teacher encoder and the student encoder;

[0159] The cosine distance of a single set of representations is calculated using the following formula:

[0160] ;

[0161] in, Represents the cosine distance. Cosine similarity between vectors Semantic representation index ; This represents the student representation vector corresponding to the semantic components; The teacher representation vector representing the corresponding semantic component;

[0162] S3-1-4: Calculate the total characterization distillation loss;

[0163] ;

[0164] in, For the first in the current batch The total representation distillation loss for each input sentence Indicates the first The cosine distance between the semantic vectors of the teacher encoder and the student encoder of each input sentence's context representation. Indicates the first The cosine distance between the semantic vectors of the head entities of the teacher encoder and the student encoder for each input sentence. Indicates the first The cosine distance between the tail entity semantic vectors of the teacher encoder and student encoder for each input sentence. , This is the current batch size. Represents the set of positive integers;

[0165] This distillation loss It participates in the overall training as a regularization term;

[0166] The training objective is to act as a regularization term, guiding the student encoder to approximate the knowledge embedded in the frozen teacher encoder. This ensures that the feature space of the student encoder maintains a certain degree of semantic consistency between semantic representations.

[0167] S3-2: Training using a multi-directional semantic alignment strategy;

[0168] S3-2-1: The pre-trained encoder As an encoder for relation description, the structured relation description text is input into the encoder for encoding;

[0169] Given relation description ,in and They are Hypersemantic description of a middle entity, wherein, For the first The original text sequence describing the relationship; For the first The first of the original text sequences describing the relationship One word;

[0170] The complete description of the original relation is encoded as follows:

[0171] ;

[0172] The text encoding of the head entity hypersemantic description is as follows:

[0173] ;

[0174] The text encoding of the tail entity hypersemantic description is as follows:

[0175] ;

[0176] in, In the relation description, the first The representation vector of each context part; In the relation description, the first The representation vector of the head entity part; In the relation description, the first The representation vector of each tail entity part.

[0177] S3-2-2: Extract the beginning of the sentence as follows:

[0178] ;

[0179] in, This represents the overall representation vector of the contextual semantics in the relation description. This represents the representation vector of the first word in the context portion of the relation description. This represents the overall semantic representation vector of the head entity in the relation description. Let represent the representation vector of the first word in the head entity part of the relation description. This represents the overall semantic representation vector of the tail entity in the relation description. This represents the representation vector of the first word in the tail entity part of the relation description;

[0180] Then for the first batch The overall representation vectors of the contextual semantics, head entity semantics, and tail entity semantics of all relation descriptions are extracted as follows: , , ;

[0181] S3-2-3: Encoder after pre-training For the first batch The following representations of the input sentence are extracted at the context, head entity, and tail entity levels: , , ;

[0182] S3-2-4: Calculate the contrastive loss of each part of the context, head entity, and tail entity representation;

[0183] ;

[0184] in, It is the contrastive learning loss function (InfoNCE) corresponding to the semantic part; Represents the cosine similarity between vectors; Indicates the first in the batch The representation vector of the corresponding semantic part in each input sentence; Indicates the first in the batch The representation vector describing the positive sample relation of the corresponding semantic part in each input sentence; Indicates the first in the batch A semantic representation vector describing a relation; This represents temperature, which is an adjustable hyperparameter. This refers to the batch size.

[0185] Based on the above formula, the contrast loss of each part of the context, head entity, and tail entity representation is calculated respectively. , , ;

[0186] S3-2-5: Calculate the total loss of the semantic alignment main task;

[0187] ;

[0188] in, Indicates the semantic weight of the head entity. Indicates the semantic weight of the tail entity. Indicates the first The total loss of the semantic alignment main task for each input sentence. Indicates the first The contrastive loss is derived from the contextual representation of each input sentence. Indicates the first The contrastive loss is calculated based on the head entity representation of each input sentence. Indicates the first The tail entity representation contrastive loss of each input sentence;

[0189] We will use the exact same weight hyperparameters as before. , We then sum the weighted contrast losses of the three semantic classes to obtain the total loss for the semantic alignment main task.

[0190] S3-3: Calculate the global final total loss;

[0191] Introducing equilibrium hyperparameters We weight and fuse the main task comparison loss and the distillation regularization loss to obtain the final optimization objective throughout the training process:

[0192] ;

[0193] in, For the first The global final total loss for each input sentence. As a weighting factor, For the first The total representation distillation loss for each input sentence Indicates the first The total loss of the semantic alignment main task for each input sentence;

[0194] S4: Output the results using the trained model;

[0195] S4-1: Input the sentence to be predicted and all its candidate no-relationship tags gather ;

[0196] S4-2: The encoder, after training, processes the input sentences separately. Each candidate relation description is encoded, and its three semantic representations—context, head, and tail entities—are extracted.

[0197] S4-3: Calculate the matching score between the sentence input and each candidate relation using cosine similarity;

[0198] Use weights during the training phase , Calculate the current sentence With the A comprehensive matching score for each candidate relation description;

[0199] ;

[0200] in, Represents the cosine similarity between vectors. Indicates the current sentence With the A comprehensive matching score for each candidate relation description; and These are the semantic weights of the head entity and the semantic weights of the tail entity, respectively; where... express a sentence Contextual semantic feature vectors express a sentence The feature vector of the head entity. express a sentence The tail entity feature vector; where, Indicates the first The contextual semantic feature vectors describing the candidate relations Indicates the first The head entity feature vectors described by the candidate relations Indicates the first Tail entity feature vectors described by candidate relations;

[0201] S4-4: Traverse all candidate relations and select the category with the highest matching score as the final predicted relation for the sentence;

[0202] The formula is as follows:

[0203] ;

[0204] in, For the final predicted relation of the sentence, To find the index corresponding to the maximum value of the independent variable, Indicates the current sentence With the The overall matching score for each candidate relation description.

[0205] To better illustrate this technical solution, two datasets widely used in the field of zero-sample relation extraction (ZSRE) (FewRel and Wiki-ZSL) were used for verification.

[0206] FewRel originates from Wikidata and has been refined through human annotation. This dataset contains 80 different relation categories. Each category contains 700 instances, ensuring a perfect balance in category distribution.

[0207] Wiki-ZSL is constructed from Wiki-KB using remote supervision. This dataset contains 113 relation categories and a total of 93,383 samples. Unlike FewRel, Wiki-ZSL exhibits a long-tailed distribution and inherent noise in its labels due to the remote supervision process.

[0208] In the experimental setup, the entire dataset was divided into training, validation, and test sets. Random selection... Five relations were used for testing, five for validation, and the remaining relations for training. Following the RE-Matching approach, to ensure the robustness and reproducibility of the experiments, all experiments were conducted on five independent dataset splits generated by five specific random seeds, and the average of the results from each set was taken as the final result.

[0209] This method was compared with different types of methods used in recent years. For semantic matching methods, ZS-BERT, RE-Matching, AlignRE, HCL, and EMMA were selected.

[0210] Among classification-based methods, GLiREL and ICA were selected.

[0211] Based on generative methods, Relation Prompt, REA, and SUMASK were selected. In addition, two representative methods, Re-Cent and NSP-RTE, were also selected. These two methods were proposed for zero-sample relation triple extraction and both showed strong effectiveness in standard relation extraction tasks.

[0212] BERT-base was used as a shared encoder for the input sentence and relation description, with AdamW set as the two-stage optimizer. In the first stage, the batch sizes for FewRel and Wiki-ZSL were set to 128 and 64, respectively, with a learning rate of 2e-5 and a dropout rate of 0.4. In the second stage, the batch size for both datasets was set to 64, with a learning rate of 1e-5. Set to 0.8 to balance the weights across different training objectives. Temperature in both phases. The number of training epochs and the number of training rounds were set to 0.02 and 5, respectively. The semantic weight hyperparameters for the two datasets were set to... All experiments were conducted using a single NVIDIA RTX 3090 24G graphics card.

[0213] Experimental Results and Analysis

[0214] To comprehensively evaluate the effectiveness of the model, three evaluation metrics were used on the FewRel and Wiki-ZSL datasets: precision, recall, and a weighted average of the two metrics, F1, as shown in the table below:

[0215] Table 1: Results of this technical solution and other models on the FewRel and Wiki-ZSL datasets.

[0216]

[0217] Table 1 shows the comparison results between our proposed solution and other baseline models under different numbers of unseen relation types (bold indicates best results, underline indicates second-best results). On the FewRel dataset, our proposed solution outperforms recent state-of-the-art (SOTA) models, achieving SOTA levels on all evaluation metrics; on the Wiki-ZSL dataset, our proposed solution still achieves second-best results. Compared with the baseline RE-Matching model, our proposed solution achieves significant improvements: on the FewRel dataset, F1 scores are improved by 3.65%, 6.14%, and 7.02%, respectively; while on the Wiki-ZSL dataset, improvements are 9.25%, 6.4%, and 4.9%, respectively. It can be seen that the superiority of our proposed solution is significant when the number of unseen relations increases. This fully demonstrates that our proposed solution not only maintains its lightweight advantage when facing more unseen relations, but also possesses excellent semantic modeling capabilities in zero-shot relation extraction tasks.

[0218] Table 2: Ablation Experiment Results of FewRel and Wiki-ZSL Datasets

[0219]

[0220] To investigate the impact of each component of this technical solution, ablation experiments were conducted on the FewRel and Wiki-ZSL datasets with no known relation m=15. The specific results are shown in Table 2.

[0221] To verify the effectiveness of fine-grained pre-training in optimizing downstream task performance, this part was removed, along with its coupled representation distillation part (without FPS), which led to a decrease in model performance. This demonstrates that pre-training using sentence-level information from the training set can effectively enhance the encoder's ability to model sentence-level semantic information. Specifically, the multi-granularity pre-training stage lays the foundation for optimizing the initial semantic space. By optimizing the bidirectional InfoNCE loss at three granularities—context, head entities, and tail entities—the encoder can learn and construct sentence feature representations that combine high alignment and high uniformity.

[0222] To verify the effectiveness of characterization distillation in supervised downstream matching tasks, this part was removed (w / oRD), i.e., the loss function corresponding to this task was removed, which led to a slight decrease in model performance. Results show that introducing this module improves the model's stability in zero-shot scenarios. Its core contribution lies in: by constraining the similarity between the student model and the teacher model in the feature space, it prevents, to some extent, from "catastrophic forgetting" when the model learns the matching logic for a specific task.

[0223] To verify the effectiveness of the multi-path semantic alignment strategy in fine-grained semantic alignment matching, the objective loss function for this part was first changed to the margin-loss (E1 w / o MSA) also used for fine-grained semantic matching in RE-Matching, and Sentence-BERT was used to pre-encode the relation description. Specifically, the experimental setup completely followed the definition in RE-Matching:

[0224] ;

[0225] in, This is a hyperparameter that controls the boundary distance. Experimental results show that, compared to the contrastive learning employed by the proposed Multi-way Semantic Alignment, the model's performance significantly decreases under the margin-loss training objective. On the FewRel and Wiki-ZSL datasets, when the number of unseen relations... At that time, the F1 score decreased by 6.57% and 3.78%, respectively. However, it's worth noting that this result is still slightly higher than the RE-Matching result, which indirectly demonstrates the effectiveness of the pre-training strategy. Margin loss imposes a fixed geometric distance constraint between positive and negative samples. In semantically complex zero-sample scenarios, this hard boundary is difficult to adapt to subtle semantic changes in high-dimensional feature spaces. In contrast, the multi-path semantic alignment mechanism based on contrastive learning can utilize global information within a batch, dynamically modeling relative similarity to more flexibly bring positive relationships closer and suppress negative interference, thereby constructing a more discriminative semantic space. Secondly, to demonstrate the importance of entity information for semantic matching (E2 w / o MSA), a specific setting was implemented. This approach, which retains only the weights of the contextual components, shows that in the absence of entity information, the F1 scores of this method on the FewRel and Wiki-ZSL datasets decreased significantly by 15.43% and 19.87%, respectively. This indicates that entity information is crucial for discriminating relational semantics in sentences, demonstrating that entity information can provide contextual clues related to relations.

[0226] The ablation experiments demonstrate that all the proposed methods have made significant contributions to varying degrees, and the combination of these methods makes the model robust and effective in handling zero-sample relation extraction tasks.

[0227] Table 3: Comparison of different relation description templates (m=15)

[0228]

[0229] To investigate the stability of this technical solution when faced with different relation description templates, the original static slot-filling template was replaced with a keyword template. The keyword template simply concatenates the hypernyms in the relation description without providing additional semantic hints. Table 3 shows that when the number of unseen relations m=15, the F1 score only decreased by 0.54% and 0.41% on the FewRel and Wiki-ZSL datasets, respectively. The model performance under the two templates showed high consistency with minimal differences. This indicates that this technical solution does not simply rely on a predefined syntactic shell, but can accurately capture and extract the core semantic information from the hypernyms. This insensitivity to template form demonstrates the robustness of the multi-granularity alignment mechanism, meaning the model has learned deep semantic associations, thus ensuring the performance stability and robustness of the model under different hint word template styles.

[0230] Table 4: Performance comparison of different backbone networks in RE-Matching and this technical solution (m=15)

[0231]

[0232] Since this technical solution employs a pure encoder architecture without introducing any additional neural network layers to further refine the encoder output representation, the model's performance depends entirely on the training method design. To verify the robustness of this technical solution, comparative experiments were conducted on various pre-trained BERT variants. Table 4 shows the comparative experimental results of this technical solution and RE-Matching on different backbone networks when the number of unseen relations m=15. The results show that this technical solution improves upon RE-Matching to varying degrees under different backbone networks. This indicates that the pre-training strategy and multi-directional semantic alignment strategy of this technical solution are robust on different backbone networks and do not depend on a specific encoder.

[0233] Table 5: FewRel processing time results for EMMA (m=10 / 15)

[0234]

[0235] As shown in Table 5, Table 5 uses two RTX-3090 GPUs to approximate the experimental setup reported in the original paper that uses RTX-A6000 as closely as possible.

[0236] To analyze the efficiency advantages of this technical solution, a lightweight comparative analysis was conducted on the inference speed, training time, and trainable parameter scale of the models under the settings of unseen relation number m=10 and m=15. To ensure fairness, each method was repeated five times and the average results were taken, and the batch size in the inference stage was uniformly set to 100. The experimental results are shown in Table 5. The comparative results show that this technical solution not only has advantages in performance, but its inference speed also reaches the current state-of-the-art level; in addition, despite adopting a two-stage training strategy, this technical solution still achieves second-best performance in training time, second only to AlignRE. It is speculated that this result is due to its encoder-only design, that is, the model training only relies on the representation directly output by the encoder. In terms of computational complexity in the inference stage, with an unseen relation number m, each relation corresponds to 700 samples. For the models RE-Matching and AlignRE, which use a dual encoder method with relation description precoding, the number of encodings required in the inference stage is For this technical solution, the relation description and sentence input share the same encoder, and the number of encodings required in the inference phase is... Even with a higher number of encoding iterations, this technical solution still achieves the best inference speed, which sufficiently demonstrates the lightweight architecture of this solution. Overall, compared to existing advanced methods, although the architecture of this technical solution is simple, it surpasses the performance of most methods of the same period in terms of performance and reaches the state-of-the-art (SOTA) level in inference speed.

[0237] Table 6: F1 scores of this technical solution using different parameter update strategies

[0238]

[0239] To improve training stability and preserve the general semantic foundation during the downstream relation extraction task stage, a local parameter freezing strategy is introduced. Considering that the lower layers of the Transformer encoder mainly model general grammatical and lexical information, while the higher layers are more focused on task-related semantics, the parameters of the first two layers closest to the input are frozen in the second training stage, preventing them from participating in gradient updates.

[0240] Specifically, the BERT-based encoding process is defined as follows:

[0241]

[0242] It is the first The representation of the layer output. And let... and Layer weights are not involved in parameter updates. As shown in Table 6, with other experimental settings unchanged, this technique improves the F1 score by 0.06%, 0.12%, and 0.22% and 0.27%, 0.19%, and 0.31% respectively on the FewRel and Wiki-ZSL datasets compared to full parameter updates. This strategy slightly alleviates representation drift during fine-tuning of downstream relation extraction tasks and plays a role in structure regularization. The model can focus more on learning fine-grained relation matching logic at higher levels, thereby enhancing generalization performance in zero-shot scenarios to some extent.

[0243] To visually demonstrate the stronger discriminative power of the semantic space learned by this technical solution for unseen relations, five unseen relations were randomly selected from FewRel and Wiki-ZSL, and their representations were visualized using t-SNE. Data points of the same color represent instances of the same relation type, such as... Figure 4 As shown in (a) and (b), on the FewRel dataset, the five unseen relations learned by this technical solution exhibit a compact intra-cluster structure and clear inter-cluster separation in two-dimensional space. The sample points corresponding to each relation basically form relatively independent clusters with well-defined boundaries, with minimal overlap between different relations, and the overall layout of the semantic space is relatively stable. This indicates that the relation representations learned by this technical solution have strong intra-class consistency and inter-class discriminative power. While the visualization results of RE-Matching still show a certain degree of relation aggregation trend, the overall structure is significantly looser, especially for the blue and yellow data points, corresponding to the relations "P1411:winner" and "P1346:award nomination" respectively. These two relations both belong to the semantic category of "honor and achievement," and RE-Matching cannot effectively distinguish between these two similar semantic relations. The Wiki-ZSL dataset suffers from imbalanced sample distribution and noise issues; therefore, this dataset more intuitively reflects the robustness of the model. Figure 4As shown in (c) and (d), on the Wiki-ZSL dataset, this technical solution can effectively distinguish the relationships corresponding to red, green, and yellow data points, forming clear relationship clusters. However, it cannot effectively distinguish the relationships "P361: part of" and "P144: based on," which both belong to the semantic category of "belonging to." Furthermore, the overall distribution of RE-Matching is relatively loose, with multiple relationships interspersed in the space, and the boundaries for distinguishing unseen relationships are unclear. By optimizing sentence representation quality through pre-training and introducing a fine-grained contrastive learning strategy, this technical solution can learn a more structured and discriminative semantic representation space, thus maintaining good clustering characteristics even when facing new relationships (especially similar relationships).

[0244] To evaluate the impact of key hyperparameters on model performance, hyperparameter analysis experiments were conducted on the FewRel dataset. The experiments primarily focused on the effects of two specific hyperparameters: those that regulate the weights of the representation distillation task and the contrastive learning task. And the dropout rate during the pre-training phase.

[0245] To analyze the impact of the dropout ratio on pre-training quality during the pre-training phase, experimental analysis was conducted on the FewRel dataset. Figure 5 As shown, when dropout is set to 0.1, the representations between augmented viewpoints are too similar, and the model tends to quickly fit the semantic distribution of sentences, thus limiting the learning of robust representations. When dropout is increased to 0.4, the model performance reaches its peak, indicating that stronger noise perturbations can provide effective regularization, forcing the model to capture semantic consistency under more challenging conditions, thereby learning more essential and invariant semantic features and significantly improving the generalization ability to unseen relations. Meanwhile, unsupervised SimCSE tasks (such as STS-B) typically achieve the best results with dropout=0.1, indicating that the multi-way semantic alignment mechanism introduced in downstream tasks has a higher tolerance for noise perturbations.

[0246] To analyze the weights of the loss function The impact on model performance was specifically demonstrated on the FewRel dataset by changing... The value of is used to observe its impact on model performance. For example Figure 6 As shown, it can be seen that the F1 value varies with... The changes generally show an upward trend. With the number of unseen relations m=10 and 15, when... The model achieves optimal performance on FewRel when the value is close to 0.8; in contrast, when m=5, It reaches its optimal value at 0.9. Smaller... (For example, 0.1, 0.2) will cause the model to overemphasize the semantic space of the alignment teacher model, thus making it difficult to effectively learn the relation matching task; while excessively large values ​​will lead to the model focusing too much on the semantic space of the alignment teacher model. Values ​​such as 0.8 or 0.9 would weaken the distillation constraint, leading to overfitting and consequently reducing the model's generalization ability. Based on the experimental results, prioritizing... Setting it to 0.8 means prioritizing the matching task while incorporating distillation constraints, thus achieving the optimal generalization balance.

[0247] In summary, this technical solution constructs a fine-grained semantic matching framework through a two-stage contrastive learning strategy and a multi-path semantic alignment mechanism, and explicitly models the multi-directional interactions between context, head entities, and tail entities, thereby effectively enhancing the feature alignment capabilities at the sentence and relation levels. This design not only alleviates the representation collapse problem that pre-trained language models are prone to in zero-shot scenarios, but also amplifies subtle semantic differences between similar relations, enabling the model to more robustly cope with complex contextual interference. Furthermore, inspired by existing work, a contrastive pre-training strategy based on dropout noise is introduced, combined with a feature distillation module. Without adding additional learnable parameters, this module improves the model's robustness to semantic perturbations by constraining intermediate representations, and effectively alleviates the catastrophic forgetting problem during the fine-tuning stage, thereby further enhancing the model's generalization and discriminative capabilities on unseen relations.

Claims

1. A lightweight zero-sample relation extraction method based on a multi-path semantic alignment strategy, characterized in that, Includes the following steps: S1: Data and Model Preparation; S2: Pre-train the model; S2-1: Encode the input sentence; The labeled input sentence is fed into BERT-base to extract three types of feature vectors: global context, head entity, and tail entity. The hidden state sequence of all tokens is obtained and the three types of representations are extracted as follows: ； ； in, The vector representation of the location of the special marker [CLS]. Indicates the input sentence The context vector representation of the first token Indicates the input sentence No. The context vector representation of each token; For input sentence The special marker [CLS] is appended to the beginning of the sentence before inputting BERT-base. Indicates the input sentence The first token, Indicates the input sentence The second token, Indicates the input sentence The One token; A representation of contextual information; and These represent the information of the head entity and the tail entity, respectively; where, This is a vector representation of the location of the head entity's starting marker [Eh]. The vector representation of the location of the tail entity's starting marker [Et]; S2-2: Refined pre-training; S2-2-1: Prepare the positive and negative sample pairs needed for contrastive learning; Given a training batch, the batch contains a total of Each input sentence constitutes a batch sample set. Using the BERT-base model as the encoder and in the encoder Enable random Dropout masking in the Transformer network layer; Each input sentence within the batch is input into the encoder. The data is encoded twice to generate positive and negative sample pairs. Sentence Repeat the input to the encoder twice. In the middle, relying on different random Dropout masks in the two forward propagations, each type of semantic representation is... Two homologous enhancement vectors are generated respectively. , ,in, This indicates the first input to the encoder. The generated augmentation vector, This indicates the second input to the encoder. The generated augmented vector; Two augmentation vectors with the same input sentence and the same representation , As positive sample pairs; Use all the augmented vectors of all other sentences in the current mini-batch as negative sample pairs for the current sample; S2-2-2: Based on bidirectional InfoNCE, calculate the bidirectional InfoNCE loss for global context, head entity, and tail entity representation respectively; The bidirectional InfoNCE loss for the global context, head entity, and tail entity representations is calculated using the following formula: ； in, Indicates the first The sentence, the first The corresponding individual loss value for the semantic representation of the class, Represents the cosine similarity between vectors. This indicates the first input to the encoder. The generated augmentation vector, This indicates the second input to the encoder. The generated augmented vector; Indicates temperature hyperparameter; S2-2-3: Calculate the total global loss; ； in, The bidirectional InfoNCE loss represents the global context representation. The bidirectional InfoNCE loss represents the head entity representation. The bidirectional InfoNCE loss represents the tail entity representation; Indicates the semantic weight of the head entity. Indicates the semantic weight of the tail entity; S2-2-4: Total Loss Minimize as the optimization objective, perform backpropagation and gradient descent, and update the encoder. All network parameters; S2-2-5: Repeat steps S2-2-1 to S2-2-4 to iterate through all training data, completing the entire pre-training process, and finally outputting the encoder optimized through contrastive learning. ; S3: Train the pre-trained model; S3-1: Employs intermediate layer semantic representation distillation to directly align the vector feature spaces of the teacher model and the student model, while preserving underlying semantic knowledge; S3-2: Training using a multi-directional semantic alignment strategy; S3-3: Calculate the global final total loss; S4: Output the results using the trained model.

2. The lightweight zero-sample relation extraction method based on a multi-path semantic alignment strategy according to claim 1, characterized in that: Step S1 also includes: S1-1: Data preparation; S1-1-1: Process the input sentences in the training dataset; training dataset Each input sentence is marked with special tags [CLS], [Eh], [\Eh], [Et], and [\Et] to indicate the positions of the first and last entities in the input sentence; Indicates the first One input sentence, Indicates the first The header entity of an input sentence. Indicates the first The tail entity of the input sentence, Indicates the first Labels for each input sentence. This represents the i-th relation description. Indicates the size of the training dataset; S1-1-2: Processing the relational descriptions of the training dataset; S1-2: Initialize the model; BERT-base is used as the encoder.

3. The lightweight zero-sample relation extraction method based on a multi-path semantic alignment strategy according to claim 1, characterized in that: Step S3-1 also includes: S3-1-1: Initialize the teacher and student models; set up and These represent the student encoder and the teacher encoder, respectively, both of which are pre-trained encoders. Freeze the teacher encoder All network parameters are fixed and do not undergo gradient updates throughout the training process, making them a fixed, unchanging original pre-trained knowledge base. S3-1-2: Enter a sentence By simultaneously processing sentences using both teacher encoders and student encoders... Encode the data and extract three types of representations; Enter sentence Simultaneously input the teacher encoder With student encoder In each case, three complete sets of semantic vectors are extracted. ； ； ； ； in, The semantic vector representing the context extracted for the student encoder. The semantic vector of the head entity extracted for the student encoder. The tail entity semantic vector extracted for the student encoder. The semantic vector representing the context extracted for the teacher encoder. The semantic vector of the head entity extracted by the teacher encoder. The semantic vector of the tail entity extracted by the teacher encoder; S3-1-3: Calculate the cosine distance between the teacher encoder and the student encoder; The cosine distance of a single set of representations is calculated using the following formula: ； in, Represents the cosine distance. Cosine similarity between vectors Semantic representation index ; This represents the student representation vector corresponding to the semantic components; The teacher representation vector representing the corresponding semantic component; S3-1-4: Calculate the total characterization distillation loss; ； in, For the first in the current batch Total representation distillation loss for each input sentence Indicates the first The cosine distance between the semantic vectors of the teacher encoder and the student encoder of each input sentence's context representation. Indicates the first The cosine distance between the semantic vectors of the head entities of the teacher encoder and the student encoder for each input sentence. Indicates the first The cosine distance between the tail entity semantic vectors of the teacher encoder and student encoder for each input sentence. , This is the current batch size. It represents the set of positive integers.

4. The lightweight zero-sample relation extraction method based on a multi-path semantic alignment strategy according to claim 1, characterized in that: Step S3-2 also includes: S3-2-1: The pre-trained encoder As an encoder for relation description, the structured relation description text is input into the encoder for encoding; Given relation description ,in and They are Hypersemantic description of a middle entity, wherein, For the first The original text sequence describing the relationship; For the first The first of the original text sequences describing the relationship One word; The complete description of the original relation is encoded as follows: ； The text encoding of the head entity hypersemantic description is as follows: ； The text encoding of the tail entity hypersemantic description is as follows: ； in, In the relation description, the first The representation vector of each context part; In the relation description, the first The representation vector of the head entity part; In the relation description, the first The representation vector of the tail entity part; S3-2-2: Extract the beginning of the sentence as follows: ； in, This represents the overall representation vector of the contextual semantics in the relation description. This represents the representation vector of the first word in the context portion of the relation description. This represents the overall semantic representation vector of the head entity in the relation description. Let represent the representation vector of the first word in the head entity part of the relation description. This represents the overall semantic representation vector of the tail entity in the relation description. This represents the representation vector of the first word in the tail entity part of the relation description; Then for the first batch The overall representation vectors of the contextual semantics, head entity semantics, and tail entity semantics of all relation descriptions are extracted as follows: , , ; S3-2-3: Encoder after pre-training For the first batch The following representations of the input sentence are extracted at the context, head entity, and tail entity levels: , , ; S3-2-4: Calculate the contrastive loss of each part of the context, head entity, and tail entity representation; ， ； in, It is the contrastive learning loss function corresponding to the semantic part; Represents the cosine similarity between vectors; Indicates the first in the batch The representation vector of the corresponding semantic part in each input sentence; Indicates the first in the batch The representation vector describing the positive sample relation of the corresponding semantic part in each input sentence; Indicates the first in the batch A semantic representation vector describing a relation; This represents temperature, which is an adjustable hyperparameter. Batch size; Based on the above formula, the contrast loss of each part of the context, head entity, and tail entity representation is calculated respectively. , , ; S3-2-5: Calculate the total loss of the semantic alignment main task; ； in, Indicates the semantic weight of the head entity. Indicates the semantic weight of the tail entity. Indicates the first The total loss of the semantic alignment main task for each input sentence. Indicates the first The contrastive loss is derived from the contextual representation of each input sentence. Indicates the first The contrastive loss is calculated based on the head entity representation of each input sentence. Indicates the first Contrast loss for tail entity representations of each input sentence.

5. A lightweight zero-sample relation extraction method based on a multi-path semantic alignment strategy according to claim 4, characterized in that: Step S3-3 also includes: Introducing equilibrium hyperparameters We then weight and fuse the main task comparison loss and the distillation regularization loss to obtain the final optimization objective throughout the training process, as shown in the following formula: ； in, For the first The global final total loss for each input sentence; As a weighting factor; For the first The total representation distillation loss for each input sentence; Indicates the first The total loss of the semantic alignment main task for each input sentence.

6. A lightweight zero-sample relation extraction method based on a multi-path semantic alignment strategy according to claim 1, characterized in that: Step S4 also includes: S4-1: Input the sentence to be predicted and all candidate relationship tags gather ; S4-2: The encoder, after training, processes the input sentences separately. Each candidate relation description is encoded, and its three semantic representations—context, head, and tail entities—are extracted. S4-3: Calculate the matching score between the sentence input and each candidate relation using cosine similarity; Use weights during the training phase , Calculate the current sentence With the A comprehensive matching score for each candidate relation description; ； in, Represents the cosine similarity between vectors. Indicates the current sentence With the A comprehensive matching score for each candidate relation description; and These are the semantic weights of the head entity and the semantic weights of the tail entity, respectively; where... express a sentence Contextual semantic feature vectors express a sentence The feature vector of the head entity. express a sentence The tail entity feature vector; where, Indicates the first The contextual semantic feature vectors describing the candidate relations Indicates the first The head entity feature vectors described by the candidate relations Indicates the first Tail entity feature vectors described by candidate relations; S4-4: Traverse all candidate relations and select the category with the highest matching score as the final predicted relation for the sentence; The formula is as follows: ； in, For the final predicted relation of the sentence, To find the index corresponding to the maximum value of the independent variable, Indicates the current sentence With the The overall matching score for each candidate relationship description.

Images