A zero-shot relation extraction method based on Prompt multi-template fusion

Abstract

The present invention provides a zero-shot relationship extraction method based on Prompt multi-template fusion. The method comprises: combining text data from which relationships need to be extracted with predefined Prompt templates, inputting the combined data into a pretrained language model, outputting the results of a cloze task, and converting the word embedding output of the result into a relationship representation; repeatedly inputting the text data from which relationships need to be extracted into different types of pretrained language models to obtain multiple relationship representations; fusing the multiple relationship representations based on the relationship weights between the parts of speech in the relationship description text and different Prompt templates; comparing the Euclidean distances between the multiple fused relationship representations and the word embeddings generated from the relationship description text, converting the distances into corresponding relationship probabilities, and outputting the zero-shot relationship category with the highest probability for the text data from which relationships need to be extracted. The present method optimizes the model's ability to generate representations and improves the F1 value for zero-shot relationship extraction tasks.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence and natural language technology, and in particular to a zero-sample relationship extraction method based on Prompt multi-template fusion. Background Art

[0002] With the rapid development of internet technology, the amount of text data that needs to be processed is growing exponentially. Relationship extraction, a crucial foundational task in natural language processing, aims to extract relationships between entity pairs from unstructured text data, supporting multiple downstream tasks such as knowledge graph construction and intelligent question-answering. Currently, most relationship extraction algorithms can only identify relationship categories known during training. However, in the real world, it is clearly impossible to collect sufficient training samples for all relationships. To address this dilemma, zero-shot relationship extraction algorithms have emerged.

[0003] The existing zero-shot relationship extraction methods can be roughly divided into three categories: supervised relationship extraction models, textual implication models, and relationship representation generation models. The method of converting a supervised relationship extraction model into a zero-shot relationship extraction model is to use the relationship extraction model constructed by supervised learning and then extract features from the input sentence in different ways for post-prediction. The textual implication model method converts the zero-shot relationship extraction task into a textual implication task, and determines the relationship category by judging whether the sentence and the relationship description have a semantic implication relationship. Most of the relationship representation generation models use pre-trained language models. Specifically, the relationship representation in the sentence is generated by the model, the distance between the sentence representation and the class representation is compared, and the most appropriate category is selected.

[0004] The disadvantages of the zero-shot relation extraction methods in the prior art are as follows: the methods based on supervised relation extraction models have poor experimental performance because the model design does not consider the zero-shot extraction problem;

[0005] Methods based on textual implication models typically redesign the zero-shot relation extraction task into different task forms. This strategy often fails to form an effective relational semantic representation space, and there are inherent differences between tasks, which affects model performance.

[0006] In the model for generating relational representations, the training process may be unstable due to the different training objectives of the pre-trained language model and the downstream tasks. Summary of the Invention

[0007] The embodiments of the present invention provide a zero-sample relationship extraction method based on Prompt multi-template fusion to achieve effective extraction of zero-sample relationships from text data.

[0008] In order to achieve the above-mentioned purpose, the present invention adopts the following technical solutions.

[0009] A zero-shot relation extraction method based on Prompt multi-template fusion, including:

[0010] Construct a pre-trained language model for a cloze task. Combine the text data from which relationships need to be extracted with a pre-defined prompt template and input the data into the pre-trained language model. The pre-trained language model outputs the result of the cloze task, and the word embedding output of this result is converted into a relational representation.

[0011] The text data for extracting relationships is repeatedly input into different types of pre-trained language models to obtain multiple relationship representations, and the multiple relationship representations are fused according to the relationship weights between the parts of speech in the relationship description text and different prompt templates;

[0012] The Euclidean distance between the fused multiple relationship representations and the word embedding generated by the relationship description text is compared, the distance is converted into the probability of the corresponding relationship, and the zero-sample relationship category of the text data with the maximum probability of the relationship to be extracted is output.

[0013] Preferably, the pre-trained language model for constructing the cloze task combines the text data from which the relationship needs to be extracted with a pre-defined prompt template and inputs the combined data into the pre-trained language model. The pre-trained language model outputs the result of the cloze task, and the word embedding output of the result is converted into a relational representation, including:

[0014] The pre-trained language model is set to a cloze task. The text data from which relationships need to be extracted is combined with a pre-defined prompt template and then input into the pre-trained language model. The pre-trained language model outputs the result of the cloze task. The word embedding output of this result is converted into a relationship representation vector. The Euclidean distance between the relationship representation vector and each class representation vector is calculated, and each relationship representation vector is classified according to the distance of the Euclidean distance.

[0015] Preferably, the text data whose relations need to be extracted is combined with a predefined Prompt template, including:

[0016] Fill each sentence (x, s, o) to be extracted into the predefined Prompt template T n (s, o) is converted into a new token sequence, each of which represents a text unit or a supplementary symbol such as [MASK];

[0017]

[0018] T n (s, o) = [t0...t p ]s[t p+1 ...t q ]o[t q+1 ,,t |T(s，o)|-1 ] (1)

[0019] in is the constructed new input, T n (s, o) is a template selected from the commonly used Prompt template library, t num is a word in the template, satisfying 0≤num≤|T n(s，o) |-1, where |T n (s, o)| represents the length of the template 0≤p<q<T n (s, o), when combining the templates, it is also necessary to select a token sequence of length β at the α position in the template to mask, that is, T n (x) α：α+β =[MASK]0...[MASK] β-1 , β takes 1, SEP represents the sentence separator, which is the segmentation template and the sign indicating the end of the sentence.

[0020] Preferably, the fusion of the multiple relationship representations according to the relationship weights between the parts of speech in the relationship description text and different prompt templates includes:

[0021] In the training phase, the relationship weights between the parts of speech in the relation description text and the prompt templates are learned. In the testing phase, the weights between the parts of speech and the prompt templates are used to assign appropriate weights to different prompt templates.

[0022] Process the relation description to obtain T n The vector of the part of speech represented in the description text of each relationship under the template, Where |R| is the number of relation categories, H |R| ∈{h NN , h IN ,...,h JJ}, H |R| ∈R γ×d , γ is the number of types of part-of-speech tags, d′ is the dimension of the vector, h NN It is a feature vector representing the part of speech NN. By judging the part of speech of the relationship description text, a weight vector P is assigned to each relationship. pos .

[0023]

[0024] Among them, P pos={P1, P2, ..., P n}, P pos ∈R n×|R| Where n is the number of template types, |R| is the number of relation types, P n Represents the weight vector of each relationship obtained by the nth relationship template, f is a feedforward neural network, W H ∈R n ×|R| and b H ∈R n×|R| is the parameter matrix of learning, tanh represents hyperbolic tangent, which is a kind of activation function in neural network;

[0025] The multiple relationship representations are fused according to the parts of speech in the relationship description text and the relationship weights between different Prompt templates.

[0026] Preferably, the method further comprises:

[0027] The pre-trained language model is optimized by minimizing the loss, as shown in formula (3):

[0028]

[0029] Among them, when |R| is the number of relationship types to be classified, when the true value label is j, k j =1, otherwise, k j =0,P m，j Refers to the parameter matrix of the mth template to the jth relationship category, f distance (·) represents the calculation formula of Euclidean distance, Refers to the relational representation generated after the sentence is input into the pre-trained language model, which represents the word vector generated by the pre-trained language model for the description text. j Represents the word vector with label j.

[0030] It can be seen from the technical solutions provided by the above-mentioned embodiments of the present invention that the present invention proposes a multi-prompt template fusion method, which determines the weights of integrating different prompt templates according to the part of speech of the relationship description text, thereby optimizing the model's ability to generate representations and improving the F1 value of the zero-sample relationship extraction task.

[0031] Additional aspects and advantages of the present invention will be set forth in part in the following description, will become apparent from the following description, or may be learned by practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0033] Figure 1 A structural diagram of a multi-prompt template fusion zero-sample relationship extraction model provided by an embodiment of the present invention;

[0034] Figure 2 A structural diagram of a single Prompt template zero-sample relationship extraction model provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0035] The embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and are not to be construed as limiting the present invention.

[0036] It will be understood by those skilled in the art that, unless expressly stated otherwise, the singular forms "a", "an", "said" and "the" used herein may also include the plural forms. It should be further understood that the term "comprising" used in the description of the present invention refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be intermediate elements. In addition, "connected" or "coupled" as used herein may include wireless connections or couplings. The term "and / or" used herein includes any unit and all combinations of one or more associated listed items.

[0037] It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by those skilled in the art in the art to which the present invention pertains. It should also be understood that terms such as those defined in common dictionaries should be understood to have meanings consistent with their meanings in the context of the prior art and, unless defined as such herein, will not be interpreted in an idealized or overly formal sense.

[0038] To facilitate understanding of the embodiments of the present invention, several specific embodiments will be further explained below with reference to the accompanying drawings. However, each embodiment does not constitute a limitation on the embodiments of the present invention.

[0039] Example 1

[0040] The present invention proposes a zero-shot relationship extraction method that integrates multiple prompt templates. This method uses the prompt paradigm to perform relationship extraction, that is, the prompt paradigm is applied to the relationship extraction task. The use of multiple prompt template fusion methods can use the prompt paradigm to activate the knowledge stored in the pre-trained language model. At the same time, it can also solve the problem of large differences in the representation spaces generated by different templates. The core idea of zero-shot relationship extraction is to let computers simulate human reasoning and identify relationship categories that have never been seen from sentences. Prompt templates refer to some pre-designed sentences or words, which will be input into the model together with the sentences to be predicted.

[0041] The method of the present invention defines the zero-shot relationship extraction task as a masked language prediction task, aligns word embeddings with the representation space of semantic relationships, and compares the word vectors output by different templates with the semantic vectors of the relationship description text to determine the relationship category, effectively improving the performance of zero-shot relationship extraction. At the same time, by introducing part-of-speech information from the description text, a relationship extraction model that integrates multiple prompt templates is constructed, thereby achieving more stable zero-shot relationship extraction performance improvements.

[0042] The embodiment of the present invention provides a processing flow of a zero-sample relationship extraction method based on Prompt multi-template fusion, which includes the following steps:

[0043] Step S10: Relationship extraction layer based on the Prompt paradigm.

[0044] This method uses pre-trained language models to better excel at cloze tasks. A pre-trained language model is one that uses unsupervised learning to learn from a large corpus of data. For example, the pre-trained language model BERT uses cloze tasks and next sentence prediction tasks to learn from a large corpus of data. In addition to BERT, there are also Google's GPT series of models and Facebook AI's BART model.

[0045] The text data from which the relationship needs to be extracted is combined with a predefined prompt template and then fed into a pre-trained language model. The pre-trained language model outputs the results of a cloze task, and the word embeddings of these results are converted into relational representations. Specifically, the cloze task outputs the word in the masked position, and we use the word embedding of this masked word as the relation vector representing the relationship. For example, if we input the sentence "[CLS]…Mississippi River bridge to replace the deteriorating Cape Girardeau Bridge.[SEP]The cape Girardeaubridge[MASK]the Mississippi River.[SEP]", the pre-trained language model will output the word "crosses" to fill the masked position. The word embedding vector of this word is used as the relation representation vector. The sentence to be predicted is fed into the model, and the Euclidean distance between the resulting relation representation vector and each class representation vector is calculated. The relation representation vectors are then classified based on the distance.

[0046] Step S20: multiple prompt templates are fused into a layer based on part of speech.

[0047] The text data from which relationships need to be extracted is repeatedly input into different types of pre-trained language models according to step S10 to obtain multiple relationship representation vectors. The relationship representation vectors output by different pre-trained language models are then fused based on the relationship weights between the parts of speech in the relationship description text and different prompt templates.

[0048] Step S30: Output the result to the module layer

[0049] The Euclidean distance between the fused relationship representation vector and the word embedding vector generated from the relationship description text is compared. The distance is then converted into the probability of the corresponding relationship, and the zero-shot relationship category with the highest probability for the text data to be extracted is output.

[0050] The above step S10 specifically includes: prompt template design and masked language model (MLM) prediction.

[0051] The present invention fills each sentence (x, s, o) whose relationship is to be extracted into a predefined Prompt template, thereby converting it into a new token sequence, each of which represents a text unit or a supplementary symbol such as [MASK].

[0052]

[0053] Tn (s, o) = [t0...t p ]s[t p+1 ...t q ]o[t q+1 ,,t |T(s，o)|-1 ] (1)

[0054] in It is a new input constructed. This part mainly combines the input sentence and the Prompt template T n (s, o) are combined together. T n (s, o) is a template selected from the commonly used Prompt template library, t num is a word in the template, satisfying 0≤num≤|T n(s，o) |-1, where |T n (s, o)| represents the length of the template 0≤p<q<T n (s, o). When combining templates, it is also necessary to select a token sequence of length β at position α in the template to mask, i.e. f distance (·)T n (x) α，α+β =[MASK]0...[MASK] β-1 , where β is 1. [SEP] represents the sentence separator, specifically the segmentation template and the sign indicating the end of the sentence.

[0055] Step S20 specifically includes: a method for fusing multiple Prompt templates based on part of speech. During the training phase, the present invention learns the relationship weights between the parts of speech in the relationship description text and the Prompt templates. During the testing phase, the learned weights between the parts of speech and the Prompt templates are used to assign appropriate weights to different Prompt templates. Because the distribution of parts of speech in the training set and the test set is very similar, this method of connecting the training set and the test set using the part of speech of the description text as a bridge effectively solves the problem that arises when the model directly selects templates based on the relationship description text.

[0056] First process the relation description to obtain T n The vector representing the part of speech in the description text of each relationship under the template Where |R| is the number of relation categories. |R| ∈{h NN , h IN ,...,h JJ},H |R| ∈R γ×d′ , γ is the number of types of part-of-speech tags, d′ is the dimension of the vector, h NN It is a feature vector representing the part of speech NN. By judging the part of speech of the relationship description text, a weight vector P is assigned to each relationship.pos .

[0057]

[0058] Among them, P pos ={P1, P2, ..., P n},P pos ∈R n×|R| Where n is the number of template types and |R| is the number of relation types. n Represents the weight vector of each relationship obtained by the nth relationship template. f is a feedforward neural network, R n×|R| and b H ∈R n×|R| These are all learnable parameter matrices. Tanh stands for hyperbolic tangent, which is an activation function in neural networks.

[0059] The above step S30 specifically includes: the model converts the input (s, r, o) into The result of the MLM task is then fed into a pre-trained language model, serving as the relational representation. Prompt-fix LM tuning is used for training. When data is limited, the language model parameters are used directly. When data is sufficient, the pre-trained language model is fine-tuned to make it more suitable for zero-shot learning tasks. The different word vectors generated for sentences under different templates are measured against the vector representation of each candidate relation description to determine the relation to which the input instance belongs. Based on this, a loss function is constructed, and a distance-based optimization method is defined to optimize the model by minimizing the loss. This is shown in the formula.

[0060]

[0061] Where |R| is the number of relationship types to be classified, and k is the true value label j. j =1, otherwise, k j =0. P m，j It refers to the parameter matrix of the mth template to the jth relationship category. The function representing the measured distance is the calculation formula of the Euclidean distance in this patent. Refers to the relational representation generated after the sentence is input into the model, q represents the word vector generated by the pre-trained language model describing the text, q j Represents the word vector with label j.

[0062] Example 2

[0063] The structure diagram of a multi-prompt template fusion zero-sample relationship extraction model provided by an embodiment of the present invention is as follows: Figure 1As shown, the structure diagram of the single Prompt template zero-sample relationship extraction model is as follows Figure 2 As shown. The present invention mainly consists of a prompt-based relationship extraction layer, a multi-prompt template fusion layer, and a result output layer. The experimental data of the present invention comes from FewRel, which contains 80 relationships, of which 65 relationships are used for model training and 15 relationships are used for model testing. Each relationship has 700 sample instances generated by remote supervision, and these instances contain entity pairs consisting of subjects and objects. In addition, the text description of each relationship consists of a relationship name and a relationship description. The steps are as follows:

[0064] Data preprocessing stage: The sentences whose relationships are to be extracted and their corresponding subject and object combinations are filled into different templates and input into the model.

[0065] The training steps of the model are as follows:

[0066] (1) Multi-Prompt template fusion layer: First, process the relationship description to obtain T n The vector representing the part of speech in the description text of each relationship under the template

[0067] (2) Then, a weight vector P is assigned to each relationship by judging the part of speech of the relationship description text. pos·

[0068] (3) Relationship comparison layer of the Prompt template: Use Euclidean distance to compare the word vectors generated by MLM with the word vectors of the relationship description text.

[0069] (4) Result output layer: The results obtained using different templates are weighted and summed and input into the output layer to decode the results.

[0070] Following the above steps, the present invention was compared with commonly used zero-shot relation extraction models, including the supervised relation extraction models Att-Bi-LSTM and R-BERT. CIM and ESIM, which use Bi-LSTM to encode input sequences, were used to generate sentence representations of relations. The sentence representations were then compared with the class representations to select the most appropriate ZS-BERT model. Precision, recall, and F1 scores were used as evaluation metrics. The comparison results are shown in Table 1.

[0071] Table 1 Experimental results of the proposed model and other models on the Boson dataset

[0072] Att-Bi-LSTM R-BERT eSIM ZS-BERT Ours Accuracy P (%) 38.13 32.25 36.97 35.54 59.82 Recall rate R (%) 32.05 25.58 32.51 38.19 66.07 F1(％) 34.82 28.53 34.60 36.82 62.79

[0073] The experimental results show that:

[0074] By comparing this model with existing entity recognition models, we can see that the proposed model achieved the highest recognition performance in the experiment, which demonstrates the effectiveness of the model designed in this patent. This model has a high recall rate but a relatively low precision, which indicates that the model is particularly sensitive to certain classes and can well identify these target classes.

[0075] In summary, the embodiments of the present invention propose a zero-shot relationship extraction model based on multiple prompt templates. This method transforms the zero-shot learning relationship extraction task into a relationship representation generation task, and uses the prompt paradigm to activate the knowledge stored in the pre-trained language model. Furthermore, to address the problem of large differences in the representation spaces generated by different templates, the present invention proposes a multi-prompt template fusion method. This method determines the weights for integrating different prompt templates based on the part of speech of the relationship description text, thereby optimizing the model's ability to generate representations and improving the F1 value of the zero-shot relationship extraction task.

[0076] This method models the downstream task as a cloze task, which the pre-trained language model is more adept at. This method reduces the difference between the pre-trained language model training phase and the downstream task fine-tuning phase, thereby reducing the amount of data required for the model to output relationship representations. The word vectors generated by different templates are then directly used as relationship representations, and these representations are compared with the word vectors generated by the relationship description after the pre-trained language model is used. Building on this foundation, part-of-speech information of the descriptive text is introduced, and this is used as a basis for integrating the relationship extraction model of multiple prompt templates to obtain the weights of the output results of different templates.

[0077] Those skilled in the art will understand that the accompanying drawings are merely schematic diagrams of an embodiment, and the modules or processes in the accompanying drawings are not necessarily required to implement the present invention.

[0078] From the above description of the embodiments, it can be seen that those skilled in the art can clearly understand that the present invention can be implemented by means of software plus the necessary general-purpose hardware platform. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium such as ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in various embodiments of the present invention or certain parts of the embodiments.

[0079] Each embodiment in this specification is described in a progressive manner. The same or similar parts between the embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments. In particular, for the device or system embodiments, since they are basically similar to the method embodiments, the description is relatively simple. For the relevant parts, refer to the partial description of the method embodiments. The device and system embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the scheme of this embodiment. A person of ordinary skill in the art can understand and implement it without making any creative efforts.

[0080] The above description is merely a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A zero-sample relationship extraction method based on Prompt multi-template fusion, characterized in that: include: Construct a pre-trained language model for a cloze task. Combine the text data from which relationships need to be extracted with a pre-defined prompt template and input the data into the pre-trained language model. The pre-trained language model outputs the result of the cloze task, and the word embedding output of this result is converted into a relational representation. The text data for extracting relationships is repeatedly input into different types of pre-trained language models to obtain multiple relationship representations, and the multiple relationship representations are fused according to the relationship weights between the parts of speech in the relationship description text and different prompt templates; Compare the Euclidean distances of the fused multiple relationship representations with the word embeddings generated by the relationship description text, convert the distances into the probability of the corresponding relationships, and output the zero-sample relationship category of the text data with the maximum probability for the relationship to be extracted; The fusion of the multiple relationship representations according to the relationship weights between the parts of speech in the relationship description text and different prompt templates includes: In the training phase, the relationship weights between the parts of speech in the relation description text and the prompt templates are learned. In the testing phase, the weights between the parts of speech and the prompt templates are used to assign appropriate weights to different prompt templates. Process the relation description to obtain T n The vector of the part of speech represented in the description text of each relationship under the template, Where |R| is the number of relation categories, H |R| ∈{h NN ,h IN ,…,h JJ },H |R| ∈R γ×d , γ is the number of types of part-of-speech tags, |d| is the dimension of the vector, h NN It is a feature vector representing the part of speech NN. By judging the part of speech of the relationship description text, a weight vector P is assigned to each relationship. pos ; Among them, P pos ={P1,P2,…,P n },P pos ∈R n×|R| Where n is the number of template types, |R| is the number of relation types, P n Represents the weight vector of each relationship obtained by the nth relationship template, f is a feedforward neural network, W H ∈R n×|R| and b H ∈R n×|R| is the parameter matrix of learning, tanh represents hyperbolic tangent, which is a kind of activation function in neural network; The multiple relationship representations are fused according to the parts of speech in the relationship description text and the relationship weights between different Prompt templates.

2. The method according to claim 1, characterized in that The pre-trained language model for constructing the cloze task combines the text data from which the relationship needs to be extracted with the pre-defined prompt template and inputs the combined data into the pre-trained language model. The pre-trained language model outputs the result of the cloze task and converts the word embedding output of the result into a relational representation, including: The pre-trained language model is set to a cloze task. The text data from which relationships need to be extracted is combined with a pre-defined prompt template and then input into the pre-trained language model. The pre-trained language model outputs the result of the cloze task. The word embedding output of this result is converted into a relationship representation vector. The Euclidean distance between the relationship representation vector and each class representation vector is calculated, and each relationship representation vector is classified according to the distance of the Euclidean distance.

3. The method according to claim 2, characterized in that The combination of the text data whose relations need to be extracted and the predefined Prompt template includes: Fill each sentence (x, s, o) to be extracted into the predefined Prompt template T n (s,o) is converted into a new token sequence, each token sequence represents a text unit or a [MASK] supplementary symbol; T n (s,o)=[t0…t p ]s[t p+1 …t q ]o[t q+1 ,,t |T(s,o)|―1 ] (1) in is the constructed new input, T n (s,o) is a template selected from the commonly used Prompt template library, t num is a word in the template, satisfying 0≤num≤|T n(s,o) |―1, where |T n (s,o)| indicates the length of the template 0≤p <q<T n (s,o), when combining templates, it is also necessary to select a token sequence of length β at position α in the template to mask, that is, T n (x) α:α+β =[MASK]0…[MASK] β―1 , SEP represents the sentence separator, which is the segmentation template and the sign indicating the end of the sentence.

4. The method according to claim 1, wherein The method further comprises: The pre-trained language model is optimized by minimizing the loss, as shown in formula (3): Among them, when |R| is the number of relationship types to be classified, when the true value label is j, k j =1, otherwise, k j =0,P m，j Refers to the parameter matrix of the mth template to the jth relationship category, f distance (·) represents the calculation formula of Euclidean distance, Refers to the relational representation generated after the sentence is input into the pre-trained language model, which represents the word vector generated by the pre-trained language model for the description text. j Represents the word vector with label j.

Images