A knowledge-assisted dialogue text relation extraction method

Abstract

The application provides a knowledge-assisted dialogue text relation extraction method, which comprises the following steps: S1, a trigger word recognition task is designed to inject trigger word knowledge into a pre-trained language model; S2, a label semantic representation learning task is designed to inject label semantic knowledge into the pre-trained language model; and S3, a relation extraction task based on prompt learning is used to mine the knowledge in the pre-trained language model and perform relation extraction. The method improves the utilization rate of trigger word knowledge and label semantic knowledge of the model, improves the performance and reasoning efficiency of dialogue relation extraction, and meets the needs of constructing a knowledge graph in a specific field and improving the performance of a dialogue system.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular, to a knowledge-assisted method for extracting relationships from dialogue text. Background Technology

[0002] In recent years, with the popularization of the internet, smartphones, and communication technologies, modern society has entered the information age. The internet generates massive amounts of information and data, and people desire to quickly and accurately obtain important information; thus, knowledge graph technology has emerged. Knowledge graphs are an information retrieval technology, and relation extraction is a crucial component of knowledge graph construction. Literature review shows that most existing relation extraction technologies are based on entities in sentences and written documents, while research on relation extraction from dialogue texts is limited. In reality, the amount of dialogue texts far exceeds that of sentences and written documents. With the evolution of artificial intelligence technology, human-computer interaction based on ChatGPT and related large-scale models will generate a large amount of dialogue text. Dialogue texts contain a wealth of relational facts. Therefore, relation extraction from dialogue texts will contribute to the improvement of intelligent systems such as knowledge graph construction, human-computer question answering, chain-of-thought, and multi-turn human-computer dialogue.

[0003] Currently, dialogue texts are characterized by low information density and high colloquialism. Therefore, dialogue text relation extraction models often capture irrelevant information and fail to focus on relevant information, ultimately leading to low model performance. This problem is difficult to solve. In recent years, many research efforts have attempted to address it, but none have achieved satisfactory results. The purpose of relation extraction is to extract the relationship between two specified entities from a text, which can be a sentence, an article, or a dialogue, etc. In dialogue text relation extraction, the input dialogue text is assumed to be represented as D = s1:u1, s2:u2…s m :u m , where s i and u i Let represent the speaker and corresponding statement in the i-th round of dialogue, respectively, and m be the round number of the dialogue. Given two entities in the dialogue, a = (a1, a2), the task of dialogue relation extraction is to predict the relationship r between entity pairs (a1, a2) from a predefined relation set R.

[0004] like Figure 2 As shown, dialogue text relation extraction aims to determine the type of relationship between two entities from the dialogue text. For example, entities "Frank" and "S2" have a sibling relationship, while entities "S2" and "Phoebe" have an alias relationship. Figure 2It can be observed that the information density of the dialogue text is relatively low. Of the seven rounds of dialogue, only two are helpful for relation extraction. In these two rounds, the most important information is the trigger words. For example, in the relation extraction between "Frank" and S2, "brother" is the trigger word, indicating that the two entities are siblings. Therefore, trigger words are rich in information and provide crucial clues for predicting the relationship type between entities, playing a decisive role. In practical applications, trigger word information can only be obtained from the training set. Therefore, how to fully utilize the trigger word information in the training set is very important for relation extraction from dialogue text.

[0005] In response, Lin et al. [1] proposed a dialogue relation extraction model based on trigger word enhancement, TREND, such as Figure 3 As shown in the figure, the process of this model is as follows: 1) The Explicit Trigger Gate module in the figure is used to predict whether there is a trigger word in the dialogue text; 2) If there is a trigger word, the position of the trigger word in the text is predicted by the Trigger Prediction module, and the predicted trigger word features are fused with the context features (Context Vector); 3) Finally, the fused features are used to predict the relationship.

[0006] The main problems with the background technology are as follows: Existing trigger word-assisted dialogue relation extraction models have three drawbacks: 1) When utilizing trigger words, existing technologies require a trigger word extraction module to identify trigger words in the dialogue text, and then fuse the trigger words with contextual features, using the fused features to assist in relation extraction. This method cannot ensure that the trigger word extraction results are completely accurate, and incorrect trigger words will introduce noise, affecting the model's performance. 2) This method also requires a trigger word extraction module to extract trigger words during the inference stage, and this module requires a significant amount of computation time, resulting in low overall inference efficiency of the model. 3) This method ignores label semantic knowledge.

[0007] The difficulty in solving the above problem lies in the fact that existing technologies generally encode tags as one-hot vectors, then obtain the probability distribution of relationship categories through a model, and finally calculate the cross-entropy loss between the two. This approach treats each category as an independent vector, without considering the correlation between categories. For example, "alumni" and "classmates" have similar semantics and are close in semantic space, while "alumni" and "parents" are distantly related. The correlation of these relationships reflects that the tag text of each relationship contains rich semantic knowledge, which this type of method ignores, thus limiting the model's performance.

[0008] The significance of solving the above technical problems is that it can improve the performance of dialogue relationship extraction and reasoning efficiency, and meet the needs of building knowledge graphs in specific domains and improving the performance of dialogue systems. Summary of the Invention

[0009] This invention provides a knowledge-assisted method for extracting dialogue text relations. This method improves the model's utilization of trigger word knowledge and tag semantic knowledge, enhances the performance of dialogue relation extraction and reasoning efficiency, and meets the needs of constructing knowledge graphs for specific domains and improving the performance of dialogue systems.

[0010] The technical solution of the present invention is as follows:

[0011] The present invention provides a knowledge-enhanced dialogue relation extraction method based on cue learning, comprising the following steps: S1. Designing a trigger word recognition task to inject trigger word knowledge into a pre-trained language model; S2. Designing a label semantic representation learning task to inject label semantic knowledge into a pre-trained language model; and S3. Using a cue learning-based relation extraction task to mine knowledge in the pre-trained language model and extract relations.

[0012] Optionally, in the above-mentioned knowledge-enhanced dialogue relation extraction method based on prompt learning, in step S1, the trigger word recognition task replaces the trigger words in the dialogue text with the [MASK] symbol during the training phase and predicts the real trigger words, so that the model learns the knowledge of trigger words.

[0013] Optionally, in the above-described knowledge-enhanced dialogue relation extraction method based on prompting learning, the input text is first preprocessed: if the training sample entity pair (a1, a2) contains a trigger word, the trigger word is replaced with the [MASK] symbol; if the entity pair does not contain a trigger word, 10% of the words in the input dialogue text D are randomly selected and replaced with the [MASK] symbol.

[0014] Process the dialogue text D using the method described above to obtain Di. MLM We obtain input 2 for the masked language model:

[0015] X MLM =[[CLS],D MLM [SEP],a1,Y text [a2]

[0016] Among them, Y text To represent the text labeled with relational tags, during training, a pre-trained language model is used to predict the true word at the [MASK] position, and the negative log-likelihood loss function is used as the loss function for this task.

[0017]

[0018] Where M is the set of masked words, |V| is the vocabulary size, θ is the parameter of the pre-trained language model, and m represents the probability distribution of the [MASK] character.

[0019] Optionally, in the above-mentioned knowledge-enhanced dialogue relation extraction method based on prompting learning, in step S2, unused words in the pre-trained language model are used as label words, and the features of the label text are used for initialization.

[0020] Optionally, in the above-described knowledge-enhanced dialogue relation extraction method based on prompting learning, in step S2, for each relation type r, assuming its text is t, the initial weights of the relation's tags are calculated as follows:

[0021]

[0022] Where W represents the weights of the word embedding layer of the pre-trained language model, and k represents the length of the labeled text.

[0023] The input to the semantic representation of the label learning task consists of dialogue text and a template, as shown below:

[0024] Z=[T1]a1[T2][MASK][T3]a2[T4]

[0025] Here, a1 and a2 represent two entities with a semantic relationship, while [T1], [T2], [T3] and [T4] are trainable prompt characters, and [MASK] is a masking symbol.

[0026] Optionally, in the above-mentioned knowledge-enhanced dialogue relation extraction method based on cue learning, for supervised contrastive learning, given input samples X = {x1, x2, ..., x...} N} and the corresponding label Y = {y1, y2, ..., y N}, where N represents the batch size, and the loss function for supervised contrastive learning is shown in the following equation:

[0027]

[0028] Among them, h i (i∈{1,2,…,N}) is the input text x i The embedding vector of the [MASK] character in the projection space, h i ·h j h i and h j The inner product, h i (i∈{1,2,…,N}) are the standardized features, and the inner product is equivalent to the cosine similarity of the vectors; τ represents the temperature coefficient, which is used to control the smoothness of training and the discriminative power of hard samples.

[0029] Optionally, in the above-described knowledge-enhanced dialogue relation extraction method based on cue learning, step S3, the relation extraction task based on cue learning, includes the following steps:

[0030] 1) Text preprocessing: Assume the input dialogue text is D = s1:u1, s2:u2, ..., s m :u m , where s i and u i Let a and b represent the speaker and corresponding statement in the i-th round of dialogue, respectively, where m is the round number of the dialogue, and the entity pair is a = (a1, a2). First, construct the text based on the dialogue text D.

[0031]

[0032] Among them, [S1] and [S2] are two newly introduced special markers that represent speaker entities;

[0033] 2) Template Construction: Add prompt text Z to the dialogue document, constructed as follows:

[0034] Z=[T1]a1[T2][MASK][T3]a2[T4]

[0035] Here, a1 and a2 represent two entities with a semantic relationship, while [T1], [T2], [T3] and [T4] are trainable cue characters, and [MASK] is a masking symbol commonly used in masked language models, which is used here for the relation extraction task;

[0036] 3) Input Construction: The input sequence for the final pre-trained language model consists of dialogue text. Concatenating it with the prompt text Z yields:

[0037] X1=[CLS]D[SEP][T1]a1[T2][MASK][T3]a2[T4][SEP]

[0038] Wherein, [CLS] and [SEP] are the start symbol and separator of the pre-trained language model, respectively;

[0039] 4) Tag mapping: When the prompt text X1 is input into the pre-trained language model, the pre-trained language model can calculate the probability distribution of the relation type through the probability distribution of [MASK]:

[0040] p(r|X1)=softmax(p([MASK]=V i |X1)))(1≤i≤n)

[0041] Among them, V iis the label for relation type i, and n is the number of relation types.

[0042] The beneficial effects of the technical solution of the present invention are as follows:

[0043] The knowledge-enhanced dialogue relation extraction method based on prompting learning of the present invention improves the model's utilization of trigger word knowledge and tag semantic knowledge, enhances the performance of dialogue relation extraction and inference efficiency, and meets the needs of constructing knowledge graphs in specific domains and improving the performance of dialogue systems. The method of the present invention can also be transferred to other relation extraction domains, such as document relation extraction. The supervised contrastive learning of the present invention can be extended to contrastive learning.

[0044] To better understand and illustrate the concept, working principle, and effects of this invention, the invention will be described in detail below with reference to the accompanying drawings and specific embodiments: Attached Figure Description

[0045] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the accompanying drawings used in the description of the specific embodiments or the prior art will be briefly introduced below.

[0046] Figure 1 This is a flowchart of the knowledge-assisted dialogue text relation extraction method of the present invention;

[0047] Figure 2 This is a schematic diagram of dialogue-oriented relation extraction in existing technologies;

[0048] Figure 3 This is a diagram of existing mainstream dialogue relationship extraction methods based on trigger words;

[0049] Figure 4 This is a flowchart illustrating the specific operations of the knowledge-assisted dialogue text relation extraction method of the present invention.

[0050] Figure 5 This is a schematic diagram of the label representation learning task involved in the method of the present invention. Detailed Implementation

[0051] To make the objectives, technical methods, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific examples. These examples are merely illustrative and not intended to limit the scope of the invention.

[0052] The knowledge-enhanced dialogue relation extraction method based on prompting learning of the present invention includes the following steps:

[0053] S1. Design a trigger word recognition task to inject trigger word knowledge into the pre-trained language model;

[0054] S2. Design a label semantic representation learning task to inject label semantic knowledge into a pre-trained language model;

[0055] S3. Employ a prompt-based learning-based relation extraction task to mine knowledge from the pre-trained language model and extract relations.

[0056] The detailed operation flowchart of the knowledge enhancement dialogue relationship extraction method based on prompting learning of the present invention is as follows: Figure 4 As shown, the label semantic representation learning task and the trigger word recognition task are performed only during the training phase, while the relation extraction task is performed during both the training and testing phases. The pre-trained language model uses the RoBERTa-base / large version.

[0057] The following is combined Figure 4 The tasks of trigger word recognition, label semantic representation learning, and relation extraction based on prompting are explained in detail.

[0058] 1. Trigger word recognition task

[0059] Existing technologies, when utilizing trigger words, require a trigger word extraction module to identify trigger words in the dialogue text, and then fuse the trigger words with contextual features, using the fused features to assist in relation extraction. This method has two drawbacks: 1) It cannot guarantee completely accurate trigger word extraction results, and incorrect trigger words introduce noise, affecting model performance. 2) This method also requires the trigger word extraction module during the testing phase, which consumes significant computation time, leading to low overall inference efficiency and limited practicality of the relation extraction model.

[0060] To address the aforementioned problems, this invention designs a trigger word recognition task based on a masked language model. During the training phase, this task replaces trigger words in the dialogue text with the [MASK] symbol and predicts the actual trigger words, enabling the pre-trained language model to learn the knowledge of trigger words. The task first preprocesses the input text: if the training sample entity pair (a1, a2) contains a trigger word, the trigger word is replaced with the [MASK] symbol. Taking the training sample "Do you know Phoebemarried Jim?" as an example, the trigger word in this input text is "married," so "married" is replaced with [MASK]. If the entity pair does not contain a trigger word, 10% of the words in the input dialogue text D are randomly selected and replaced with the [MASK] symbol.

[0061] Process the dialogue text D using the method described above to obtain Di. MLM We obtain input 2 for the masked language model:

[0062] X MLM=[[CLS],D MLM [SEP],a1,Y text [a2]

[0063] Among them, Y text This represents the text labeled with the relation. During training, a pre-trained language model is used to predict the true word at the [MASK] position, and the negative log-likelihood loss function is used as the loss function for this task.

[0064]

[0065] Where M is the set of masked words, |V| is the vocabulary size, θ is the parameter of the pre-trained language model, and m represents the probability distribution of the [MASK] character. This task is the same as the masked language model of the pre-trained language model [5], so the knowledge of trigger words can be effectively injected into the pre-trained language model through this task.

[0066] 2. Label semantic representation learning task

[0067] This invention discovers that the labeled text of relation categories contains rich and complex semantic knowledge, and a single word may not be sufficient to represent such a wealth of information. For example, the relation "per:positive_impression" encompasses various meanings of words describing a positive impression, such as "talented," "beautiful," and "wonderful." "Positive_impression" is a relatively abstract term; using it as a label would result in a semantically limited scope, failing to cover the rich semantic information related to positive impressions. Furthermore, some relation pairs within the category set have similar semantics, such as "dates" and "positive impression," boyfriend / girlfriend relationships and married relationships, "school_attended" and "member_of," etc. These relation pairs exhibit semantic overlap, making them difficult to distinguish. Therefore, to enable the model to fully utilize label semantic knowledge and enhance its ability to distinguish similar relation semantics, this invention introduces label semantic knowledge based on cue learning. Specifically, label words are treated as trainable tokens. [2] And introduce supervised contrastive learning [3]( Supervised Contrastive Learning (SCL) methods are used to learn representations of label words. For example... Figure 5As shown, supervised contrastive learning can make the feature representations of samples of the same class [MASK] in the embedding space closer, and make the representations of samples of different classes farther apart in the space, thereby enabling the model to learn more discriminative features.

[0068] To construct this task, this invention uses unused tokens from a pre-trained language model (PLM) as tag words, and initializes them using features from the tag text. Specifically, for each relation type r, assuming its text is t, the initial weights of the tag words for that relation are calculated as follows:

[0069]

[0070] Where W represents the weights of the word embedding layer in the pre-trained language model, and k represents the length of the labeled text. For example, the word embedding weights of the relation “per:employee_or_member_of” (person:employee / member) are the average of the word embedding weights of the five words {person,employee,or,member,of}.

[0071] The input for this task consists of dialogue text and a template. The template is as follows:

[0072] Z=[T1]a1[T2][MASK][T3]a2[T4]

[0073] Here, a1 and a2 represent two entities with a semantic relationship, while [T1], [T2], [T3] and [T4] are trainable prompt characters, and [MASK] is a masking symbol.

[0074] The following section introduces supervised contrastive learning, given input samples X = {x1, x2, ..., x...} N} and the corresponding label Y = {y1, y2, ..., y N}, where N represents the batch size, and the loss function for supervised contrastive learning is shown in the following equation:

[0075]

[0076] Where h i (i∈{1,2,…,N}) is the input text x i The embedding vector of the [MASK] character in the projection space, h i ·h j h i and h j The inner product of h. i(i∈{1,2,…,N}) are the standardized features, so the inner product is equivalent to the cosine similarity of the vectors. τ represents the temperature coefficient, used to control the smoothness of training and the discriminative power of hard samples.

[0077] 3. Relation extraction task based on prompting learning

[0078] Since both the pre-training task and the trigger word prediction auxiliary task of the pre-trained language model are in the form of Masked Language Model (MLM), this invention transforms relation extraction into an MLM task format, reducing the gap between the three tasks and thus better utilizing the knowledge of the pre-trained language model during the testing phase. The input to this task is the same as the input to the label semantic representation learning task, such as... Figure 4 The input to the masked language model is shown in Figure 1. The relation extraction method is to judge the probability of [MASK] on all relation tags by the pre-trained language model, and the tag category corresponding to the tag with the highest probability is the prediction result of relation extraction.

[0079] The relation extraction task based on prompting learning proposed in this invention includes the following steps:

[0080] 1) Text preprocessing: First, the text preprocessing method used by Yu et al. was adopted. [1] The proposed method preprocesses the dialogue text, assuming the input dialogue text is D = s1:u1, s2:u2, ..., s m :u m , where s i and u i Let a and b represent the speaker and corresponding statement in the i-th round of dialogue, respectively, where m is the round number of the dialogue, and the entity pair is a = (a1, a2). First, construct the text based on the dialogue text D.

[0081]

[0082] Among them, [S1] and [S2] are two newly introduced special markers that represent speaker entities.

[0083] 2) Template Construction: Add prompt text Z to the dialogue document, constructed as follows:

[0084] Z=[T1]a1[T2][MASK][T3]a2[T4]

[0085] Here, a1 and a2 represent two entities with a semantic relationship, while [T1], [T2], [T3] and [T4] are trainable cue characters, and [MASK] is a masking symbol commonly used in masked language models, used here for the relation extraction task.

[0086] 3) Input Construction: The input sequence for the final pre-trained language model consists of dialogue text. Concatenating it with the prompt text Z yields:

[0087] X1=[CLS]D[SEP][T1]a1[T2][MASK][T3]a2[T4][SEP]

[0088] Here, [CLS] and [SEP] are the start and separator symbols of the pre-trained language model, respectively. By adding this template, the relation extraction task is transformed into a "cloze test" task in the form of a masked language model.

[0089] 4) Tag mapping: When the prompt text X1 is input into the pre-trained language model, the pre-trained language model can calculate the probability distribution of the relation type through the probability distribution of [MASK]. [4] :

[0090] p(r|X1)=softmax(p([MASK]=V i |X1)))(1≤i≤n)

[0091] Where V i is the label for relation type i, and n is the number of relation types.

[0092] Model training and loss function design

[0093] This invention employs a multi-task approach to train the model. The loss functions for the trigger word prediction task based on the masked language model and the label representation learning task based on supervised contrastive learning have been described previously. The relation extraction task uses Binary Cross Entropy (BCE) as its loss function.

[0094]

[0095] Where y r ∈{0,1} indicates whether r is in the label set, and n represents the total number of relation categories.

[0096] During the training phase, the total loss function of the algorithm of this invention is a weighted sum of the loss from the trigger word prediction task based on the masked language model, the loss from the label representation learning task based on supervised contrastive learning, and the loss from the relation extraction task. The calculation method is shown in the following formula:

[0097]

[0098] Where, λ RE , λ SCL and λ MLMThese are the weighting coefficients for the three tasks. During the testing phase, only the relation extraction task is included in the calculation.

[0099] Model testing:

[0100] The steps in the testing phase of this invention are the same as those in step S3, which is the relation extraction task based on cue learning.

[0101] Experimental results:

[0102] This invention is a Knowledge Enhanced Method based on Prompt Tuning for Dialogue Relation Extraction (KEPT-DRE). In the following experimental results section, the method of this invention will be referred to as KEPT-DRE.

[0103] 1. Comparison with mainstream dialogue text relation extraction methods:

[0104] As can be seen from Table 1 below, the method of the present invention outperforms all existing methods on both Chinese and English datasets.

[0105] Table 1 compares the performance of the method of this invention with existing methods on Chinese and English datasets.

[0106]

[0107]

[0108] 2. Robustness verification

[0109] To verify the robustness of the algorithm in this invention, experiments were conducted in a low-resource scenario. Table 2 shows the experimental results of each method on the DialogRE dataset in a low-resource scenario, where K represents the number of samples for each class in the training set. For fair comparison, all methods used the RoBETRa-base pre-trained model. Analyzing the experimental results under four different training set sizes based on the F1 evaluation metric, the following conclusions can be drawn:

[0110] The method of this invention maintains superior performance even in low-resource scenarios. The KEPT-DRE method proposed in this invention can maintain good robustness in the case of scarce data, especially when samples with K=8 are extremely scarce, and its performance exceeds the baseline method by 4.9%.

[0111] Table 2 shows the experimental results of each method in low-resource scenarios on the DialogRE dataset.

[0112]

[0113] 3. Comparison of inference speeds of the models

[0114] In practical applications, inference speed is crucial for relation extraction from dialogue text. Previous methods primarily modeled trigger words by extracting and fusing their information, resulting in slow inference speeds. The KEPT-DRE method of this invention, however, introduces trigger word information through auxiliary task training, without involving trigger word extraction during inference. Table 3 below shows the time required for each method to test all samples on the DiaogRE-Chinese test dataset. All methods are based on the RoBERTa base model. Except for the RoBERTa model, all other models utilize trigger word information. The GRASP model and the proposed KEPT-DRE method are both cue-based learning methods, while the other baseline models are fine-tuned. Compared to the RoBERTa model, the KEPT-DRE method performs slightly better; however, by incorporating trigger word knowledge, it achieves superior relation extraction performance. Compared to fine-tuning-based methods, the KEPT-DRE method achieves a 6.8x speedup in inference compared to TREND and TLAG-DRE, because these models incorporate an additional module to extract trigger words during inference. Compared to the cue-based GRASP model, the KEPT-DRE method achieves a 4.95x improvement in inference efficiency. In summary, the KEPT-DRE method, without adding extra parameters or model structure, introduces trigger word information, sacrificing a small amount of inference efficiency for a significant improvement in relation extraction performance.

[0115] Table 3 shows the time required for each method to test all samples on the DiaogRE-Chinese test dataset.

[0116]

[0117] The knowledge-enhanced dialogue relation extraction method based on cue learning of this invention mainly includes two training tasks: 1) a trigger word recognition task based on a masked language model: trigger word knowledge is injected into the pre-trained language model during the training phase, avoiding error accumulation and additional inference overhead during the inference phase; 2) a label semantic representation learning task based on supervised contrastive learning: supervised contrastive learning is used to inject rich and highly class-discriminative label semantic knowledge into the model. In summary, this algorithm improves the model's utilization of trigger word knowledge and label semantic knowledge, enhances the performance of dialogue relation extraction and inference efficiency, and meets the needs of constructing knowledge graphs in specific domains and improving the performance of dialogue systems. The method of this invention can also be transferred to other relation extraction domains, such as document relation extraction; the supervised contrastive learning of this invention can be extended to contrastive learning.

[0118] The above description represents the preferred embodiment based on the inventive concept and working principle. The above embodiments should not be construed as limiting the scope of protection of these claims; other embodiments and combinations of implementations based on the inventive concept are all within the scope of protection of this invention.

[0119] [1]Lin PW, Su SY, Chen Y N.TREND: Trigger-Enhanced Relation-ExtractionNetwork for Dialogues[C] / / Proceedings of the 23rd Annual Meeting of the Special Interest Group on Discourse and Dialogue.2022:623-629.

[0120] [2]Zhang N, Li L, Chen X, et al.Differentiable Prompt Makes Pre-trainedLanguage Models Better Few-shot Learners[C] / / International Conference onLearning Representations.

[0121] [3]Khosla P, Teterwak P, Wang C, et al. Supervised contrastive learning [J]. Advances in neural information processing systems, 2020, 33: 18661-18673.

[0122] [4]Han X, Zhao W, Ding N, et al. Ptr: Prompt tuning with rules for textclassification[J]. AI Open, 2022, 3: 182-192.

[0123] [5]Devlin J,Chang M W,Lee K,et al.BERT:Pre-training of DeepBidirectional Transformers for Language Understanding[C] / / Proceedings of the2019 Conference of the North American Chapter of the Association forComputational Linguistics:Human Language Technologies,Volume 1(Long and ShortPapers).2019:4171-4186.

Claims

1. A knowledge-enhanced dialogue relation extraction method based on cue-based learning, characterized in that, Includes the following steps: S1. Design a trigger word recognition task to inject trigger word knowledge into the pre-trained language model. Specifically, First, the input text is preprocessed: if the training sample entity pairs If a trigger word exists, replace the trigger word with the [MASK] symbol; if the entity pair does not have a trigger word, randomly select 10% of the words in the input dialogue text D and replace them with the [MASK] symbol. Process the dialogue text D using the method described above to obtain... We obtain input 2 for the masked language model: in, To represent the text labeled with relational tags, during training, a pre-trained language model is used to predict the true word at the [MASK] position, and the negative log-likelihood loss function is used as the loss function for this task. in It is a collection of obscured words. It refers to the size of the vocabulary. These are the parameters of the pre-trained language model. This represents the probability distribution of the [MASK] character; S2. Design a label semantic representation learning task to inject label semantic knowledge into the pre-trained language model, specifically, For each type of relation Assuming its text is The initial weights of the tags for this relationship are calculated as follows: in, This represents the weights of the word embedding layer in the pre-trained language model. Indicates the length of the label text. The input to the label semantic representation learning task consists of dialogue text and a template, as follows: in, and This represents two entities that have a semantic relationship, while , , and [MASK] is a masking symbol, which is a trainable prompt character; and S3. A relation extraction task based on cue-based learning is used to mine knowledge from the pre-trained language model and perform relation extraction. Specifically, The relation extraction task based on prompting learning includes the following steps: 1) Text preprocessing: Assume the input dialogue text is , ,in and They represent the first The speakers and corresponding statements in a turn-based dialogue. It is the number of dialogue rounds, and the entity pairs are... First, based on the dialogue text Building text , : in, and These are two newly introduced special markers that represent speaker entities; 2) Template Construction: Add prompt text to the dialogue document. Its construction method is as follows: in, and This represents two entities that have a semantic relationship, while , , and [MASK] is a trainable cue character, a masking symbol commonly used in masked language models, and is used here for relation extraction tasks; 3) Input Construction: The input sequence of the final pre-trained language model consists of dialogue text. and prompt text The result obtained by piecing together: Wherein, [CLS] and [SEP] are the start symbol and separator of the pre-trained language model, respectively; 4) Tag word mapping: When the prompt text After being input into the pre-trained language model, the pre-trained language model can calculate the probability distribution of relation types using the probability distribution of [MASK]: in, It is a relation type Tags It represents the number of relation types.

2. The knowledge-enhanced dialogue relation extraction method based on prompting learning according to claim 1, characterized in that, In step S1, a trigger word recognition task based on a masked language model is designed. During the training phase, the trigger word recognition task replaces the trigger words in the dialogue text with the [MASK] symbol and predicts the real trigger words, so that the model learns the knowledge of trigger words.

3. The knowledge-enhanced dialogue relation extraction method based on prompting learning according to claim 1, characterized in that, In step S2, unused words from the pre-trained language model are used as tag words, and the features of the tag text are used for initialization.

4. The knowledge-enhanced dialogue relation extraction method based on cue learning according to claim 1, characterized in that, For supervised contrastive learning, given input samples and the corresponding tags , The loss function for supervised contrastive learning, representing the batch size, is shown in the following equation: in, It is the input text The embedding vector of the [MASK] character in the projection space. express and The inner product, These are standardized features, and the inner product is equivalent to the cosine similarity of vectors. This represents the temperature coefficient, used to control the smoothness of training and the discriminative power of difficult samples.

Images