Two-stage low-resource dialogue state tracking method based on state value generation

Abstract

The application discloses a two-stage low-resource dialogue state tracking method based on state value generation, which comprises the following steps: adopting Prompt Learning fine-tuning pre-training language model to obtain all state value sets mentioned by the current round user from dialogue context; adopting self-training to use large-scale easily obtained dialogue data to enhance the state value generation effect in the low-resource scene; in order to filter out the low-confidence pseudo-label data in the self-training process, a state value evaluator is designed to evaluate the quality of the generated state value, and the training data of the evaluator is obtained by negative sampling on the limited training set data; finally, the pre-training language model is fine-tuned based on Prompt Learning to generate the corresponding slot for each generated state value. The application makes full use of the pre-training language model to complete the low-resource dialogue state tracking task, and adopts self-training to use the unlabeled dialogue data to enhance the state value generation effect. The effect of low-resource state tracking is greatly improved.

Description

Technical Field

[0001] This invention relates to task-oriented dialogue systems in the field of natural language processing, specifically to low-resource dialogue state tracking, namely a two-stage low-resource dialogue state tracking method based on state value generation. Background Technology

[0002] Currently, artificial intelligence technology is advancing rapidly, and task-oriented dialogue systems, as an important branch of natural language processing, are gaining increasing attention from academia and industry. Task-oriented dialogue systems aim to help users complete a specific task through multiple rounds of natural language interaction. These tasks can be things like booking a hotel or a flight, or complex tasks composed of multiple sub-tasks. A typical task-oriented dialogue system usually consists of four parts: a Natural Language Understanding (NLU) module, a Dialogue State Tracking (DST) module, a Policy Management (PM) module, and a Dialogue Generation (NLG) module. As a core module, the Dialogue State Tracking module is responsible for acquiring and updating the user's dialogue state in multiple rounds of dialogue, and its tracking performance has a significant impact on the success rate of the task-oriented dialogue system. In DST tasks, dialogue state refers to a combination of slot-value pairs. For example, in "hotel-area:centre", "hotel-area" is a slot, and "centre" is the value of that slot. Slot-value pairs can be understood as the user's constraints on their needs.

[0003] With the continuous development of deep learning technology, dialogue state tracking has shifted from traditional rule-based methods to neural network-based methods, which have greatly improved the performance of dialogue state tracking tasks. However, these methods often require a large amount of labeled data. In real-world scenarios, dialogue domains are diverse, making it impractical to annotate dialogues in all domains. Therefore, more and more researchers have turned to researching few-shot dialogue state tracking. These methods mainly classify cross-domain knowledge transfer and cross-task knowledge transfer, achieving good results. However, these methods are constrained by domain and task similarity and still require a large amount of labeled data. In recent years, large-scale pre-trained language models have seen significant development, and generating state values ​​and then their corresponding slots has become a new research direction for low-resource dialogue state tracking. Summary of the Invention

[0004] The purpose of this invention is to provide a two-stage low-resource dialogue state tracking method based on state value generation, and to further improve the performance of state value generation through self-training. This invention employs Prompt Learning to fully utilize the powerful understanding capabilities of pre-trained language models to complete the low-resource dialogue state tracking task, decoupling it into two tasks: state value generation and slot generation. This two-stage model approach decomposes the complex dialogue state tracking task, making it easier to learn the essence of the task. Simultaneously, this invention also employs a self-training method to utilize a large amount of easily accessible unlabeled dialogue data to enhance the state value generation performance, significantly improving the effectiveness of low-resource state tracking.

[0005] The specific technical solution for achieving the objective of this invention is as follows:

[0006] A two-stage low-resource dialogue state tracking method based on state value generation includes the following steps:

[0007] Step 1: State value generation model based on Prompt Learning

[0008] The Prompt Learning approach is used to extract state values ​​from the dialogue text through a pre-trained language model; specifically, given the dialogue history D... ≤t ={S1,U1,S2,U2,...,S t U t The model's goal is to transform the current round of dialogue D. t ={S t U t All state values ​​in} are generated, where S t U t Let represent the dialogue text between the system and the user in the t-th round of dialogue, respectively; use the Transformer model to extract state values; input the Prompt and dialogue history into the pre-trained language model encoder, whose input is represented as:

[0009] X t =Prompt + [History] + D ＜t +[Turn]+D t

[0010] Prompt is a natural language prompt, and [History] and [Turn] represent the start markers of the dialogue history and the current round of dialogue text, respectively.

[0011] Given the input for the current round, obtain the output of the Encoder:

[0012] H t=Encoder(X) t )

[0013] The pre-trained language model decoder then performs cross attention on the output of the pre-trained language model encoder and generates all the state values ​​mentioned in the current round. Cross attention is used here; since the state values ​​are a set, they are concatenated using quotation marks (""), resulting in the final output:

[0014]

[0015] The loss function for model training is the cross-entropy loss function:

[0016] L=-∑log P(V output |D ≤t Prompt)

[0017] Where P(V) output |D ≤t Prompt) means that given D ≤t In the case of Prompt, output the correct set of state values ​​V output The probability of;

[0018] Step Two: Self-Training

[0019] A self-training method is employed to further improve the accuracy of state value extraction in low-resource scenarios; specifically, there are two batches of data, one of which is labeled data D. L Secondly, there is the unlabeled data D. U First in D L Train a model and then use it as a teacher model on a large amount of unlabeled data D. U The pseudo-label data D is obtained by generating state values. P Then, a state-value evaluation model is used to filter noisy data in the pseudo-label data, retaining the data with high confidence. The filtered pseudo-label data D P With D L The student model is then merged to train a new teacher model; this process is repeated until the accuracy of state value generation no longer improves.

[0020] The state value evaluation model is implemented based on Transformer Encoder. Specifically, the output of the state value generation model is first manually constructed into a Prompt template. Different natural language prompts are set for situations where no state value is generated and situations where a state value exists. The dialogue history and the Prompt are connected with the [SEP] label to obtain a part of the input, and an additional [CLS] label is added at the beginning for classification. The final model input is represented as follows:

[0021] E t = [CLS] + [History] + D ＜t +[Turn]+D t +[SEP]+Prompt

[0022] Inputting the above content into the Encoder will yield the corresponding hidden layer output. Take the hidden layer output h corresponding to [CLS]. [cls] To classify, h [CLS] The data is fed into a classifier to evaluate whether the currently generated state value is correct and complete, as shown below:

[0023] P = Softmax(MLP(h[ CLS ]))

[0024] MLP stands for Multilayer Perceptron, which consists of two linear transformation layers and one tanh activation layer; P represents the probability of a three-class classification problem, which is defined as: (1) correct generation; (2) incomplete generation; (3) incorrect generation;

[0025] Its loss function is the cross-entropy loss function:

[0026] L=-∑log P(y|D ≤t Prompt)

[0027] Where P(y|D) ≤t Prompt) means that given D ≤t The probability that the model outputs the correct label y during the Prompt event.

[0028] Step 3: Slot generation model based on Prompt Learning and Inverse Prompt Learning

[0029] After obtaining all state values ​​for the current dialogue round, a corresponding slot is generated for each state value to obtain all slot-value pairs for the current dialogue round. Finally, the state is updated. Slot generation: A prompt-based generative model is used to generate the corresponding slot for each state value. The model input is:

[0030] St =[History]+D ＜t +[Turn]+D t +[SEP]+Prompt

[0031] Prompt provides natural language prompts.

[0032] When training the slot generation model, Inverse Prompt Learning is incorporated, which means replacing the Prompt in the input with Inverse Prompt, i.e., natural language prompts in Inverse Prompt Learning.

[0033] During training, each state value constitutes a pair of samples: one for Prompt Learning and the other for Inverse Prompt Learning; the loss functions are denoted as follows:

[0034] L1=-∑log P(slot|D ≤t Prompt)

[0035] L2=-∑lo g P(value|D ≤t Inverse Prompt

[0036] Where P(slot|D) ≤t Prompt) means that given D ≤t The probability that the model outputs the correct slot under the Prompt condition, P(value|D ≤t Inverse Prompt) means that given D ≤t The probability that the model outputs the correct slot value in the case of Inverse Prompt;

[0037] The final loss function for model training is a combination of these two loss functions:

[0038] L * =L1+w*L2

[0039] Where w is a hyperparameter that controls the impact of Inverse Prompt Learning on model training.

[0040] Compared with the prior art, the present invention has the following advantages:

[0041] 1. Ease of use: Generative models based on pre-trained language models can perform dialogue state tracking tasks well with only a small amount of domain-labeled data.

[0042] 2. Correctness: This method combines Prompt Learning and self-training techniques to generate state values ​​and slots by leveraging the powerful language understanding capabilities of pre-trained language models. It also employs self-training techniques to effectively mitigate the impact of insufficient data on tracking performance by utilizing large-scale unlabeled data.

[0043] 3. Practicality: This method has broad practical significance. It can reduce data annotation. When a new dialogue domain emerges, only a small amount of annotation is needed to complete the dialogue state tracking task well, thereby effectively improving the final performance of the intelligent dialogue system. Attached Figure Description

[0044] Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation

[0045] The present invention will be further described in detail with reference to specific embodiments and accompanying drawings. Except for the contents specifically mentioned below, the processes, conditions, and experimental methods for implementing the present invention are all common knowledge and general knowledge in the art, and the present invention does not have any particular limitations.

[0046] The implementation of this invention consists of three parts, including the following specific steps:

[0047] 1) Use Prompt to leverage the powerful reading comprehension capabilities of large-scale pre-trained language models to extract state values ​​from dialogues;

[0048] 2) Further enhance the state value generation model in step 1 above using self-training techniques;

[0049] 3) Use Prompt to generate the corresponding slot for each extracted state value using a pre-trained language model; where:

[0050] In step 1), for the dialogue text of the current dialogue round, Prompt is used to guide the pre-trained language model to generate state values ​​hidden in the dialogue text.

[0051] In step 2), for state value generation in low-resource scenarios, a self-training approach is used to further improve the performance. Specifically, an initial state value generation model is used to label the remaining data with pseudo-labels. Then, a state value evaluator is used to evaluate the pseudo-labeled data and select samples with high confidence to add to the training set. This new training set is then used to train a new model. Finally, this new model is used to label the remaining data, and this process is repeated. The initial model is trained using a finite training set, and the state value evaluator is also trained using a finite training set. The specific training data is obtained from the finite training set using a negative sampling strategy.

[0052] In step 3), for each generated state value, the system uses Prompt to guide the pre-trained language model to generate its corresponding slot.

[0053] Example

[0054] See Figure 1 Dialogue state tracking is completed in steps one and three. Step two further enhances the state value generator in step one using large-scale unlabeled data. The specific tracking process is as follows: the state value generator in step one generates all state values ​​mentioned in the current round of dialogue. Then, for each generated state value, the "domain-slot" generator in step three generates corresponding "domain-slot" information; thus, the round state is obtained, which consists of all "domain-slot-state values" mentioned in the current round. Updating the dialogue state using the round state completes the dialogue state tracking. During the update process, if a "domain-slot" does not appear in the dialogue state, it is added; otherwise, its corresponding state value is updated.

[0055] This invention mainly includes the following steps:

[0056] Step 1: State value generation model based on Prompt Learning

[0057] The state value of the current round often appears explicitly or implicitly in the dialogue text of the current round. Some difficult-to-generate state values ​​require the model to directly generate the corresponding state values ​​or infer them from the context. Therefore, this invention employs Prompt Learning to extract state values ​​from the dialogue text using the powerful language understanding capabilities of a pre-trained language model. Specifically, given the dialogue history D... ≤t ={S1,U1,S2,U2,...,S t U t The goal of this model is to determine the current round of dialogue D. t ={S t U t All state values ​​in} are generated, where S t U t Let represent the dialogue text between the system and the user in the t-th round of dialogue, respectively. This invention uses the Transformer model for state value extraction. The Prompt and dialogue history are input into the Encoder, which can be represented as:

[0058] X t =Prompt + [History] + D ＜t +[Turn]+D t

[0059] In this embodiment, "Prompt" means "Get the requests that the user confirmed or mentioned in this turn," and "[History]" and "[Turn]" are two markers that respectively indicate the start of the dialogue history and the current round of dialogue text.

[0060] Given the input for the current round, we can obtain the output of the Encoder:

[0061] H t =Encoder(X) t )

[0062] The Decoder model then performs cross-attention with the Encoder output to generate all the state values ​​mentioned in the current round. However, since the state values ​​are a set, the pre-trained language model's Decoder does not support the direct generation of this data structure. Therefore, these state values ​​are connected using "|" (e.g., "centre|expensive"). The final output is:

[0063]

[0064] The loss function for model training is the cross-entropy loss function:

[0065] L=-∑log P(V output |D ≤t Prompt)

[0066] The pre-trained language model used is T5.

[0067] Step Two: Self-Training

[0068] In real-world scenarios, acquiring task-oriented dialogue corpora is relatively easy. Therefore, this invention employs a self-training method to further improve the accuracy of state value extraction in low-resource scenarios. Specifically, there are two batches of data: one is a small amount of labeled data D. L Secondly, there is a large amount of unlabeled data D. U First in D L Train a model on these large-scale unlabeled data D and then use it as a teacher model. U State values ​​are generated, and then a state value evaluation model is used to filter these pseudo-labeled data, retaining the data with higher confidence. The filtered pseudo-labeled data D is then processed. P With D L The models are merged to train a single student model. This student model then becomes the new teacher model, and this process is repeated until the accuracy of state value generation no longer improves.

[0069] The most critical step in the self-training process is filtering out pseudo-labeled data with relatively low confidence, especially in generation tasks, where low-quality pseudo-labeled data can significantly impact model performance. Observation of the output of the state value generator in step one revealed that the model occasionally generates only partial state values, and this problem accounts for the largest proportion of all misclassified cases. Therefore, a state value evaluation model was designed to identify these errors and filter out these low-quality pseudo-labeled data. However, training such an evaluator requires a large number of negative samples, which are obtained by negative sampling from a limited training set. For each training sample, n state values ​​are randomly removed as incomplete generated samples, and a previously generated state value is randomly added as an erroneously generated sample.

[0070] The state value evaluation model is implemented based on Transformer Encoder. Specifically, the output of the state value generation model is first constructed into a Prompt. These Prompt templates are manually created. For cases where no state values ​​are generated, the Prompt is "There are no values ​​mentioned in this turn." For cases where state values ​​exist, the Prompt is "All the values ​​mentioned above are value1, value2,...". The dialogue history is concatenated with the Prompt using "[SEP]" to obtain a portion of the input, with an additional "[CLS]" label added at the beginning for classification. Therefore, the final model input can be represented as:

[0071] E t = [CLS] + [History] + D ＜t +[Turn]+D t +[SEP]+Prompt

[0072] Inputting the above content into the Encoder will yield the corresponding hidden layer output. Take the hidden layer output h corresponding to [CLS]. [cls] To classify, h [CLS] The generated state value can be fed into a classifier to evaluate whether it is correct and complete, as shown below:

[0073] P = Softmax(MLP(h) [CLS] ))

[0074] The MLP (Multilayer Perceptron) consists of two linear transformation layers and one tanh activation layer.

[0075] Its loss function is the cross-entropy loss function:

[0076] L=-∑log P(y|D ≤t Prompt)

[0077] The pre-trained language model used is RoBERTa.

[0078] Step 3: Slot generation model based on Prompt Learning and Inverse Prompt Learning

[0079] After obtaining all state values ​​for the current dialogue round, it is necessary to generate a corresponding slot for each state value to obtain all slot-value pairs for the current dialogue round, and finally update the state. The method for obtaining slots is described below, also using a Prompt-based generative model to generate a corresponding slot for each state value. The input and state value generation model of this model are as follows:

[0080] S t =[History]+D ＜t +[Turn]+D t +[SEP]+Prompt

[0081] The Prompt option is "What is the slot type of [state value]", which can be replaced with the generated state value.

[0082] To help the slot generation model better understand the dialogue state tracking task, Inverse Prompt Learning was added during the training of the slot generation model. This strategy only requires changing the Prompt in the above input to InversePrompt, and then to "What is the value of [slot type]".

[0083] During training, each state value constitutes a pair of samples: one for Prompt Learning and the other for Inverse Prompt Learning. Their loss functions can be denoted as follows:

[0084] L1=-∑log P(slot|D ≤t Prompt)

[0085] L2=-∑log P(value|D ≤t Inverse Prompt

[0086] The final loss function for model training is a combination of the two loss functions mentioned above:

[0087] L * =L1+w*L2

[0088] Where w is a hyperparameter that controls the impact of Inverse Prompt Learning on model training.

[0089] The pre-trained language model used in this model is T5.

[0090] After all steps are completed, the label for the current round is obtained, consisting of multiple slot-value pairs. The overall dialogue state is updated with the label for the current round; the dialogue state also consists of multiple slot-value pairs. During the update process, slots that have not yet appeared are directly inserted. If a slot has already appeared, its corresponding value is updated. For example:

[0091] Dialogue status in round t-1 (before update): {Address: City center, Price: Medium}

[0092] Current round (round t) tags: {Location: Suburbs, Food type: Chinese}

[0093] Dialogue status in round t (after update): {Address: Suburbs, Price: Medium, Food type: Chinese}.

Claims

1. A two-stage low-resource dialogue state tracking method based on state value generation, characterized in that, Includes the following steps: Step 1: State value generation model based on Prompt Learning The Prompt Learning approach is used to extract state values ​​from dialogue text through a pre-trained language model; specifically, given the dialogue history... Its model aims to make the current round of dialogue... All state values ​​are generated, among which , Let represent the dialogue text between the system and the user in the t-th round of the conversation; State values ​​are extracted using a Transformer model; the Prompt and dialogue history are input into a pre-trained language model encoder, whose input representation is as follows: ； Prompt is a natural language prompt. and These represent the start markers for the dialogue history and the current round of dialogue text, respectively. Given the input for the current round, obtain the output of the Encoder: ； Next, the pre-trained language model decoder performs cross-attention on the output of the pre-trained language model encoder and generates all the state values ​​mentioned in the current round. Here, cross-attention is a mechanism where the state values ​​are a set, and they are connected using "|". The final output is: ； The loss function for model training is the cross-entropy loss function: ； in Indicates a given Output the correct set of state values ​​in the following case The probability of; Step Two: Self-Training A self-training method is employed to further improve the accuracy of state value extraction in low-resource scenarios; specifically, there are two batches of data, one of which is labeled data. Secondly, there is unlabeled data. First of all Train a model and then use it as a teacher model on a large amount of unlabeled data. Pseudo-label data is obtained by generating state values. Then, a state-value evaluation model was used to filter noisy data in the pseudo-label data, retaining only the data with high confidence. The filtered pseudo-label data was then... and The student model is then merged to train a new teacher model; this process is repeated until the accuracy of state value generation no longer improves. The state value evaluation model is implemented based on Transformer Encoder. Specifically, the output of the state value generation model is first manually constructed into a Prompt template, and different natural language prompts are set for situations where no state value is generated and situations where state values ​​exist. To integrate dialogue history with by The tag connection obtains a portion of the input, and adds an extra tag at the beginning. Labels are used for classification; the final model input is represented as: ； Input the model The input is fed into the Encoder, which then produces the corresponding hidden layer output. Corresponding hidden layer output To classify, The data is fed into a classifier to evaluate whether the currently generated state value is correct and complete, as shown below: ； in It is a multilayer perceptron, consisting of two linear transformation layers and one... The activation layer is composed of P, which represents the probability of a three-class classification problem. The three-class classification problem is defined as: (1) correct generation; (2) incomplete generation; (3) incorrect generation. Its loss function is the cross-entropy loss function: ； in Indicates that in a given The model outputs the correct label. The probability of; Step 3: Slot generation model based on Prompt Learning and Inverse Prompt Learning After obtaining all state values ​​for the current dialogue round, a corresponding slot is generated for each state value to obtain all slot-value pairs for the current dialogue round. Finally, the state is updated. Slot generation: A prompt-based generative model is used to generate the corresponding slot for each state value. The model input is: ； in Provide natural language prompts; When training the slot generation model, inverse prompt learning is incorporated, that is, the input... Change to This refers to natural language prompts in Inverse Prompt Learning; During training, each state value constitutes a pair of samples: one for Prompt Learning and the other for Inverse Prompt Learning; the loss functions are denoted as follows: ； ； in Indicates a given Under these circumstances, the model outputs the correct slot. The probability, Indicates a given In the case that the model outputs the correct slot value The probability of; The final loss function for model training is a combination of these two loss functions: ； Where w is a hyperparameter that controls the impact of Inverse Prompt Learning on model training.

Images