Spoken dialogue state tracking model training method and spoken dialogue state tracking method
By integrating text correction functionality and semantic vector concatenation into the spoken dialogue state tracking model, the cascading error problem is solved, improving the accuracy and robustness of spoken dialogue state tracking.
Patent Information
- Application Number
- CN202211464999.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-11-22
- Publication Date
- 2026-02-03
- Estimated Expiration
- 2042-11-22
AI Technical Summary
In existing technologies, the independent training of text correction models and spoken dialogue state tracking models leads to cascaded errors, affecting the accuracy of spoken dialogue state tracking.
Text correction functionality is integrated into the spoken dialogue state tracking model, and text correction and slot extraction are performed simultaneously during training. By concatenating the semantic vectors before and after text correction, the model parameters are optimized using the total loss function to reduce cascading errors.
It improves the accuracy of spoken dialogue state tracking, avoids errors caused by independent training of text correction models, and enhances the robustness and accuracy of the model.
Smart Images
Figure CN115795008B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to artificial intelligence technology, and in particular to a method for training a spoken dialogue state tracking model and a spoken dialogue state tracking method. Background Technology
[0002] Spoken dialogue systems aim to extract intent and corresponding slot values from user statements and generate responses by taking appropriate dialogue actions. In this system, a spoken sentence is often first converted into text by an Automatic Speech Recognition (ASR) module, and then the recognized text is input into downstream modules (such as a Dialogue State Tracking (DST) module). In this pipeline structure, ASR may transcribe a certain entity word into another homonym, and the transcribed ASR text contains noise, which seriously affects the performance of downstream modules. To address this problem, a solution is proposed that introduces a text correction model to restore noisy sentences into clean sentences before inputting them into downstream task modules.
[0003] In the process of developing this invention, the inventors discovered that using the above solution for dialogue state tracking results in a cascading error problem, as detailed below:
[0004] In the aforementioned solution incorporating a text correction model, the input data for the dialogue state tracking module is a sequence of sentences corrected by the text correction model. Therefore, the performance of the dialogue state tracking module depends on the accuracy of the text correction model in recovering the sentence sequence. That is, if the text correction model cannot accurately recover the sentence sequence, the corresponding dialogue state tracking module will output semantically incorrect text. Thus, this solution suffers from cascading errors, which in turn affect the accuracy of spoken dialogue state tracking. Summary of the Invention
[0005] In view of this, the main objective of the present invention is to provide a training method for a spoken dialogue state tracking model and a spoken dialogue state tracking method, which can improve the accuracy of spoken dialogue state tracking.
[0006] To achieve the above objectives, the technical solution proposed in this embodiment of the invention is as follows:
[0007] A training method for a spoken dialogue state tracking model includes:
[0008] Obtain preset sample data; the sample data includes correct dialogue process text data, dialogue process text data after noise fusion processing, and target slot value labels;
[0009] Using a spoken dialogue state tracking model, text correction and slot extraction are performed on the dialogue text data after noise fusion processing. Based on the results of text correction and slot extraction, the corrected dialogue text data, and the target slot label, a total loss function value is calculated. The parameters of the spoken dialogue state tracking model are then optimized and adjusted using the total loss function value. Specifically, slot extraction is performed using a dialogue state tracking decoder based on the concatenation results of the semantic vectors corresponding to the dialogue text data before and after text correction.
[0010] This invention also proposes a method for tracking the state of spoken dialogue, including:
[0011] Obtain the target dialogue text data;
[0012] The target dialogue process text data is input into a pre-trained spoken dialogue state tracking model to extract slot values, thereby obtaining the slot values corresponding to the target dialogue process text data.
[0013] The spoken dialogue state tracking model is obtained based on the above training method.
[0014] In summary, the oral dialogue state tracking model training scheme proposed in this invention integrates text correction functionality into the oral dialogue state tracking model. During the training of the oral dialogue state tracking model, text correction and slot extraction are trained simultaneously. The semantic vector after text correction is concatenated with the semantic vector before text correction, and slot extraction is then performed based on this concatenation result. This avoids the cascading error problem caused by the independent training of the text correction model and the oral dialogue state tracking model in existing technologies, thereby effectively improving the accuracy of oral dialogue state tracking. Attached Figure Description
[0015] Figure 1 This is a schematic diagram of the training method for the spoken dialogue state tracking model according to an embodiment of the present invention;
[0016] Figure 2 This is a schematic diagram of the spoken dialogue state tracking model framework according to an embodiment of the present invention;
[0017] Figure 3 This is a schematic diagram of the text correction and slot extraction method according to an embodiment of the present invention.
[0018] Figure 4 This is a schematic diagram of the spoken dialogue state tracking method according to an embodiment of the present invention. Detailed Implementation
[0019] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.
[0020] Figure 1 This is a schematic diagram of the training method of the spoken dialogue state tracking model according to an embodiment of the present invention, as shown below. Figure 1 As shown, this embodiment mainly includes:
[0021] Step 101: Obtain preset sample data; the sample data includes correct dialogue process text data, dialogue process text data after noise fusion processing, and target slot value labels.
[0022] Here, in order to improve the robustness of the spoken dialogue state tracking model during training, the sample data needs to include dialogue text data that has been noise-fused.
[0023] The dialogue text data in the sample data, i.e., the context data of the dialogue, is obtained by concatenating all preceding user statements and system statements, and can be represented as x = {x0, x1, ..., x...}. n}, where n represents the number of words contained in the dialogue text data. Correspondingly, the dialogue text data after noise fusion processing can be represented as: Wherein, n′ represents the number of words contained in the dialogue process text data after noise fusion processing, in which only the user's statements contain noise.
[0024] In one implementation, to better simulate spoken transcription noise and improve the model's noise resistance, the following method can be used to perform noise fusion processing on the dialogue process text data:
[0025] Step a1: Convert the text data of the dialogue process into voice data.
[0026] Step a2: Fuse the voice data with preset noise audio.
[0027] Step a3: Convert the fusion result into text data to obtain the dialogue process text data after noise fusion processing.
[0028] In one implementation, to better increase the robustness of the model, sample data can be constructed based on a large-scale MultiWOZ dialogue state tracking dataset.
[0029] Considering that the spoken dialogue state tracking model in this invention is a multi-task model, that is, it integrates the text correction task and the dialogue state tracking task (i.e., the task of extracting slot values), in practical applications, for ease of implementation, for a correct dialogue process text data, two sample data can be generated accordingly, which can be used for training the text correction task and the dialogue state tracking task respectively. Accordingly, these two sample data also contain task type information to indicate which task the model performs based on the sample data. The two sample data are identical except for the task type.
[0030] Step 102: Using the spoken dialogue state tracking model, based on the dialogue process text data after noise fusion processing, perform text correction and slot extraction. Based on the results of text correction and slot extraction, the correct dialogue process text data, and the target slot label, calculate the total loss function value. Use the total loss function value to optimize and adjust the parameters of the spoken dialogue state tracking model. Specifically, based on the concatenation results of the semantic vectors corresponding to the dialogue process text data before and after text correction, the slot extraction is performed using the dialogue state tracking decoder.
[0031] It should be noted that this invention integrates text correction functionality into the spoken dialogue state tracking model. When training the spoken dialogue state tracking model, text correction and slot extraction are trained simultaneously. The semantic vector after text correction is concatenated with the semantic vector before text correction, and slot extraction is then performed based on the concatenation result. This avoids the cascading error problem caused by the independent training of the text correction model and the spoken dialogue state tracking model in existing technologies, thereby effectively improving the accuracy of spoken dialogue state tracking.
[0032] Figure 2 This is a schematic diagram of the spoken dialogue state tracking model framework used in an embodiment of the present invention. In one embodiment, as... Figure 2 and Figure 3 As shown, the following methods can be used for text correction and slot extraction:
[0033] Step 301: Generate a vector representation of the dialogue process text data.
[0034] This step is used to generate corresponding vector representations based on the dialogue process text data that has undergone noise fusion processing. Existing methods can be used to achieve this, which will not be elaborated here.
[0035] Step 302: Based on the vector representation, use an error correction encoder and an error correction decoder to perform text error correction processing to obtain the first semantic vector of the dialogue process text data.
[0036] In one implementation, text error correction processing can be performed based on the vector representation using an error correction encoder and an error correction decoder:
[0037] The vector representation is input to the error correction encoder for encoding, and the corresponding encoding result is input to the error correction decoder for decoding to obtain the first semantic vector.
[0038] In the above method, the error correction encoder is first used to perform multi-head attention calculation on the vector representation generated in step 201, and transform it into a latent vector sequence H′ (semantic vector). The error correction encoder contains N layers of fully connected feedforward network, and the specific implementation is the same as the existing technology, which will not be described in detail here.
[0039] The final output of the error correction encoder is input to the error correction decoder for decoding to obtain the semantic vector of the text after error correction, i.e., the first semantic vector. The specific implementation of the error correction decoder is the same as existing technology and will not be described in detail here.
[0040] Step 303: Based on the vector representation, use the dialogue state tracking encoder to perform encoding processing to obtain the second semantic vector of the dialogue process text data.
[0041] This step is used to utilize the dialogue state tracking encoder to encode the vector representation corresponding to the dialogue process text data after noise fusion processing obtained in step 201, and obtain the semantic vector of the dialogue process text data before text correction after noise fusion processing, namely the second semantic vector.
[0042] Step 304: Concatenate the first semantic vector and the second semantic vector, and use a self-attention network layer to perform semantic fusion on the concatenation result.
[0043] Here, in order to improve the accuracy of subsequent slot value extraction, a self-attention network layer is used to fuse the concatenation of semantic vectors before and after text correction.
[0044] Step 305: Input the semantic fusion result into the dialogue state tracking decoder for decoding processing to obtain the slot value extraction result of the dialogue process text data.
[0045] Considering that text correction processing may generate incorrect correction results, which may affect the accuracy of dialogue state tracking, in order to reduce the occurrence of this problem, in this step, based on the concatenation and fusion result of semantic vectors before and after text correction, slot value extraction is performed using a dialogue state tracking decoder. Since the semantic fusion result incorporates information before and after text correction, the aforementioned cascading error problem caused by text correction can be effectively reduced.
[0046] In one implementation, to improve the accuracy of the loss function value, preferably, in step 102, the following method can be used to calculate the total loss function value based on the results of the text correction and the slot extraction, the correct dialogue process text data, and the target slot label:
[0047] Step x1, according to Calculate the value of the first loss function
[0048] Where α is a preset hyperparameter, α≤1, The text data represents the dialogue process after noise fusion processing; x represents the correct dialogue process text data, x = {x0, x1, ..., x...} n}, where n represents the number of words in x; x i This represents the i-th word in x. x represents the output probability of the error correction decoder. ＜i Indicates that in relation to x i All words that have been decoded before the decoding process begins. x represents i Not a entity word. x represents i It is a entity word.
[0049] It should be noted here that, in order to improve the error correction capability of the error correction encoder and decoder for entity words, a hyperparameter α is introduced to distinguish between entity words and non-entity words, thereby controlling the loss of entity words and the loss of non-entity words. That is, when x i When it is an entity word, the corresponding loss value is based on We get that when x i When it is not an entity word, the corresponding loss value is calculated according to... get.
[0050] Step x2, according to Calculate the value of the second loss function Where y represents the target slot value label; This represents the output probability of the dialogue state tracking decoder.
[0051] Step x3: Calculate the sum of the first loss function value and the second loss function value to obtain the total loss function value.
[0052] Since the spoken dialogue state tracking model integrates text correction and dialogue state tracking tasks, the total loss function value in this step is obtained by calculating the sum of the loss function values corresponding to the two tasks, and the model parameters are optimized using the total loss function value.
[0053] In one embodiment, the error correction encoder and the dialogue state tracking encoder can be initialized using the parameters of a pre-trained first encoder; the error correction decoder and the dialogue state tracking decoder can be initialized using the parameters of a pre-trained first decoder.
[0054] To address the issue of large variance in error correction task loss during the initial pre-training phase, a preferred approach is to introduce a course-based learning method. This method utilizes sentence perplexity levels to train the first encoder and the first decoder, specifically implemented using the following steps y1-y4:
[0055] Step y1: Obtain a preset pre-training sample data set, which includes an error correction task sample data set and a dialogue state tracking task sample data set; each sample data in the pre-training sample data set includes task type indication information, correct dialogue process text data and corresponding dialogue process text data after noise fusion processing, as well as target slot value labels; the task types include error correction tasks and dialogue state tracking tasks.
[0056] Step y2: Determine the sentence perplexity of each sample data in the error correction task sample data set; based on the sentence perplexity and the preset sentence perplexity level classification strategy, determine the sentence perplexity level of the corresponding sample data; classify the sample data in the error correction task sample data set according to the sentence perplexity level to obtain the error correction task sample data subset corresponding to each sentence perplexity level.
[0057] This step is used to divide the sample data in the error correction task sample dataset into subsets corresponding to different sentence perplexity levels, so that in the subsequent step y3, each subset can be combined with the dialogue state tracking task sample dataset to obtain the pre-trained sample dataset sets corresponding to different sentence perplexity levels.
[0058] The sentence perplexity of each sample can be determined using existing methods, which will not be elaborated here.
[0059] The sentence perplexity level classification strategy can be set by those skilled in the art according to actual application needs. For example, the sentence perplexity level can be divided into three levels: easy, medium, and difficult, with each level corresponding to a different range of sentence perplexity, but it is not limited to this.
[0060] Step y3: Combine the dialogue state tracking task sample data set with each error correction task sample data subset to obtain the pre-training sample data set group corresponding to the sentence perplexity level.
[0061] Step y4: Select one of the pre-training sample data sets in order of sentence perplexity level from low to high, and use the sample data in the pre-training sample data set to train the codec training model.
[0062] The training includes: selecting sample data in batches from the pre-trained sample data set, and using each batch of selected sample data to train the codec training model for the corresponding task; in each batch of sample data, error correction task and dialogue state tracking task each account for half of the sample data; the codec training model consists of the first encoder and the first decoder.
[0063] It should be noted that, as mentioned above, since the spoken dialogue state tracking model in this embodiment of the invention integrates text correction task and dialogue state tracking task, there are two types of sample data used for pre-training: one type for error correction task and the other type for dialogue state tracking task.
[0064] In practical applications, the first encoder and the first decoder mentioned above can be trained using the T5 model.
[0065] As can be seen from the above technical solutions, the above method embodiments only use the text data of the dialogue process after speech recognition, without introducing additional feature information, to build a highly accurate and robust spoken language understanding model, which has good scalability.
[0066] Based on the above-described training method embodiment for the spoken dialogue state tracking model, correspondingly, this invention provides a spoken dialogue state tracking method, such as... Figure 4 As shown, the method includes the following steps:
[0067] Step 401: Obtain the target dialogue process text data.
[0068] Step 402: Input the target dialogue process text data into a pre-trained spoken dialogue state tracking model to extract slot values and obtain the slot values corresponding to the target dialogue process text data.
[0069] The spoken dialogue state tracking model is obtained based on the above-described training method embodiment of the spoken dialogue state tracking model.
[0070] In the above-described oral dialogue state tracking method embodiment, since the model used is obtained by training the oral dialogue state tracking model of the present invention, the above-described oral dialogue state tracking method can improve the accuracy and reliability of the slot extraction results.
[0071] Each embodiment of the present invention can be implemented by a data processing program executed by a data processing device such as a computer. Clearly, the data processing program constitutes the present invention. Furthermore, a data processing program typically stored in a storage medium is executed by directly reading the program from the storage medium or by installing or copying the program to the storage device (such as a hard disk and / or memory) of the data processing device. Therefore, such a storage medium also constitutes the present invention. The storage medium can use any type of recording method, such as paper storage media (e.g., paper tape), magnetic storage media (e.g., floppy disks, hard disks, flash memory), optical storage media (e.g., CD-ROMs), magneto-optical storage media (e.g., MO), etc.
[0072] Therefore, the present invention also discloses a storage medium storing a data processing program for performing any of the above-described embodiments of the methods of the present invention.
[0073] Furthermore, the steps described in this invention can be implemented not only by a data processing program but also by hardware, such as logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Therefore, such hardware capable of implementing the methods described in this invention can also constitute this invention.
[0074] In summary, the above are merely preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.
Claims
1. A training method for a spoken dialogue state tracking model, characterized in that, include: Obtain the preset sample data; The sample data includes correct dialogue text data, dialogue text data processed by noise fusion, and target slot value labels; Using a spoken dialogue state tracking model, text correction and slot extraction are performed on the dialogue text data after noise fusion processing. Based on the results of text correction and slot extraction, the corrected dialogue text data, and the target slot label, a total loss function value is calculated. The parameters of the spoken dialogue state tracking model are then optimized and adjusted using the total loss function value. Specifically, slot extraction is performed using a dialogue state tracking decoder based on the concatenation results of the semantic vectors corresponding to the dialogue text data before and after text correction. The calculation of the total loss function value based on the results of text correction and slot extraction, the correct dialogue process text data, and the target slot label includes: according to Calculate the first loss function value ;in, These are preset hyperparameters. , This represents the text data of the dialogue process after noise fusion processing; This represents the correct dialogue process text data. n represents The number of words in the text; express The i-th word in This represents the output probability of the error correction decoder. Indicating in the All words that have been decoded before the decoding process begins. express Not a entity word. express It is a entity word; according to Calculate the value of the second loss function. ;in, Indicates the target slot value label; This represents the output probability of the dialogue state tracking decoder; The sum of the first loss function value and the second loss function value is calculated to obtain the total loss function value; The method further includes: initializing the error correction encoder and the dialogue state tracking encoder using a pre-trained first encoder; initializing the error correction decoder and the dialogue state tracking decoder using a pre-trained first decoder; wherein the training method for the first encoder and the first decoder includes: Obtain a preset pre-training sample data set, which includes an error correction task sample data set and a dialogue state tracking task sample data set; each sample data in the pre-training sample data set includes task type indication information, correct dialogue process text data and corresponding dialogue process text data after noise fusion processing, as well as target slot value labels; the task types include error correction tasks and dialogue state tracking tasks. The sentence perplexity of each sample data in the error correction task sample data set is determined. Based on the sentence perplexity and a preset sentence perplexity level classification strategy, the sentence perplexity level of the corresponding sample data is determined. According to the sentence perplexity level, the sample data in the error correction task sample data set is classified to obtain a subset of error correction task sample data corresponding to each sentence perplexity level. The dialogue state tracking task sample data set is combined with each error correction task sample data subset to obtain a pre-training sample data set group corresponding to the sentence perplexity level. According to the sentence perplexity level from low to high, a pre-training sample data set is selected sequentially, and the codec training model is trained using the sample data in the pre-training sample data set; wherein, the training includes: selecting sample data in batches from the pre-training sample data set, and using each batch of selected sample data to train the codec training model for the corresponding task; in each batch of sample data, the error correction task and the dialogue state tracking task each account for half of the sample data; the codec training model consists of the first encoder and the first decoder.
2. The method according to claim 1, characterized in that, The text correction and slot extraction process includes: Generate a vector representation of the text data from the dialogue process; Based on the vector representation, text error correction processing is performed using an error correction encoder and an error correction decoder to obtain the first semantic vector of the dialogue process text data; Based on the vector representation, the second semantic vector of the dialogue process text data is obtained by encoding the dialogue state tracking encoder. The first semantic vector and the second semantic vector are concatenated, and the semantic fusion of the concatenation result is performed using a self-attention network layer. The semantic fusion result is input into the dialogue state tracking decoder for decoding processing to obtain the slot value extraction result of the dialogue process text data.
3. The method according to claim 2, characterized in that, The text error correction processing based on the vector representation, using an error correction encoder and an error correction decoder, includes: The vector representation is input to the error correction encoder for encoding, and the corresponding encoding result is input to the error correction decoder for decoding to obtain the first semantic vector.
4. The method according to claim 1, characterized in that, The noise fusion process includes: Convert the text data of the dialogue process into voice data; The voice data is fused with a preset noise audio. The fusion result is converted into text data to obtain the dialogue process text data after noise fusion processing.
5. A method for tracking the state of spoken dialogue, characterized in that, include: Obtain the target dialogue text data; The target dialogue process text data is input into a pre-trained spoken dialogue state tracking model to extract slot values, thereby obtaining the slot values corresponding to the target dialogue process text data. The spoken dialogue state tracking model is obtained based on any of the training methods in claims 1 to 4.