Dialogue state tracking method and system based on knowledge enhancement and context awareness

Abstract

The present invention relates to a method and system for tracking a conversation state based on knowledge enhancement and context perception, the method comprising the following steps: step A: collecting conversation context and conversation state data to construct a conversation training set; step B: using the training set to train a deep learning network model based on knowledge enhancement; step C: inputting the conversation data between the user and the system into the deep learning network model in sequence, and outputting the current conversation state. The method and system are conducive to improving the accuracy of conversation state tracking.

Description

Technical Field

[0001] The present invention belongs to the field of natural language processing, and specifically relates to a conversation state tracking method and system based on knowledge enhancement and context perception. Background Art

[0002] Dialogue State Track (DST) is an important part of the task-based dialogue task flow in natural language processing. Its basic goal is to obtain the current dialogue state based on the dialogue context. The dialogue state is a summary of the user's goal from the beginning of the dialogue to the current dialogue, usually in the form of a combination of multiple slot-value pairs, and sometimes also includes information such as the field to which the dialogue belongs and the user's intention. Dialogue state tracking refers to the process of inferring and updating the current dialogue state by combining information such as the dialogue history, the current dialogue, and the previous round of dialogue state. With the continuous advancement of information technology, people can access information, applications, and services almost instantly at any time and place with wireless connections. Nowadays, such as smartphones and tablets have been widely used to access the Internet. However, content acquisition is usually limited to browsers, which rely on traditional graphical user interfaces (GUIs). More advanced human-computer interaction methods are urgently needed, such as having a more intelligent, intuitive and lightweight interface that can communicate in human language and provide a transparent and human-like environment. In film and television works, it is usually presented in the form of intelligent robots. However, current technology makes it difficult to achieve this kind of true human intelligence. Therefore, dialogue systems that can communicate with humans in natural language are favored by researchers.

[0003] Before the rise of deep learning, the DST modules of early dialogue systems all adopted a rule-based approach, which did not require training data and directly used artificial rules to update the dialogue state. However, when the amount of data increases, this method requires a lot of manpower to design rule templates, and these manually formulated rules are difficult to guarantee their quality, and they are not flexible and scalable when the data changes. These shortcomings limit the rule-based dialogue state tracking model.

[0004] In recent years, deep learning methods have been widely used in many fields of natural language processing. Deep learning is used to track dialogue states, without the need for manual heavy rule design work, and can automatically extract semantic feature information from the dialogue context. The proposal and development of attention mechanisms and pre-training models have attracted widespread attention from industry and academia. Many pre-training models such as BERT and GPT have shown extraordinary results when applied to downstream tasks. In view of this, the research on DST has shifted to building new models based on these powerful pre-training models. For example, SUMBT proposed by Lee et al. uses the slot and dialogue utterance representation encoded by BERT to learn the relationship between slots and dialogue utterances through the attention mechanism. CHAN is built on the basis of SUMBT. In order to better utilize the dialogue context, a hierarchical attention mechanism of slot-word attention and slot-turn attention is proposed. In order to better simulate the dialogue behavior in the pre-training process, TOD-BERT further pre-trains the original BERT model using multiple task-oriented dialogue datasets. SOM-DST treats the dialogue state as an explicit fixed-size memory, which is selectively overwritten by the classifier to avoid predicting the dialogue state from scratch. TripPy uses three replication mechanisms to extract slot values. MinTL uses T5 and BART as dialogue discourse encoders to jointly learn dialogue states and system responses. NP-DST and SimpleTOD use GPT-2 as dialogue context encoders and formulate DST as a language generation task.

[0005] However, the above solutions only model the relationship between slots and conversation contexts while ignoring the correlation between slots, which is very important due to the co-citation of slot values ​​in multi-domain conversation data. Studies focusing on slot correlation, such as CSFN-DST and SST, construct a pattern graph to capture the dependencies between different slots. However, these relationships need to rely on accurate prior knowledge. Although SAS models the relationship between slots by calculating a slot similarity matrix, it involves several hyperparameters that are difficult to set, and if two slots are considered to be related, their similarity coefficients are simply fixed to 1, which is obviously not realistic. Although STAR uses multi-head attention to discover the relationship between slots, the relationship between slots calculated by the attention mechanism is equivalent to a weighted complete graph, which will generate a lot of noise (unnecessary connections) that affects the model performance. T5-DST uses enhanced slot descriptions to enable the model to better discover the relationship between slots, but it does not explicitly model the relationship between slots. Summary of the invention

[0006] The purpose of the present invention is to provide a method and system for tracking dialog states based on knowledge enhancement and context awareness, which method and system are conducive to improving the accuracy of dialog state tracking.

[0007] To achieve the above object, the technical solution adopted by the present invention is: a conversation state tracking method based on knowledge enhancement and context awareness, comprising the following steps:

[0008] Step A: Collect dialogue context and dialogue state data to build a dialogue training set TS;

[0009] Step B: Use the training set TS to train the knowledge-enhanced deep learning network model G;

[0010] Step C: Input the conversation data between the user and the system into the deep learning network model G in sequence and output the current conversation status.

[0011] Furthermore, the step B specifically includes the following steps:

[0012] Step B1: Encode the tth round of dialogue of a dialogue sample in the training set TS to obtain the initial representation vector The concatenation of the current dialogue, dialogue history and dialogue status of the t-th dialogue round;

[0013] Step B2: Initial representation vector of the dialogue state Input into the multi-head attention mechanism module to obtain the attention-enhanced dialogue state semantic representation vector

[0014] Step B3: Add the attention-enhanced dialogue state semantic representation vector obtained in step B2 and the initial representation vector of the conversation history Perform cross-attention gating fusion to obtain the fusion representation vector of dialogue history and dialogue state

[0015] Step B4: Initial representation vector of the current dialogue The fusion representation vector obtained in step B3 Perform word-level concatenation to obtain the conversation context representation vector

[0016] Step B5: Encode the enhanced domain-slot description through BERT to obtain the initial representation vector The enhanced domain-slot description is input into the knowledge extraction module to obtain the corresponding knowledge embedding representation vector Then the obtained initial characterization vector and knowledge embedding representation vector Fusion is performed to obtain the domain-slot description representation vector after knowledge enhancement

[0017] Step B6: Transform the conversation context representation vector obtained in step B4 into The characterization vector obtained in step B5 Perform attention modeling to obtain a knowledge-enhanced domain-slot representation vector containing contextual information

[0018] Step B7: All domain-slot representation vectors obtained in step B6 After multi-head attention modeling, the final domain-slot representation vector Υ is obtained S ;

[0019] Step B8: The domain-slot representation vector obtained in step B7 The candidate slot value of the corresponding domain-slot pair is represented by the encoded vector V j '∈V j Perform similarity matching and select the most similar slot value as the prediction result; compare the predicted value with the true value to calculate the loss, use the back propagation algorithm to calculate the gradient of each parameter in the deep network, and use the stochastic gradient descent algorithm to update the parameters;

[0020] Step B9: When the loss value generated by the deep learning network model is less than the set threshold or reaches the maximum number of iterations, the training of the deep learning model G is terminated.

[0021] Furthermore, the step B1 specifically includes the following steps:

[0022] Step B11: The dialogue training set is represented as Where N represents the number of training samples, that is, the number of multi-round dialogue samples; n represents the number of rounds of each multi-round dialogue, (D t ,B t-1 ,H t ) represents a round of training samples in the dialogue training set; Indicates the current round of dialogue. Indicates the conversation history except the current round of conversation, B t-1 represents the current dialogue state and is generated by the model during the prediction process; therefore, the input of the model is [cls] and [sep] are special tokens used to separate the parts of the input;

[0023] Step B12: X the result of step B12 t After BERT encoding, the corresponding three parts are split into initial representations: Where L is the length of the current conversation, L h is the length of the current dialogue history, J is the length of the current dialogue state, and d is the dimension of the token representation vector.

[0024] Furthermore, the step B2 specifically includes the following steps:

[0025] Step B21: The historical conversation status portion output from step B12 After multi-head attention modeling:

[0026]

[0027]

[0028]

[0029]

[0030]

[0031] in, are learnable weight parameters, SelfAttention(Q,K,V) and MultiHeadAttention(Q,K,V) are self-attention and multi-head attention mechanisms respectively, and the Concat(·) function concatenates the outputs of h self-attention heads;

[0032] Step B22: Splice the outputs of multiple heads in step B21 Mapped to the original space, we get the attention-enhanced dialogue state semantic representation vector

[0033]

[0034] in, is the learnable parameter matrix.

[0035] Furthermore, the step B3 specifically includes the following steps:

[0036] Step B31: Calculate the interaction attention matrix:

[0037]

[0038]

[0039]

[0040] in, is the learnable weight parameter matrix corresponding to the dialogue history and dialogue state, A T represents the matrix transpose, a b→h ,a h→b They are the interactive attention matrices of the dialogue state to the dialogue history and the dialogue history to the dialogue state respectively;

[0041] Step B32: The interactive attention matrix a obtained in step B31 h→b,a b→h , compute the interaction context representation:

[0042]

[0043]

[0044] Step B33: Calculate the two context representations obtained in step B32 The fusion weight of , and the two are fused according to the fusion weight:

[0045] L'=max(J,L h )

[0046]

[0047]

[0048]

[0049] Among them, the max(·) function obtains the maximum value L' of the input length; Padding(x,y,z) aligns the two dimensions to the larger one, and fills z=0 if it is insufficient; is the learnable parameter matrix; σ(·) is the activation function, ⊙ represents the matrix dot product, and finally the fused context representation vector is obtained

[0050] Furthermore, in step B4, the fused context representation vector obtained in step B33 is The initial representation of the current dialogue round obtained in step B12 Concatenate to get the summarized context information

[0051]

[0052] The | operation indicates concatenation at the token sequence level, and the length change to L'+J indicates concatenation in the length direction.

[0053] Furthermore, the step B5 specifically includes the following steps:

[0054] Step B51: Encode the enhanced domain-slot description through BERT to obtain the initial representation vector

[0055]

[0056] Among them, BERT fixed Indicates that the BERT does not participate in training. represents the length of the embedded sequence described by the enhanced slot, Indicates enhanced slot description;

[0057] Step B52: The enhanced domain-slot description is passed through the knowledge extraction module to obtain the domain-slot knowledge representation

[0058]

[0059] Among them, KE represents the function of obtaining ConceptNet knowledge embedding, represents the length of the embedded sequence described by the enhanced slot;

[0060] Step B53: Obtained from step B51 Knowledge embedding obtained with B52 After splicing, we get:

[0061]

[0062]

[0063] Among them, d k represents the dimension of ConceptNet knowledge embedding, “;” represents the concatenation operation, U j Indicates slot The representation after concatenating knowledge embedding;

[0064] Step B54: Use the self-attention mechanism to further calculate the fusion information of the representation obtained in step B53:

[0065]

[0066]

[0067] in, express The first token representation vector of represents the set of all enhanced slot representations;

[0068] Step B55: In order not to lose the original slot information, merge the result of step B54 with the [CLS] position output of step B51 After splicing, we get:

[0069]

[0070] Step B56: All slot representations obtained in step B55 are passed through a fully connected layer and a LayerNorm layer to obtain the final slot representation

[0071]

[0072] in, is the learnable parameter matrix.

[0073] Furthermore, in step B6, the enhanced domain slot representation and the context are subjected to multi-head cross attention modeling:

[0074]

[0075] in, Contains information that enhances domain slot representation and context;

[0076] The step B7 specifically includes the following steps:

[0077] Step B71: Input the obtained domain slot representation into L att Layer multi-head attention layer, let For the lth self-attention layer:

[0078]

[0079]

[0080]

[0081]

[0082] Among them, W1 F is the learnable parameter matrix, is the bias term, is the output of the last layer;

[0083] Step B72: Input the output of B71 into a linear layer and a LayerNorm layer to obtain the final domain slot representation Y that combines context information and knowledge information S :

[0084]

[0085]

[0086] in, represents the representation of the jth slot, Υ S Represents the set of all slot representations.

[0087] Furthermore, the step B8 specifically includes the following steps:

[0088] Step B81: For each slot, first encode the candidate value through BERT to obtain the value feature

[0089]

[0090] in Indicates the i-th candidate value of the j-th slot, and finally takes The [cls] bit represents the final value Encode each candidate value to get the candidate value set Since the number of candidate values ​​for each slot is different, the value range of i is different;

[0091] Step 82: Combine all candidate value representations obtained in B81 with the slot representations obtained in B72 Calculate the semantic distance and then select the slot value with the minimum distance as slot S j The final prediction result; here we use L2 norm as the distance metric; in the training phase, we calculate the slot S in round t j The true value of The probability is:

[0092]

[0093] The value with the highest probability is taken as the predicted value; where exp(·) represents the exponential function and ||·||2 represents the L2 norm;

[0094] Step B83: The model is trained to maximize the joint probability of all slots, i.e. The loss function for each round t is defined as the accumulation of negative log-likelihoods:

[0095]

[0096] Step B84: Update the learning rate of the loss calculated in B83 through the gradient optimization algorithm AdamW, and use back propagation to iteratively update the model parameters to minimize the loss function to train the model.

[0097] The present invention also provides a dialogue state tracking system using the above method, comprising:

[0098] Build a training set module to collect conversation context and answer data and build a conversation training set TS;

[0099] A model training module, used for training a deep learning network model G based on knowledge enhancement; and

[0100] The dialogue state tracking module is used to input the user speech and system speech into the trained deep network model to output the current dialogue state during the dialogue with the user.

[0101] Compared with the prior art, the present invention has the following beneficial effects: First, the present invention uses knowledge entity embedding of common sense graph to enhance slot representation, so that slot representation has richer semantic information and the model can better discover the semantic relationship between slots. In addition, the present invention takes into account that the conversation history and the previous round of conversation state in the context should have different levels of importance, and allows the model to learn a gating module to control the information fusion of the two, improve the accuracy of context modeling to reduce data sparsity and past problems caused by lengthy context, thereby improving the accuracy of the model in tracking the conversation state. BRIEF DESCRIPTION OF THE DRAWINGS

[0102] Figure 1 is a flow chart of a method implementation of an embodiment of the present invention;

[0103] Figure 2 This is a model architecture diagram in an embodiment of the present invention;

[0104] Figure 3 Schematic diagram of the system structure of an embodiment of the present invention. DETAILED DESCRIPTION

[0105] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0106] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present application belongs.

[0107] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0108] like Figure 1 As shown, this embodiment provides a method for tracking a dialog state based on knowledge enhancement and context awareness, comprising the following steps:

[0109] Step A: Collect dialogue context and dialogue state data to build a dialogue training set TS.

[0110] Step B: Use the training set TS to train the knowledge-enhanced deep learning network model G. The architecture of the deep learning network model G in this embodiment is as follows: Figure 2 shown.

[0111] Step C: Input the conversation data between the user and the system into the deep learning network model G in sequence and output the current conversation status.

[0112] In this embodiment, step B specifically includes the following steps:

[0113] Step B1: Encode the tth round of dialogue of a dialogue sample in the training set TS to obtain the initial representation vector The concatenation of the current dialogue, dialogue history and dialogue status of the t-th round of dialogue. The step B1 specifically includes the following steps:

[0114] Step B11: The dialogue training set is represented as Where N represents the number of training samples, that is, the number of multi-round dialogue samples. n represents the number of rounds of each multi-round dialogue, (D t ,B t-1 ,H t ) represents a round of training samples in the dialogue training set. Indicates the current round of dialogue. Indicates the conversation history except the current round of conversation, Represents the current conversation state and is generated by the model during the prediction process. represents the domain-slot pair S in round t-1 j The initial value of each domain-slot pair is none. j With V j spliced, and there are J in total. Here it means that these J S j With V j Put together, it can also be expressed as

[0115] Therefore, the input to the model is [cls] and [sep] are special tokens used to separate the parts of the input.

[0116] Step B12: X the result of step B12 t After BERT encoding, the corresponding three parts are split into initial representations: in, is a set of real numbers in mathematics, L is the length of the current conversation, L h is the length of the current dialogue history, J is the length of the current dialogue state, and d is the dimension of the token representation vector.

[0117] Step B2: Initial representation vector of the dialogue state Input into the multi-head attention mechanism module to obtain the attention-enhanced dialogue state semantic representation vector The step B2 specifically comprises the following steps:

[0118] Step B21: The historical conversation status portion output from step B12 After multi-head attention modeling:

[0119]

[0120]

[0121]

[0122]

[0123]

[0124] in, are learnable weight parameters, SelfAttention(Q,K,V) and MultiHeadAttention(Q,K,V) are self-attention and multi-head attention mechanisms respectively, and the Concat(·) function concatenates the outputs of h self-attention heads.

[0125] Step B22: Splice the outputs of multiple heads in step B21 Mapped to the original space, we get the attention-enhanced dialogue state semantic representation vector

[0126]

[0127] in, is the learnable parameter matrix.

[0128] Step B3: Add the attention-enhanced dialogue state semantic representation vector obtained in step B2 and the initial representation vector of the conversation history Perform cross-attention gating fusion to obtain the fusion representation vector of dialogue history and dialogue state The step B3 specifically comprises the following steps:

[0129] Step B31: Calculate the interaction attention matrix:

[0130]

[0131]

[0132]

[0133] in, is the learnable weight parameter matrix corresponding to the dialogue history and dialogue state, A T represents the matrix transpose, ab→h ,a h→b They are the interactive attention matrices of the dialogue state for the dialogue history and the dialogue history for the dialogue state, respectively.

[0134] Step B32: The interactive attention matrix a obtained in step B31 h→b ,a b→h , compute the interaction context representation:

[0135]

[0136]

[0137] Step B33: Calculate the two context representations obtained in step B32 The fusion weight of , and the two are fused according to the fusion weight:

[0138] L'=max(J,L h )

[0139]

[0140]

[0141]

[0142] The max(·) function obtains the maximum value of the input length L'. Padding(x,y,z) aligns the two dimensions to the larger one, and fills in z=0 if it is insufficient. is a learnable parameter matrix. σ(·) is the activation function, ⊙ represents the matrix dot product, and finally the fused context representation vector is obtained

[0143] Step B4: Initial representation vector of the current dialogue The fusion representation vector obtained in step B3 Perform word-level concatenation to obtain the conversation context representation vector

[0144] Specifically, the fused context representation vector obtained in step B33 is The initial representation of the current dialogue round obtained in step B12 Concatenate to get the summarized context information

[0145]

[0146] The | operation indicates concatenation at the token sequence level, and the length change to L'+J indicates concatenation in the length direction.

[0147] Step B5: Encode the enhanced domain-slot description through BERT to obtain the initial representation vector The enhanced domain-slot description is input into the knowledge extraction module to obtain the corresponding knowledge embedding representation vector Then the obtained initial characterization vector and knowledge embedding representation vector Fusion is performed to obtain the domain-slot description representation vector after knowledge enhancement The step B5 specifically comprises the following steps:

[0148] Step B51: Encode the enhanced domain-slot description through BERT to obtain the initial representation vector

[0149]

[0150] Among them, BERT fixed Indicates that the BERT does not participate in training. represents the length of the embedded sequence described by the enhanced slot, Indicates enhanced slot description. For example, the domain-slot hotel-name will be enhanced to name of the hotel.

[0151] Step B52: The enhanced domain-slot description is passed through the knowledge extraction module to obtain the domain-slot knowledge representation

[0152]

[0153] Among them, KE represents the function of obtaining ConceptNet knowledge embedding, Indicates the length of the embedded sequence described by the enhanced slot.

[0154] Step B53: Obtained from step B51 Knowledge embedding obtained with B52 After splicing, we get:

[0155]

[0156]

[0157] Among them, d k represents the dimension of ConceptNet knowledge embedding, “;” represents the concatenation operation, U j Indicates slot The representation after concatenating knowledge embedding.

[0158] Step B54: Use the self-attention mechanism to further calculate the fusion information of the representation obtained in step B53:

[0159]

[0160]

[0161] in, express The first token representation vector of Represents the set of all enhanced slot representations.

[0162] Step B55: In order not to lose the original slot information, merge the result of step B54 with the [CLS] position output of step B51 After splicing, we get:

[0163]

[0164] Step B56: All slot representations obtained in step B55 are passed through a fully connected layer and a LayerNorm layer to obtain the final slot representation

[0165]

[0166] in, is the learnable parameter matrix.

[0167] Step B6: Transform the conversation context representation vector obtained in step B4 into The characterization vector obtained in step B5 Perform attention modeling to obtain a knowledge-enhanced domain-slot representation vector containing contextual information

[0168] Specifically, the enhanced domain slot representation and context are modeled with multi-head cross attention:

[0169]

[0170] in, Contains information about enhanced domain slot representation and context.

[0171] Step B7: All domain-slot representation vectors obtained in step B6 After multi-head attention modeling, the final domain-slot representation vector Υ is obtained S The step B7 specifically comprises the following steps:

[0172] Step B71: Input the obtained domain slot representation into L att Layer multi-head attention layer, let For the lth self-attention layer:

[0173]

[0174]

[0175]

[0176]

[0177] Among them, W1 F is the learnable parameter matrix, is the bias term, is the output of the last layer.

[0178] Step B72: Input the output of B71 into a linear layer and a LayerNorm layer to obtain the final domain slot representation Y that combines context information and knowledge information S :

[0179]

[0180]

[0181] in, represents the representation of the jth slot, Υ S Represents the set of all slot representations.

[0182] Step B8: The domain-slot representation vector obtained in step B7 The candidate slot value of the corresponding domain-slot pair is represented by the encoded vector V j '∈V j Perform similarity matching and select the most similar slot value as the prediction result. Compare the predicted value with the true value to calculate the loss, use the back propagation algorithm to calculate the gradient of each parameter in the deep network, and use the stochastic gradient descent algorithm to update the parameters. The step B8 specifically includes the following steps:

[0183] Step B81: For each slot, first encode the candidate value through BERT to obtain the value feature

[0184]

[0185] in Indicates the i-th candidate value of the j-th slot, and finally takes The [cls] bit represents the final value Encode each candidate value to get the candidate value set Since the number of candidate values ​​for each slot is different, the value range of i is different.

[0186] Step 82: Combine all candidate value representations obtained in B81 with the slot representations obtained in B72 Calculate the semantic distance and then select the slot value with the minimum distance as slot Sj The final prediction result of . Here, L2 norm is used as the distance metric. In the training phase, the slot S is calculated at round t. j The true value of The probability is:

[0187]

[0188] The value with the highest probability is taken as the predicted value. exp(·) represents the exponential function, and ||·||2 represents the L2 norm.

[0189] Step B83: The model is trained to maximize the joint probability of all slots, i.e. The loss function for each round t is defined as the accumulation of negative log-likelihoods:

[0190]

[0191] Step B84: Update the learning rate of the loss calculated in B83 through the gradient optimization algorithm AdamW, and use back propagation to iteratively update the model parameters to minimize the loss function to train the model.

[0192] Step B9: When the loss value generated by the deep learning network model is less than the set threshold or reaches the maximum number of iterations, the training of the deep learning model G is terminated.

[0193] like Figure 3 As shown, this embodiment also provides a dialog state tracking system for implementing the above method, including a training set construction module, a model training module and a dialog state tracking module.

[0194] The training set building module is used to collect dialogue context and answer data to build a dialogue training set TS.

[0195] The model training module is used to train the deep learning network model G based on knowledge enhancement.

[0196] The dialogue state tracking module is used to input user speech and system speech into the trained deep network model to output the current dialogue state during the dialogue with the user.

[0197] The above is only a preferred embodiment of the present invention, and does not limit the present invention in other forms. Any technician familiar with the profession may use the above disclosed technical content to change or modify it into an equivalent embodiment with equivalent changes. However, any simple modification, equivalent change and modification made to the above embodiment according to the technical essence of the present invention without departing from the technical solution of the present invention still belongs to the protection scope of the technical solution of the present invention.

Claims

1. A conversation state tracking method based on knowledge enhancement and context awareness, characterized in that: The following steps are involved: Step A: Collect dialogue context and dialogue state data to build a dialogue training set TS; Step B: Use the training set TS to train the knowledge-enhanced deep learning network model G; Step C: Input the conversation data between the user and the system into the deep learning network model G in sequence, and output the current conversation status; The step B specifically comprises the following steps: Step B1: Encode the tth round of dialogue of a dialogue sample in the training set TS to obtain the initial representation vector The tth round of dialogue is the concatenation of the current dialogue, dialogue history and dialogue status; Step B2: Initial representation vector of the dialogue state Input into the multi-head attention mechanism module to obtain the attention-enhanced dialogue state semantic representation vector Step B3: Add the attention-enhanced dialogue state semantic representation vector obtained in step B2 and the initial representation vector of the conversation history Perform cross-attention gating fusion to obtain the fusion representation vector of dialogue history and dialogue state Step B4: Initial representation vector of the current dialogue The fusion representation vector obtained in step B3 Perform word-level concatenation to obtain the conversation context representation vector Step B5: Encode the enhanced domain-slot description through BERT to obtain the initial representation vector The enhanced domain-slot description is input into the knowledge extraction module to obtain the corresponding knowledge embedding representation vector Then the obtained initial characterization vector and knowledge embedding representation vector Fusion is performed to obtain the domain-slot description representation vector after knowledge enhancement Step B6: Transform the conversation context representation vector obtained in step B4 into The characterization vector obtained in step B5 Perform attention modeling to obtain a knowledge-enhanced domain-slot representation vector containing contextual information Step B7: All domain-slot representation vectors obtained in step B6 After multi-head attention modeling, the final domain-slot representation vector Υ is obtained S ; Step B8: The domain-slot representation vector obtained in step B7 The candidate slot value of the corresponding domain-slot pair is represented by the encoded vector V j '∈V j Perform similarity matching and select the most similar slot value as the prediction result; compare the predicted value with the true value to calculate the loss, use the back propagation algorithm to calculate the gradient of each parameter in the deep network, and use the stochastic gradient descent algorithm to update the parameters; Step B9: When the loss value generated by the deep learning network model is less than the set threshold or reaches the maximum number of iterations, the training of the deep learning model G is terminated; The step B1 specifically comprises the following steps: Step B11: The dialogue training set is represented as Where N represents the number of training samples, that is, the number of multi-round dialogue samples; n represents the number of rounds of each multi-round dialogue, (D t ,B t-1 ,H t ) represents a round of training samples in the dialogue training set; where D t Indicates the current round of dialogue. Indicates the conversation history except the current round of conversation, B t-1 represents the current dialogue state and is generated by the model during the prediction process; therefore, the input of the model is [cls] and [sep] are special tokens used to separate the parts of the input; Step B12: X the result of step B12 t After BERT encoding, the corresponding three parts are split into initial representations: Where L is the length of the current conversation, L h is the length of the current conversation history, J is the length of the current conversation state, and d is the dimension of the token representation vector; The step B2 specifically comprises the following steps: Step B21: The historical conversation status portion output from step B12 After multi-head attention modeling: in, are learnable weight parameters, SelfAttention(Q,K,V) and MultiHeadAttention(Q,K,V) are self-attention and multi-head attention mechanisms respectively, and the Concat(·) function concatenates the outputs of h self-attention heads; Step B22: Splice the outputs of multiple heads in step B21 Mapped to the original space, we get the attention-enhanced dialogue state semantic representation vector in, is the learnable parameter matrix; The step B3 specifically comprises the following steps: Step B31: Calculate the interaction attention matrix: in, is the learnable weight parameter matrix corresponding to the dialogue history and dialogue state, A T represents the matrix transpose, a b→h ,a h→b They are the interactive attention matrices of the dialogue state to the dialogue history and the dialogue history to the dialogue state respectively; Step B32: The interactive attention matrix a obtained in step B31 h→b ,a b→h , compute the interaction context representation: Step B33: Calculate the two context representations obtained in step B32 The fusion weight of , and the two are fused according to the fusion weight: L'max(J,L h ) Among them, the max(·) function obtains the maximum value L' of the input length; Padding(x,y,z) aligns the two dimensions to the larger one, and fills z=0 if it is insufficient; is the learnable parameter matrix; σ(·) is the activation function, ⊙ represents the matrix dot product, and finally the fused context representation vector is obtained In step B4, the fused context representation vector obtained in step B33 is The initial representation of the current dialogue round obtained in step B12 Concatenate to get the summarized context information The | operation indicates concatenation at the token sequence level, and the length change of L'+J indicates concatenation in the length direction. The step B5 specifically comprises the following steps: Step B51: Encode the enhanced domain-slot description through BERT to obtain the initial representation vector Among them, BERT fixed Indicates that the BERT does not participate in training. represents the length of the embedded sequence described by the enhanced slot, Indicates enhanced slot description; Step B52: The enhanced domain-slot description is passed through the knowledge extraction module to obtain the domain-slot knowledge representation Among them, KE represents the function of obtaining ConceptNet knowledge embedding, represents the length of the embedded sequence described by the enhanced slot; Step B53: Obtained from step B51 Knowledge embedding obtained with B52 After splicing, we get: Among them, d k represents the dimension of ConceptNet knowledge embedding, ";" represents the concatenation operation, U j Indicates slot The representation after concatenating knowledge embedding; Step B54: Use the self-attention mechanism to further calculate the fusion information of the representation obtained in step B53: in, express The first token representation vector of represents the set of all enhanced slot representations; Step B55: In order not to lose the original slot information, merge the result of step B54 with the [CLS] position output of step B51 After splicing, we get: Step B56: All slot representations obtained in step B55 are passed through a fully connected layer and a LayerNorm layer to obtain the final slot representation in, is the learnable parameter matrix; In step B6, the enhanced domain slot representation and context are subjected to multi-head cross attention modeling: in, Contains information that enhances domain slot representation and context; The step B7 specifically includes the following steps: Step B71: Input the obtained domain slot representation into L att Layer multi-head attention layer, let For the lth self-attention layer: Among them, W1 F is the learnable parameter matrix, is the bias term; Step B72: Input the output of B71 into a linear layer and a LayerNorm layer to obtain the final domain slot representation Y that combines context information and knowledge information S : in, represents the representation of the jth slot, Υ S represents the set of all slot representations; The step B8 specifically includes the following steps: Step B81: For each slot, first encode the candidate value through BERT to obtain the value feature in Indicates the i-th candidate value of the j-th slot, and finally takes The [cls] bit represents the final value Encode each candidate value to get the candidate value set Since the number of candidate values ​​for each slot is different, the value range of i is different; Step 82: Combine all candidate value representations obtained in B81 with the slot representations obtained in B72 Calculate the semantic distance and then select the slot value with the minimum distance as slot S j The final prediction result; here we use L2 norm as the distance metric; in the training phase, we calculate the slot S in round t j The true value of The probability is: The value with the highest probability is taken as the predicted value; where exp(·) represents the exponential function and ||·||2 represents the L2 norm; Step B83: The model is trained to maximize the joint probability of all slots, i.e. The loss function for each round t is defined as the accumulation of negative log-likelihoods: Step B84: Update the learning rate of the loss calculated in B83 through the gradient optimization algorithm AdamW, and use back propagation to iteratively update the model parameters to minimize the loss function to train the model.

2. A conversation state tracking system using the method as claimed in claim 1, characterized in that: include: Build a training set module to collect conversation context and answer data and build a conversation training set TS; Model training module, used to train the deep learning network model G based on knowledge enhancement; as well as The dialogue state tracking module is used to input the user speech and system speech into the trained deep network model to output the current dialogue state during the dialogue with the user.

Images