Conversation state tracking method and system based on AMR image information enhancement

Abstract

The invention relates to a dialogue state tracking method and system based on AMR image information enhancement, and the method comprises the following steps: A, collecting dialogue data, and building a dialogue training set DS containing dialogues and dialogue states; step B: constructing a deep learning network model M based on AMR image information enhancement, the deep learning network model M extracting an AMR dialogue structure from each round of dialogue through an AMR parser, obtaining dialogue representation based on AMR by using an image neural network, fusing image and text information by using a multi-head attention mechanism and original dialogue information interaction, and obtaining a dialogue representation based on AMR; the conversation state is tracked through the deep learning network model M; training a deep learning network model M based on AMR image information enhancement by using the dialogue training set DS; and step C, inputting the dialogue into the trained deep learning network model M, and outputting a corresponding dialogue state. According to the method and system, the accuracy of dialogue state tracking is improved, the adaptability is high, and the user experience is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and in particular to a method and system for tracking a dialog state based on AMR graph information enhancement. Background Art

[0002] AI chatbots are software applications that use artificial intelligence (AI) and natural language processing (NLP) to simulate human conversations with customers. They can answer common questions, provide information, and perform simple tasks, such as booking appointments, processing payments, or updating account details. AI chatbots can be integrated with various platforms, such as websites, mobile applications, social media, or messaging applications to provide a lot of customer service without the need for human agents.

[0003] The core technologies of AI chatbots can be divided into two major categories: open domain dialogue systems and task-oriented dialogue systems. However, since human dialogue is inherently complex and ambiguous, creating an open domain dialogue system that can perform arbitrary tasks is an unsolved problem. The focus of current practice is to build task-oriented dialogue (TOD) systems that are limited to specific domains (such as flight reservations). For example, consider a hotel reservation dialogue system. The user may make a request like: "I want to book a hotel in the city center tomorrow night, a single room, for two nights." The task-oriented dialogue system needs to understand the user's requirements, which may involve entity recognition (date, location, room type, etc.), and interact with the hotel reservation system to complete the reservation.

[0004] Dialogue State Track (DST) is an important part of the task-based dialogue task flow in natural language processing. Its basic goal is to obtain the current dialogue state based on the dialogue context. The dialogue state is a summary of the user's goals from the beginning of the dialogue to the current dialogue, usually in the form of a combination of multiple slot-value pairs, and sometimes also includes information such as the domain to which the dialogue belongs and the user's intention. Dialogue state tracking refers to the process of inferring and updating the current dialogue state by combining information such as the dialogue history, the current dialogue, and the previous round of dialogue state. The goal of a task-oriented dialogue system is to give appropriate answers to questions raised by users in a specified domain, or to give candidate information to users' requests, communicate with users naturally, and complete the requirements raised by users, so that users themselves feel that they are talking to "people". It is of great help in accelerating social development and progress, increasing user efficiency, and improving users' life experience. In addition, it has a wide range of commercial uses and has become a field that major companies are competing to research, with great research value and significance.

[0005] In recent years, deep learning methods have been widely used in many fields of natural language processing. Deep learning is used to track dialogue states, without the need for manual heavy rule design work, and can automatically extract semantic feature information from the dialogue context. Some existing methods consider that slots are related, and they believe that slots are not conditionally independent, such as hotel star rating and price range. Modeling slot relationships can be divided into explicit and implicit. One is to use self-attention to explicitly consider these relationships. Similarly, Manotumruksa adopted a hybrid architecture to achieve sequence value prediction based on the GPT-2 model while using the graph attention network (GAT) to model the relationship between slots and values. Another type of work uses knowledge obtained from domain ontology, such as using its hierarchical structure to implicitly model the relationship between slots. These relationships can also be represented by a graph with slots and domains as nodes. Wang first built a pattern graph based on the ontology, and then used GAT to fuse the information in the dialogue history and the pattern graph. Chen extended this method by dynamically updating the slot relationships in the pattern graph based on the dialogue context. Guo used both dialogue turns and slot-value pairs as nodes, only considering relevant turns and solving coreference relationships. Other approaches consider adapting models to new domains. Dingliwal et al. adopt meta-learning, which uses meta-learned model parameters and initializes fine-tuning on the target domain. Work around pattern-based datasets

[19] uses slot descriptions to handle unseen domains and slots. A drawback of these approaches is that they rely on the similarity between the unseen domain and the initial fine-tuning domain. Another group of approaches attempts to mine external knowledge from other tasks with richer resources. Use frame network semantic analysis as weak supervision to identify potential slots. Li proposed different methods to pre-train on reading comprehension data before applying the model to DST. Similarly, Shin redefined DST as a dialogue summarization task based on templates and leveraging external annotated data. It is worth noting that the above-mentioned PLM adaptation method allows more effective learning when less data is available, and is also a potential solution to the problem of data scarcity. Along this line of thought, Mi proposed a small-sample DST self-learning method that is complementary to TOD-BERT.

[0006] However, the above solutions only model the relationship between slots and dialogue contexts and the correlation between slots. The representation information of the dialogue text is not clear enough, resulting in insufficient expression of dialogue information, using the entire dialogue history in the training phase, introducing too many dialogues unrelated to the slots, resulting in dialogue information redundancy, and using the real dialogue state for training during training and using the previous round of dialogue states that may be wrong for prediction in the prediction phase, which leads to inconsistency of dialogue information. Summary of the invention

[0007] The purpose of the present invention is to provide a method and system for tracking a conversation state based on AMR graph information enhancement, which improves the accuracy of conversation state tracking, has strong adaptability, and enhances user experience.

[0008] In order to achieve the above object, the technical solution adopted by the present invention is: a method and system for tracking the dialogue state based on AMR image information enhancement, comprising the following steps:

[0009] Step A: Collect dialogue data and establish a dialogue training set DS including dialogues and dialogue states;

[0010] Step B: Construct a deep learning network model M based on AMR graph information enhancement. The deep learning network model M extracts the AMR dialogue structure from each round of dialogue through the AMR parser, obtains the AMR-based dialogue representation using the graph neural network, and uses the multi-head attention mechanism to interact with the original dialogue information to fuse the graph and text information, so as to track the dialogue state through the deep learning network model M; use the dialogue training set DS to train the deep learning network model M based on AMR graph information enhancement;

[0011] Step C: Input the dialogue into the trained deep learning network model M and output the corresponding dialogue state.

[0012] Furthermore, the specific implementation method of step A is:

[0013] Step A1: Collect conversation data, including the following:

[0014] User input: questions asked by users in the dialogue, in text form;

[0015] System response: The dialogue system responds to user input in the form of text reply;

[0016] Conversation history: records each round of interaction in the conversation, including user input, system response, and contextual information involved;

[0017] Slot candidate value: the value that a slot may appear in; a slot is a category of key entities in a conversation;

[0018] Conversation status: status information in the conversation, including the current conversation status and user context information;

[0019] Step A2: A round of dialogue samples are formed using the collected data of a round of dialogue, and then a dialogue training set DS is formed; a round of dialogue samples includes the current round of dialogue, the dialogue state, the slots to be tracked and their candidate values.

[0020] Furthermore, the step B specifically includes the following steps:

[0021] Step B1: Initially encode each round of dialogue samples in the dialogue training set DS to obtain the initial features H of each round of dialogue, dialogue state, slots to be tracked, and corresponding candidate values t HB t-1 , Indicates slot s j The initial characteristics of Indicates slot s j The initial features of the nth candidate value, V is the slot candidate value; the conversation is parsed using the AMR parser to obtain the AMR parse tree of each round of conversation, and the AMR parse tree is linearized to obtain the conversation AMR entity node set N = {n1, n2, n3…n k}, k∈K and the corresponding relation set R={r 11 , r 12 , r 13 …r xy}, x∈k, y∈k, where K is the number of dialog AMR entity nodes; r xy is the relationship between nodes x and y; entity nodes represent concepts or entities, while node relationships represent the semantic relationship between these concepts or entities;

[0022] Step B2: Connect the conversation AMR entity node set N obtained in step B1 according to the node relationship set R to form a conversation node structure subgraph G = (N, R), and use N and R to construct the adjacency matrix A of graph G; the adjacency matrix A is a matrix of |N|×|N|, and the element A in the adjacency matrix A is ij Indicates whether there is an edge between node i and node j. If there is an edge between the two nodes, it can be found in the relationship R. The AMR entity nodes of the dialogue are encoded using the pre-trained model to obtain the initial vector representation C = {c1, c2, c3…c k}; Then, a K-layer graph convolutional network is used to encode the AMR entity nodes and edges, and the node representation is updated by aggregating the information of the node neighbors. For each node, the information of its neighboring nodes is considered and aggregated through the weight matrix to update the node representation and obtain the semantic features of the dialogue node.

[0023] Step B3: In order to enhance the representation of the dialogue text, the semantic features of the dialogue nodes are dynamically fused through the semantic information dynamic fusion module and the initial feature H t , highlighting the focus of the conversation without losing the original information in the conversation, and obtaining strong information features of the conversation The obtained dialogue strong information features The slot H that needs to be tracked sInput to the multi-head attention module to obtain the strong information vector of the conversation of the specific slot after attention enhancement The initial feature HB of the dialogue state obtained in step B1 t-1 With slot H s Input to the multi-head attention module to obtain the paragraph-level dialogue semantic representation vector P of the specific slot after attention enhancement t ;

[0024] Step B4: The fusion representation vector obtained in step B3 The paragraph-level conversation semantic representation vector P obtained in step B1 t Perform gated fusion to obtain the final slot representation vector

[0025] Step B5: The slot characterization vector obtained in step B4 The candidate value of the corresponding slot is represented by the encoded vector V′ j ∈V j Use Euclidean distance to calculate the final value v of each slot j The probability distribution of

[0026] Perform loss calculations, use the back-propagation algorithm to calculate the gradients of each parameter in the deep learning network model, and use the stochastic gradient descent algorithm to update the parameters;

[0027] Step B6: When the loss value generated by the deep learning network model is less than the set threshold or reaches the maximum number of iterations, the training of the deep learning model G is terminated.

[0028] Furthermore, the step B1 specifically includes the following steps:

[0029] Step B11: The dialogue training set is represented as Where N represents the number of training samples, that is, the number of multi-round dialogue samples; n represents the number of dialogue rounds, CD t , B t-1 ) represents a round of training samples in the dialogue training set; Represents the current round of dialogue, where R t is the system discourse, U t is the user's words; B t-1 = Indicates the current dialogue state, V j Indicates slot S j The initial value of each slot is none. s Indicates the slot that needs to be tracked Where j∈J; V represents V j Indicates the corresponding slot Candidate value of Therefore, the input of the deep learning network model is X t =[CLS]D t [SEP], XB t-1 =[CLS]B t-1 [SEP]; [CLS] and [SEP] are special tokens used to separate the parts of the input;

[0030] Step B12: Input X to the model t ,XB t-1 After BERT encoding, the corresponding initial representation is obtained: Where L and LB are the lengths of the current conversation and conversation state respectively, and d is the dimension of the token representation vector;

[0031] Step B13: Generate an AMR parse tree for each round of dialogue in the dialogue training set through the AMR parser to obtain the tree-shaped AMR representation T of the dialogue dia =AMR(H t ), T dia The semantics of a natural language sentence is represented as a directed acyclic graph, where nodes represent concepts and edges represent relationships between concepts, thus forming an abstract semantic tree structure. Then, the graph is linearized to make it suitable for BERT. A depth-first search is used to traverse the AMR graph to generate a linear sequence: linear =LINE(T dia ); finally from T linear Extract the entity set N = {n1, n2, n3...n k} and the corresponding relation set R = {r1, r2, r3…r k}.

[0032] Furthermore, in step B2, an adjacency matrix A of |N|×|N| is created using the entity set N and the relationship set R obtained in step B13, wherein the initial values ​​of all elements are zero. If there is an edge (i, j) between entities, the corresponding element A of the adjacency matrix is ​​replaced by ij Set to 1; the entity set N is input into the pre-trained model BERT to encode the initial vector representation of the entity C = BERT (N); the constructed adjacency matrix and the initial vector representation C of the entity are input into the K-layer relationship graph convolutional network to obtain the aggregate representation of each entity node. For each node i, its new representation Where Neighbors(i) represents the set of neighbor nodes of node i; xj is the initial vector of the entity; deg(i) represents the degree of node i, that is, the number of edges connected to node i; W is a learnable weight matrix; σ is an activation function; for a round of dialogue, the semantic features of the dialogue node are , d is the representation of the AMR entity node in the conversation.

[0033] Furthermore, the step B3 specifically includes the following steps:

[0034] Step B31: The current conversation representation H outputted in step B11 is t The semantic features of the dialogue nodes in step B2 After the dynamic fusion module of semantic information, the model is built to obtain strong information features of the dialogue

[0035] First, feature H t and As the input of the multi-head attention layer, As the query vector, define an attention weight matrix Use dot product attention to compute this matrix:

[0036]

[0037] in, Yes H t and The matching score, W attention is a learnable weight matrix used to adjust the importance between different dimensions; the attention weight matrix W is then used attention To calculate H t The weighted feature representation A dia :

[0038] A dia =W attention ·H t

[0039] In order to enhance the importance of graph information to certain things in the conversation, a gating mechanism is used to fuse graph information. and weighted conversation text information A dia :

[0040]

[0041] Among them, z is the graph information and text information A dia The fusion weight between h ∈R d×d , W b ∈R d×d is the learnable parameter matrix; σ(·) is the Relu activation function;

[0042] Step B32: Represent the dialog state obtained in step B11 t-1 Similarly, the slot is input into the multi-head attention module to model the paragraph-level dialogue semantic representation vector Pt ;

[0043] Step B33: The strong information features of the conversation obtained in step B31 Similarly, the slot is input into the multi-head attention module to model the specific representation of the strong information of the conversation of a specific slot

[0044] Furthermore, the specific implementation method of step B4 is:

[0045] Calculate the strong information representation of the specific slot dialogue obtained in step B31 and the paragraph-level conversation semantic representation vector P obtained in step B32 t Fusion weights:

[0046]

[0047] After the linear layer, we get the weights:

[0048]

[0049] Where W is the weight matrix of the linear layer and b is the bias vector;

[0050] Finally, we get the slot representation vector

[0051]

[0052] Furthermore, the step B5 specifically includes the following steps:

[0053] Step B51: For each slot, first encode the candidate value through BERT to obtain the value representation:

[0054]

[0055] in, Indicates the i-th candidate value of the j-th slot, and finally takes The [CLS] bit represents the final value Encode each candidate value to get the candidate value set Since the number of candidate values ​​for each slot is different, the value range of i is different;

[0056] Step 52: Combine all candidate value representations obtained in B51 with the slot representations obtained in B4 Calculate the semantic distance and then select the slot value with the smallest distance as the final prediction result of slot Sj; use L2 norm as the distance metric; in the training phase, calculate the slot S at round t j The true value V j ∈V j The probability is:

[0057]

[0058] The value with the highest probability is taken as the predicted value; where exp(·) represents the exponential function and ||·||2 represents the L2 norm;

[0059] For slot S j , considering that some slots can be inherited without updating relative to the previous round of dialogue state, while other slots need to be updated, the loss of whether to update or maintain is as follows:

[0060]

[0061] in, is the loss function for state transition prediction, whose value set is {keep, update}, W c , W p are the learnable parameters of the linear layer, is the update probability of slot s in the tth round, is the state transition label, when updated Keep

[0062] Step B53: Train the model to maximize the joint probability of all slots, i.e. The loss function for each round t is defined as the accumulation of negative log-likelihoods:

[0063]

[0064] The final loss is:

[0065]

[0066] Step B53: Update the learning rate of the loss calculated in B52 through the stochastic gradient descent algorithm, and use back propagation to iteratively update the model parameters to minimize the loss function to train the model.

[0067] The present invention also provides a dialogue state tracking system based on AMR image information enhancement, including a memory, a processor, and computer program instructions stored in the memory and capable of being executed by the processor. When the processor executes the computer program instructions, the above-mentioned method steps can be implemented.

[0068] Compared with the prior art, the present invention has the following beneficial effects: The present invention provides a method and system for tracking a dialogue state based on AMR graph information enhancement, thereby effectively solving the problems mentioned in the background technology, such as inaccuracy in dialogue state tracking, insufficient processing of complex dialogues, etc., and optimizing dialogue management. The present invention adopts a mechanism for integrating AMR graph information to dynamically obtain additional dialogue information by analyzing the user's input and dialogue context in real time. The present invention can flexibly respond to changes in user intent and improve the accuracy of dialogue state tracking; it combines a variety of advanced deep learning models (such as attention mechanisms, etc.) to capture complex patterns and relationships in dialogues. This model integration method can better understand the nuances of language, thereby improving the accuracy of state tracking; the present invention attaches importance to the contextual information of the dialogue and understands the current user intent by analyzing the dialogue history. This context-aware capability makes state tracking more accurate and can effectively handle multi-round dialogue scenarios. Therefore, the present invention has the following beneficial effects: (1) Improved accuracy: By integrating AMR graph information and deep learning models, the present invention significantly improves the accuracy of dialogue state tracking and can accurately capture the user's intentions and needs; (2) Enhanced adaptability: Context-aware analysis and user feedback loops enhance the system's adaptability to complex dialogue situations, enabling the system to flexibly respond to changes in user intentions and provide more personalized services; (3) Improved user experience: Accurate state tracking and highly personalized services significantly improve the user experience, reduce the user's operating burden, and make the interaction more smooth and natural. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] Figure 1 is a flow chart of a method implementation of an embodiment of the present invention;

[0070] Figure 2 It is a model architecture diagram in an embodiment of the present invention. DETAILED DESCRIPTION

[0071] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0072] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present application belongs.

[0073] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0074] like Figure 1 As shown, this embodiment provides a method for tracking a dialog state based on AMR graph information enhancement, comprising the following steps:

[0075] Step A: Collect dialogue data and establish a dialogue training set DS including dialogues and dialogue states;

[0076] Step B: Construct a deep learning network model M based on AMR graph information enhancement. The deep learning network model M extracts the AMR dialogue structure from each round of dialogue through the AMR parser, obtains the AMR-based dialogue representation using the graph neural network, and uses the multi-head attention mechanism to interact with the original dialogue information to fuse the graph and text information, so as to track the dialogue state through the deep learning network model M; use the dialogue training set DS to train the deep learning network model M based on AMR graph information enhancement;

[0077] Step C: Input the dialogue into the trained deep learning network model M and output the corresponding dialogue state.

[0078] In this embodiment, the specific implementation method of step A is:

[0079] Step A1: Collect conversation data, including the following:

[0080] (1) User input: questions asked by users in a conversation, in the form of text input;

[0081] (2) System response: The dialogue system responds to the user input in the form of text reply;

[0082] (3) Conversation history: records each round of interaction in the conversation, including user input, system response, and contextual information involved; this helps understand the coherence and context of the conversation;

[0083] (4) Slot candidate value: the possible value of a slot, usually a predefined value; a slot is a category of key entities in a conversation;

[0084] (5) Dialogue state: state information in the dialogue, including the current dialogue state and user context information; usually in the form of slot-value pairs (slot and value concatenated), where the value is the key entity in the dialogue.

[0085] Step A2: A round of dialogue samples are formed using the collected data of a round of dialogue, and then a dialogue training set DS is formed; a round of dialogue samples includes the current round of dialogue, the dialogue state, the slots to be tracked and their candidate values.

[0086] Figure 2 : is an architecture diagram of the deep learning network model M based on AMR image information enhancement in this embodiment. In this embodiment, the specific implementation steps of step B are as follows.

[0087] Step B1: Initially encode each round of dialogue samples in the dialogue training set DS to obtain the initial features H of each round of dialogue, dialogue state, slots to be tracked, and corresponding candidate values t HB t-1 , Indicates slot The initial characteristics of Indicates slot The initial features of the nth candidate value, V is the slot candidate value; the conversation is parsed using the AMR parser to obtain the AMR parse tree of each round of conversation, and the AMR parse tree is linearized to obtain the conversation AMR entity node set N = {n1, n2, n3…n k}, k∈K and the corresponding relation set R={r 11 , r 12 , r 13 …r xy}, x∈k, y∈k, where K is the number of dialog AMR entity nodes; r xy is the relationship between nodes x and y; entity nodes represent concepts or entities, and node relationships represent the semantic relationship between these concepts or entities.

[0088] The step B1 specifically comprises the following steps:

[0089] Step B11: The dialogue training set is represented as Where N represents the number of training samples, that is, the number of multi-round dialogue samples; n represents the number of dialogue rounds, CD t , B t-1 ) represents a round of training samples in the dialogue training set; Represents the current round of dialogue, where R t is the system discourse, U t For user discourse; Indicates the current dialogue state, V j Indicates slot S j The initial value of each slot is none. s Indicates the slot that needs to be tracked Where j∈J; V represents V j Indicates the corresponding slot Candidate value of Therefore, the input of the deep learning network model is X t =[CLS]D t [SEP], XB t-1 =[CLS]B t-1 [SEP]; [CLS] and [SEP] are special tokens used to separate the various parts of the input.

[0090] Step B12: Input X to the model t , XB t- 1After BERT encoding, the corresponding initial representation is obtained: Where L and LB are the lengths of the current conversation and conversation state respectively, and d is the dimension of the token representation vector.

[0091] Step B13: Generate an AMR parse tree for each round of dialogue in the dialogue training set through the AMR parser to obtain the tree-shaped AMR representation T of the dialogue dia =AMR(H t ), T dia The semantics of a natural language sentence is represented as a directed acyclic graph, where nodes represent concepts and edges represent relationships between concepts, thus forming an abstract semantic tree structure. Then, the graph is linearized to make it suitable for BERT. A depth-first search is used to traverse the AMR graph to generate a linear sequence: linear =LINE(T dia ); finally from T linear Extract the entity set N = {n1, n2, n3...n k} and the corresponding relation set R = {r1, r2, r3…r k}.

[0092] Step B2: Connect the conversation AMR entity node set N obtained in step B1 according to the node relationship set R to form a conversation node structure subgraph G = (N, R), and use N and R to construct the adjacency matrix A of graph G; the adjacency matrix A is a matrix of |N|×|N|, and the element A in the adjacency matrix A is ij Indicates whether there is an edge between node i and node j. If there is an edge between the two nodes, it can be found in the relationship R. The AMR entity nodes of the dialogue are encoded using the pre-trained model to obtain the initial vector representation C = {c1, c2, c3…c k}; Then, a K-layer graph convolutional network is used to encode the AMR entity nodes and edges, and the node representation is updated by aggregating the information of the node neighbors. For each node, the information of its neighboring nodes is considered and aggregated through the weight matrix to update the node representation and obtain the semantic features of the dialogue node.

[0093] Specifically, use the entity set N and relationship set R obtained in step B13 to create an |N|×|N| adjacency matrix A, in which all elements are initially zero. If there is an edge (i, j) between entities, the corresponding element A of the adjacency matrix is ij Set to 1; the entity set N is input into the pre-trained model BERT to encode the initial vector representation of the entity C = BERT (N); the constructed adjacency matrix and the initial vector representation C of the entity are input into the K-layer relationship graph convolutional network to obtain the aggregate representation of each entity node. For each node i, its new representation Where Neighbors(i) represents the set of neighbor nodes of node i; x j is the initial vector of the entity; deg(i) represents the degree of node i, that is, the number of edges connected to node i; W is a learnable weight matrix; σ is an activation function; for a round of dialogue, the semantic features of the dialogue node are d is the representation of the AMR entity node in the conversation.

[0094] Step B3: In order to enhance the representation of the dialogue text, the semantic features of the dialogue nodes are dynamically fused through the semantic information dynamic fusion module and the initial feature H t , highlighting the focus of the conversation without losing the original information in the conversation, and obtaining strong information features of the conversation The obtained dialogue strong information features The slot H that needs to be tracked s Input to the multi-head attention module to obtain the strong information vector of the conversation of the specific slot after attention enhancement The initial feature HB of the dialogue state obtained in step B1 t-1 With slot H s Input to the multi-head attention module to obtain the paragraph-level dialogue semantic representation vector P of the specific slot after attention enhancement t .

[0095] The step B3 specifically comprises the following steps:

[0096] Step B31: The current conversation representation H outputted in step B11 is t The semantic features of the dialogue nodes in step B2 After the dynamic fusion module of semantic information, the model is built to obtain strong information features of the dialogue

[0097] First, feature H t and As the input of the multi-head attention layer, As the query vector, define an attention weight matrix Use dot product attention to compute this matrix:

[0098]

[0099] in, Yes H t and The matching score, W attention is a learnable weight matrix used to adjust the importance between different dimensions; the attention weight matrix W is then used attentio n to calculate H t The weighted feature representation A dia :

[0100] A dia =W attention ·H t

[0101] In order to enhance the importance of graph information to certain things in the conversation, a gating mechanism is used to fuse graph information. and weighted conversation text information A dia :

[0102]

[0103] Among them, z is the graph information and text information A dia The fusion weight between h ∈R d×d , W b ∈R d×d is the learnable parameter matrix; σ(·) is the ReLU activation function.

[0104] Step B32: Represent the dialog state obtained in step B11 t-1 Similarly, the slot is input into the multi-head attention module to model the paragraph-level dialogue semantic representation vector P t .

[0105] Step B33: The strong information features of the conversation obtained in step B31 Similarly, the slot is input into the multi-head attention module to model the specific representation of the strong information of the conversation of a specific slot

[0106] Step B4: The fusion representation vector obtained in step B3 The paragraph-level conversation semantic representation vector P obtained in step B1 t Perform gated fusion to obtain the final slot representation vector

[0107] The specific implementation method of step B4 is:

[0108] Calculate the strong information representation of the specific slot dialogue obtained in step B31 and the paragraph-level conversation semantic representation vector P obtained in step B32 t Fusion weights:

[0109]

[0110] After the linear layer, we get the weights:

[0111]

[0112] Where W is the weight matrix of the linear layer and b is the bias vector.

[0113] Finally, we get the slot representation vector

[0114]

[0115] Step B5: The slot characterization vector obtained in step B4 The candidate value of the corresponding slot is represented by the encoded vector V′ j ∈V j Use Euclidean distance to calculate the final value v of each slot j Probability distribution of . Loss calculation is performed, the gradient of each parameter in the deep learning network model is calculated using the back propagation algorithm, and the parameters are updated using the stochastic gradient descent algorithm.

[0116] The step B5 specifically comprises the following steps:

[0117] Step B51: For each slot, first encode the candidate value through BERT to obtain the value representation:

[0118]

[0119] in, Indicates the i-th candidate value of the j-th slot, and finally takes The [CLS] bit represents the final value Encode each candidate value to get the candidate value set Since the number of candidate values ​​for each slot is different, the value range of i is different.

[0120] Step 52: Combine all candidate value representations obtained in B51 with the slot representations obtained in B4 Calculate the semantic distance and then select the slot value with the minimum distance as slot S j The final prediction result; use L2 norm as the distance metric; in the training phase, calculate the slot S in round t j The true value V j ∈V j The probability is:

[0121]

[0122] The value with the highest probability is taken as the predicted value; where exp(·) represents the exponential function, and ||·||2 represents the L2 norm.

[0123] For slot S j , Considering that some slots can be inherited without updating relative to the previous round of dialogue state, while other slots need to be updated, a slot value distillation method is designed. For whether to update or keep, the loss is as follows:

[0124]

[0125] in, is the loss function for state transition prediction, whose value set is {keep, update}, W c , W p are the learnable parameters of the linear layer, is the update probability of slot s in the tth round, is the state transition label, when updated Keep

[0126] Step B53: Train the model to maximize the joint probability of all slots, i.e. The loss function for each round t is defined as the accumulation of negative log-likelihoods:

[0127]

[0128] The final loss is:

[0129]

[0130] Step B53: Update the learning rate of the loss calculated in B52 through the stochastic gradient descent algorithm, and use back propagation to iteratively update the model parameters to minimize the loss function to train the model.

[0131] Step B6: When the loss value generated by the deep learning network model is less than the set threshold or reaches the maximum number of iterations, the training of the deep learning model G is terminated.

[0132] This embodiment also provides a conversation state tracking system based on AMR graph information enhancement, including a memory, a processor, and computer program instructions stored in the memory and executable by the processor. When the processor executes the computer program instructions, the above method steps can be performed.

[0133] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0134] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0135] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0136] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0137] The above is only a preferred embodiment of the present invention, and does not limit the present invention in other forms. Any technician familiar with the profession may use the above disclosed technical content to change or modify it into an equivalent embodiment with equivalent changes. However, any simple modification, equivalent change and modification made to the above embodiment according to the technical essence of the present invention without departing from the technical solution of the present invention still belongs to the protection scope of the technical solution of the present invention.

Claims

1. A method for tracking conversation status based on AMR graph information enhancement, characterized in that: The following steps are involved: Step A: Collect dialogue data and establish a dialogue training set DS including dialogues and dialogue states; Step B: Construct a deep learning network model M based on AMR graph information enhancement. The deep learning network model M extracts the AMR dialogue structure from each round of dialogue through the AMR parser, obtains the AMR-based dialogue representation using the graph neural network, and uses the multi-head attention mechanism to interact with the original dialogue information to fuse the graph and text information, so as to track the dialogue status through the deep learning network model M. Use the dialogue training set DS to train the deep learning network model M based on AMR graph information enhancement; Step C: Input the dialogue into the trained deep learning network model M and output the corresponding dialogue state.

2. The method for tracking a conversation state based on AMR graph information enhancement according to claim 1, characterized in that: The specific implementation method of step A is: Step A1: Collect conversation data, including the following: User input: questions asked by users in the dialogue, in text form; System response: The dialogue system responds to user input in the form of text reply; Conversation history: records each round of interaction in the conversation, including user input, system response, and contextual information involved; Slot candidate value: the value that a slot may have; Slots are categories of key entities in a conversation; Conversation status: status information in the conversation, including the current conversation status and user context information; Step A2: A round of dialogue samples are formed using the collected data of a round of dialogue, and then a dialogue training set DS is formed; a round of dialogue samples includes the current round of dialogue, the dialogue state, the slots to be tracked and their candidate values.

3. The method for tracking a dialog state based on AMR image information enhancement according to claim 1, characterized in that: The step B specifically comprises the following steps: Step B1: Initially encode each round of dialogue samples in the dialogue training set DS to obtain the initial features H of each round of dialogue, dialogue state, slots to be tracked, and corresponding candidate values t HB t-1 , Indicates slot The initial characteristics of Indicates slot The initial features of the nth candidate value, V is the slot candidate value; the conversation is parsed using the AMR parser to obtain the AMR parse tree of each round of conversation, and the AMR parse tree is linearized to obtain the conversation AMR entity node set N = {n1, n2, n3…n k }, k∈K and the corresponding relation set R={r 11 , r 12 , r 13 …r xy }, x∈k, y∈k, where K is the number of dialog AMR entity nodes; r xy is the relationship between nodes x and y; entity nodes represent concepts or entities, while node relationships represent the semantic relationship between these concepts or entities; Step B2: Connect the conversation AMR entity node set N obtained in step B1 according to the node relationship set R to form a conversation node structure subgraph G = (N, R), and use N and R to construct the adjacency matrix A of graph G; the adjacency matrix A is a matrix of |N|×|N|, and the element A in the adjacency matrix A is ij Indicates whether there is an edge between node i and node j. If there is an edge between the two nodes, it can be found in the relationship R. The AMR entity nodes of the dialogue are encoded using the pre-trained model to obtain the initial vector representation C = {c1, c2, c3…c k }; Then, a K-layer graph convolutional network is used to encode the AMR entity nodes and edges, and the node representation is updated by aggregating the information of the node neighbors. For each node, the information of its neighboring nodes is considered and aggregated through the weight matrix to update the node representation and obtain the semantic features of the dialogue node. Step B3: In order to enhance the representation of the dialogue text, the semantic features of the dialogue nodes are dynamically fused through the semantic information dynamic fusion module and the initial feature H t , highlighting the focus of the conversation without losing the original information in the conversation, and obtaining strong information features of the conversation The obtained dialogue strong information features The slot H that needs to be tracked s Input to the multi-head attention module to obtain the strong information vector of the conversation of the specific slot after attention enhancement The initial feature HB of the dialogue state obtained in step B1 t-1 With slot H s Input to the multi-head attention module to obtain the paragraph-level dialogue semantic representation vector P of the specific slot after attention enhancement t ; Step B4: The fusion representation vector obtained in step B3 The paragraph-level conversation semantic representation vector P obtained in step B1 t Perform gated fusion to obtain the final slot representation vector Step B5: The slot characterization vector obtained in step B4 The candidate value of the corresponding slot is represented by the encoded vector V′ j ∈V j The Euclidean distance is used to calculate the probability distribution of each slot value vj; Perform loss calculation, use the back-propagation algorithm to calculate the gradients of each parameter in the deep learning network model, and use the stochastic gradient descent algorithm to update the parameters; Step B6: When the loss value generated by the deep learning network model is less than the set threshold or reaches the maximum number of iterations, the training of the deep learning model G is terminated.

4. The method for tracking a dialog state based on AMR image information enhancement according to claim 3, characterized in that: The step B1 specifically comprises the following steps: Step B11: The dialogue training set is represented as Where N represents the number of training samples, that is, the number of multi-round dialogue samples; n represents the number of dialogue rounds, (D t , B t-1 ) represents a round of training samples in the dialogue training set; Represents the current round of dialogue, where R t is the system discourse, U t For user discourse; Indicates the current dialogue state, V j Indicates slot S j The initial value of each slot is none. s Indicates the slot that needs to be tracked Where j∈J; V represents V j Indicates the corresponding slot Candidate value of Therefore, the input of the deep learning network model is X t =[CLS]D t [SEP], XB t-1 =[CLS]B t-1 [SEP]; [CLS] and [SEP] are special tokens used to separate the parts of the input; Step B12: Input X to the model t , XB t-1 After BERT encoding, the corresponding initial representation is obtained: Where L and LB are the lengths of the current conversation and conversation state respectively, and d is the dimension of the token representation vector; Step B13: Generate an AMR parse tree for each round of dialogue in the dialogue training set through the AMR parser to obtain the tree-shaped AMR representation T of the dialogue dia =AMR(H t ), T dia The semantics of a natural language sentence is represented as a directed acyclic graph, where nodes represent concepts and edges represent relationships between concepts, thus forming an abstract semantic tree structure. Then, the graph is linearized to make it suitable for BERT. A depth-first search is used to traverse the AMR graph to generate a linear sequence: linear =LINE(T dia ); finally from T linear Extract the entity set N = {n1, n2, n3...n k } and the corresponding relation set R = {r1, r2, r3…r k }.

5. The method for tracking a dialog state based on AMR image information enhancement according to claim 4, characterized in that: In step B2, the entity set N and relationship set R obtained in step B13 are used to create a The adjacency matrix A of the entity, where all elements are initially zero, if there is an edge (i, j) between entities, the corresponding element A of the adjacency matrix ij Set to 1; the entity set N is input into the pre-trained model BERT to encode the initial vector representation of the entity C = BERT (N); the constructed adjacency matrix and the initial vector representation C of the entity are input into the K-layer relationship graph convolutional network to obtain the aggregate representation of each entity node. For each node i, its new representation Where Neighbors(i) represents the set of neighbor nodes of node i; x j is the initial vector of the entity; deg(i) represents the degree of node i, that is, the number of edges connected to node i; W is a learnable weight matrix; σ is an activation function; for a round of dialogue, the semantic features of the dialogue node are d is the representation of the AMR entity node in the conversation.

6. The method for tracking a dialog state based on AMR image information enhancement according to claim 5, characterized in that: The step B3 specifically comprises the following steps: Step B31: The current conversation representation H outputted in step B11 is t The semantic features of the dialogue nodes in step B2 After the dynamic fusion module of semantic information, the model is built to obtain strong information features of the dialogue First, feature H t and As the input of the multi-head attention layer, As the query vector, define an attention weight matrix Use dot product attention to calculate this matrix: in, Yes H t and The matching score, W attention is a learnable weight matrix used to adjust the importance between different dimensions; the attention weight matrix W is then used attention To calculate H t The weighted feature representation A dia : A dia =W attention ·H t In order to enhance the importance of graph information to certain things in the conversation, a gating mechanism is used to fuse graph information. and weighted conversation text information A dia : Among them, z is the graph information and text information A dia The fusion weight between h ∈R d×d , W b ∈R d×d is the learnable parameter matrix; σ(·) is the Relu activation function; Step B32: Represent the dialog state obtained in step B11 t-1 Similarly, the slot is input into the multi-head attention module to model the paragraph-level dialogue semantic representation vector P t ; Step B33: The strong information features of the conversation obtained in step B31 Similarly, the slot is input into the multi-head attention module to model the specific representation of the strong information of the conversation of a specific slot 7. The method for tracking a dialog state based on AMR image information enhancement according to claim 6, characterized in that: The specific implementation method of step B4 is: Calculate the strong information representation of the specific slot dialogue obtained in step B31 and the paragraph-level conversation semantic representation vector P obtained in step B32 t Fusion weights: After the linear layer, we get the weights: Where W is the weight matrix of the linear layer and b is the bias vector; Finally, we get the slot representation vector 8. The method for tracking a dialog state based on AMR image information enhancement according to claim 7, characterized in that: The step B5 specifically comprises the following steps: Step B51: For each slot, first encode the candidate value through BERT to obtain the value representation: in, Indicates the i-th candidate value of the j-th slot, and finally takes The [CLS] bit represents the final value Encode each candidate value to get the candidate value set Since the number of candidate values ​​for each slot is different, the value range of i is different; Step 52: Combine all candidate value representations obtained in B51 with the slot representations obtained in B4 Calculate the semantic distance and then select the slot value with the smallest distance as slot S j The final prediction result; use L2 norm as the distance metric; in the training phase, calculate the slot S in round t j The true value of The probability is: The value with the highest probability is taken as the predicted value; where exp(·) represents the exponential function and ||·||2 represents the L2 norm; For slot S j , considering that some slots can be inherited without updating relative to the previous round of dialogue state, while other slots need to be updated, the loss of whether to update or maintain is as follows: in, is the loss function for state transition prediction, whose value set is {keep, update}, W c , W p are the learnable parameters of the linear layer, is the update probability of slot s in the tth round, is the state transition label, when updated Keep Step B53: Train the model to maximize the joint probability of all slots, i.e. The loss function for each round t is defined as the accumulation of negative log-likelihoods: The final loss is: Step B53: Update the learning rate of the loss calculated in B52 through the stochastic gradient descent algorithm, and use back propagation to iteratively update the model parameters to minimize the loss function to train the model.

9. A dialogue state tracking system based on AMR image information enhancement, characterized in that: The method comprises a memory, a processor, and computer program instructions stored in the memory and executable by the processor. When the processor executes the computer program instructions, the method steps as described in any one of claims 1 to 8 can be implemented.

Images