A Dialogue Act Recognition Method Based on Multi-Level Graph Attention

Abstract

The present invention provides a method for dialogue act recognition based on multi-level graph attention, which relates to the technical field of natural language processing. The method first constructs a corpus dataset and performs sentence-level semantic encoding on the sentences in the corpus dataset; then performs context-aware semantic encoding on the sentences in the dialogue; then constructs a local context graph of consecutive dialogues within a window; performs multi-level semantic feature encoding based on the attention mechanism on the local context graph within the window, and fuses the final hidden-layer semantic features for classification output at each level; and based on the hidden-layer semantic features, uses a classifier to classify the dialogue; finally, calculates the loss between the predicted action label and the true action label using a loss function, and updates each weight. This method takes into account the mutual connection between speakers and analyzes the connection between the utterances of speakers; constructs context-level feature selection based on a gated graph convolutional network for selecting prominent context information for DAs prediction.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and in particular, to a dialogue act recognition method based on multi-level graph attention. Background Art

[0002] Dialogue Acts (DA) are semantic markers of utterances and are crucial for understanding communication. Dialogue act classification is to assign one or more dialogue acts to each dialogue sentence, thereby representing the speaker's intention, which is closely related to people's lives. Therefore, more and more scholars have carried out research on dialogue act classification tasks. Context information provides key information for better explaining the dialogue. Therefore, predicting DAs requires being sensitive to the context. Deep learning methods have attracted extensive attention from researchers because they usually simulate the dependencies between adjacent utterances and have achieved good results. In terms of prediction, the existing technology combines RNN and attention mechanisms to further improve the prediction of DAs. In terms of classification, the existing technology regards DAs classification as a sequence-to-sequence translation task and uses Conditional Random Field (CRF) to model the sequential dependencies of DAs. In terms of speaker modeling, the existing technology uses graph neural networks to separately model speaker-specific utterances. In addition, graph neural networks are also used to capture context information in the dialogue.

[0003] However, these studies discuss the ability of the model to capture global or speaker-internal context information, while the impact of connections between speakers has not been well studied. There is also research on hierarchical attention in data mining tasks, but the existing work mainly focuses on words and local information as well as context information (at the word and sentence levels, sentence and dialogue, i.e., semantic connections). Few studies discuss the sentence and context levels by considering the connections between speakers. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a dialogue act recognition method based on multi-level graph attention for the deficiencies of the above-mentioned existing technologies, so as to consider the mutual connections between contexts and the semantics carried by the sentence itself.

[0005] To solve the above technical problem, the technical solution adopted by the present invention is: A dialogue act recognition method based on multi-level graph attention, comprising the following steps:

[0006] Step 1: Construct a corpus dataset;

[0007] Constructing a corpus dataset Where Nc is the number of conversations in the dataset, is a dialogue data composed of a series of statements, where u represents each specific utterance, and C m tag set of y represents the dialogue act behind each sentence, and n cm is the dialogue C m the number of statements in u i represents the composition of the dialogue C m the i-th statement, i = 1, …, n cm ;

[0008] Step 2: Perform sentence-level semantic encoding on the statements in the corpus dataset:

[0009] Use a pre-trained language model to encode the words in each statement into vectors, where represents the semantic vectors of each word in the i-th statement u i and represents the word w k in the i-th statement u i the dimension of the semantic vector, is the number of words in the i-th statement u i ; then perform mean pooling on the semantic vectors of all words in the statement to obtain the sentence-level semantic encoding of the statement;

[0010] Step 3: Perform context-aware semantic encoding on the statements in the dialogue;

[0011] After obtaining the sentence-level semantic encoding of each statement, use a fixed window to select a continuous dialogue for context-aware encoding, and the length of the window is n win ; secondly, use a bidirectional long short-term memory network to capture the context information within the window to obtain the context-aware semantic encoding features of the selected continuous dialogue within the window;

[0012] Step 4: Construct a local context graph for the continuous dialogue within the window;

[0013] After obtaining the context-aware semantic encoding features, consider N node nodes around the sentence for which the dialogue act needs to be predicted currently. There are p past utterances and f future utterances within the window, and construct a directed graph N node = p + f; where each statement is regarded as a node of the directed graph, and the node edge represents the connection between node v i and v j , j = [i - p, …, i + f], represents the type of relationship between nodes, t ∈ [0, 7] is the relationship index, represents the edge The weight value, with a value range of 0 ≤ α ij ≤ 1;

[0014] The connection weight α ij is obtained by designing a learnable adjacency matrix using the softmax function; First, define 8 types of connection methods between contexts, that is, the relationship types between nodes, to initialize the adjacency matrix. Specifically, the forward and backward directed connections of the speaker himself, the forward and backward directed connections of the interlocutor himself, the forward and backward directed connections from the speaker to the interlocutor, and the forward and backward directed connections from the interlocutor to the speaker; In the initialized adjacency matrix, the connection relationship between each two nodes is first filled with semantic similarity, and then the softmax function is used to process the similarity values of all nodes from node i to node i + f to node i - p into connection weight values with a sum of 1;

[0015] Step 5: Perform multi-level semantic feature encoding on the local context graph within the window based on the attention mechanism, and fuse the final hidden layer semantic features for classification output at each level;

[0016] Use the Relational Graph Convolutional Network (RGCN) and Graph Attention Network (GAT) to aggregate the dependencies at the speaker level; Use the Gated Graph Convolutional Network (GatedGCN) to update the semantic features at the context level, emphasizing the significant context semantic information that helps classification;

[0017] For aggregating the dependencies at the speaker level, first use RGCN to consider the relationship between the speaker and the local context information within the window to obtain the semantic features with speaker dependencies As shown in the following formula:

[0018]

[0019] where s i and s j are the semantic encodings of the i-th and j-th sentences in the conversation respectively, and are both learnable weight matrices, α ij and α ii are the weight values of edges r ij and edge r ii respectively, is the neighborhood index of node i under relationship r t , and σ is the activation function;

[0020] Semantic correlation features based on speaker dependencies Then use GAT to update the weight α ij , and then use the updated weight to update the semantic features based on speaker dependencies again:

[0021]

[0022]

[0023] where α′ ij is the updated weight, is the updated speaker-dependent semantic-related feature, is the learnable weight matrix, a is the parameterized weight vector, is the speaker-dependent semantic feature of sentence j that is relevant to sentence i; LeakyReLU is a neural network activation function;

[0024] For the update of context-level semantic features, a Gated Graph Convolutional Network (GatedGCN) is used to select significant sentences that contribute to classification, as shown in the following formula:

[0025]

[0026] where g i is the updated context semantic feature, W cg1 , W cg2 and W cg3 are learnable weight matrices, ⊙ represents element-wise inner product, and η ij serves as the gate parameter on the connection weight between nodes i and j;

[0027] Finally, the speaker-dependent semantic feature and the context-level semantic feature are connected for feature fusion to obtain the hidden-layer semantic feature of node i

[0028] Step 6: Based on the hidden-layer semantic feature in Step 5, use a classifier to classify the conversation;

[0029] By obtaining the context semantic feature and the speaker-related semantic feature in Step 5 and concatenating them to get the hidden-layer semantic feature , the final process of dialogue act recognition is carried out; the recognition process is completed by a classifier, and the classifier consists of 2 fully connected layers to convert the hidden-layer semantic feature into an action label, as shown in the following formula:

[0030]

[0031]

[0032] where z′ i is the hidden-layer semantic feature after dimensionality reduction, p i is the predicted dialogue act label of sentence i, and are learnable weight matrices, and All are deviations;

[0033] Step 7: Iteratively execute Steps 1-6, calculate the loss between the predicted action label and the true action label using the loss function, and update the weights in Steps 1-6 until the set termination condition is reached.

[0034] During the iteration process, the loss function is set to the categorical cross-entropy with L2 regularization, as shown in the following formula:

[0035]

[0036] where, P mi is the predicted dialogue action label for the i-th utterance in dialogue m, y mi is the true dialogue action label, λ is the weight of L2 regularization, and θ represents all trainable parameters.

[0037] The beneficial effects of adopting the above technical solution are as follows: A dialogue action recognition method based on multi-level graph attention provided by the present invention, 1) proposes a multi-level attention dialogue action recognition model based on a graph network; 2) considers the mutual connection between speakers, designs speaker-level feature transformation, and analyzes the connection between speakers' utterances; 3) in order to better utilize context information, constructs context-level feature selection based on a gated graph convolutional network for selecting prominent context information for DAs prediction. The method of the present invention can be applied to many fields, such as daily conversations, language learning, empathy responses, psychological counseling, and hot topic discussions, etc. Brief Description of the Drawings

[0038] Figure 1 is a flowchart of a dialogue action recognition method based on multi-level graph attention provided by an embodiment of the present invention;

[0039] Figure 2 is 8 kinds of initialized relationship graphs of adjacency matrices provided by an embodiment of the present invention, where p1 and p2 respectively represent Speaker 1 and Speaker 2 in a two-way dialogue;

[0040] Figure 3 is the visual recognition result on a Japanese dialogue action dataset provided by an embodiment of the present invention. Detailed Embodiment

[0041] The following combines the drawings and embodiments to further describe in detail the specific embodiments of the present invention. The following embodiments are used to illustrate the present invention, but are not used to limit the scope of the present invention.

[0042] In this embodiment, a dialogue action recognition method based on multi-level graph attention, as Figure 1 shown, includes the following steps:

[0043] Step 1: Construct a corpus dataset;

[0044] Construct a corpus dataset where Nc is the number of conversations in the dataset, is a conversation data composed of a series of statements, u represents each specific utterance, and C m the tag set of y represents the conversation act behind each sentence (such as objective information giving, subjective information giving, asking, agreeing, approving, thanking, etc.), and n cm is the conversation C m the number of statements in, u i represents the composition of the conversation C m the i-th statement, i = 1,..., n cm ;

[0045] Step 2: Perform sentence-level semantic encoding on the statements in the corpus dataset:

[0046] Use a pre-trained language model to encode the words in each statement into vectors, where represents the semantic vectors of each word in the i-th statement u i in, represents the word w k in the i-th statement u i the dimension of the semantic vector in, is the number of words in the i-th statement u i ; then perform mean pooling on the semantic vectors of all words in the statement to obtain the sentence-level semantic encoding of the statement;

[0047] Step 3: Perform context-aware semantic encoding on the statements in the conversation;

[0048] After obtaining the sentence-level semantic encoding of each statement, use a fixed window to select a continuous conversation for context-aware encoding, and the length of the window is n win ; secondly, use a bidirectional long short-term memory network (Bi-LSTM) to capture the context information within the window to obtain the context-aware semantic encoding features of the selected continuous conversation within the window;

[0049] Step 4: Construct a local context graph for the continuous conversation within the window;

[0050] After obtaining the context-aware semantic encoding features, consider the N node nodes around the sentence for which the conversation act needs to be predicted currently. There are p past utterances and f future utterances within the window, and construct a directed graph N node= p + f; where each statement is regarded as a node of a directed graph, and the node edge represents the connection between nodes v i and v j The connection between them is j = [i - p, …, i + f], represents the type of relationship between nodes, t ∈ [0, 7] is the relationship index, represents the edge The weight of is, and the value range is 0 ≤ α ij ≤ 1;

[0051] Use the softmax function to design a learnable adjacency matrix to obtain the connection weight α ij , instead of using a fixed connection with a self-attention process; first, define 8 connection methods between contexts, that is, the types of relationships between nodes, to initialize the adjacency matrix, as Figure 2 shown, that is, the forward and backward directed connections of the speaker himself, the forward and backward directed connections of the interlocutor himself, the forward and backward directed connections from the speaker to the interlocutor, and the forward and backward directed connections from the interlocutor to the speaker; in the initialized adjacency matrix, the connection relationship between each two nodes is first filled with semantic similarity, and then the softmax function is used to process the similarity values of all nodes from node i and nodes i - p to i + f into connection weight values with a sum of 1, so as to emphasize the context information most relevant to the current node;

[0052] Step 5: Perform multi-level semantic feature encoding on the local context graph within the window based on the attention mechanism, and fuse the final hidden layer semantic features for classification output from each level;

[0053] The semantic encoding in the local context graph within the window constructed in Step 4 is initially independent of the speaker, that is, it does not contain the semantic features perceived by the speaker; after constructing the context graph for the conversation, the present invention uses a relational graph convolutional network (RGCN) and a graph attention network (GAT) to aggregate the dependencies at the speaker level; a gated graph convolutional network (GatedGCN) is used to update the semantic features at the context level, emphasizing the significant context semantic information helpful for classification;

[0054] For aggregating the dependencies at the speaker level, first use RGCN to consider the relationship between the speaker and the local context information within the window to obtain the semantic features with speaker dependencies as shown in the following formula:

[0055]

[0056] where, s i and s jare the semantic encodings of the i-th and j-th sentences in the conversation, respectively, and are both learnable weight matrices, α ij and α ii are the weights of edges r ij and edge r ii respectively, is the neighborhood index of node i under relationship r t , and σ is the activation function;

[0057] Semantic correlation features based on speaker dependence Then use GAT to update the weight α ij , and then use the updated weight to update the semantic features based on the speaker dependence again:

[0058]

[0059]

[0060] where α′ ij is the updated weight, is the updated semantic correlation feature with speaker dependence, is the learnable weight matrix, a is the parameterized weight vector, is the semantic feature with speaker dependence of sentence j that is relevant to sentence i; LeakyReLU is a neural network activation function that can effectively improve the "gradient disappearance" problem in deep neural networks, thereby improving the accuracy and precision of the model;

[0061] For the update of context-level semantic features, use the Gated Graph Convolutional Network (GatedGCN) to select significant sentences that contribute to classification, as shown in the following formula:

[0062]

[0063] η ij =σ(W cg2 s i +w cg3 s j )

[0064] where g i is the updated context semantic feature, W cg1 , W cg2 and W cg3 are learnable weight matrices, ⊙ represents element-wise inner product, and η ij serves as the gate parameter on the connection weight between nodes i and j;

[0065] Finally, connect the speaker-dependent semantic features and context-level semantic features for feature fusion to obtain the hidden-layer semantic features of node i

[0066] Step 6: Based on the hidden-layer semantic features in Step 5, use a classifier to classify the dialogue;

[0067] Through Step 5, the context semantic features and speaker-related semantic features are obtained, and they are concatenated to obtain the hidden-layer semantic features After that, perform the final dialogue act recognition process; the recognition process is completed by a classifier, which consists of 2 fully connected layers to convert the hidden-layer semantic features into act labels, as shown in the following formula:

[0068]

[0069]

[0070] where, z′ i is the hidden-layer semantic feature after dimensionality reduction, p i is the predicted dialogue act label of utterance i, and are learnable weight matrices, and are both biases;

[0071] Step 7: Calculate the loss in the process of Steps 1-6 to provide a direction for updating each weight; Iteratively execute Steps 1-6, and use the loss function to calculate the loss between the predicted act label and the true act label, and update the weights in Steps 1-6 until the set iteration termination condition is reached. In this embodiment, the iteration termination condition is until the loss converges or the maximum number of loops is reached.

[0072] During the iteration process, the loss function is set to the classification cross-entropy with L2 regularization, as shown in the following formula:

[0073]

[0074] where, P mi is the predicted dialogue act label of utterance i in dialogue m, y mi is the true dialogue act label, λ is the weight of L2 regularization, and θ represents all trainable parameters.

[0075] In this embodiment, the method of the present invention is evaluated on the Japanese dialogue act set (JPS-DA). This data set contains 53 interview dialogues. Therefore, Nc = 53 in step 1. The interview dialogue content discusses the past, present, and future of the participants. Each dialogue is about 30 minutes. The dialogues in this embodiment are binary dialogues, that is, there are two speakers p1 and p2. The total number of sentences is 20942, which is divided into 16 categories. Dialogues 1-39 are used as the training set, the last 6 dialogues are used as the test set, and the rest are used as the validation set. In this embodiment, the latest available pre-trained model BERT is used to convert Japanese sentences into sentence embeddings. Finally, a 768-dimensional vector is obtained as the context-independent representation of the sentence. The length of the window is set to n win is 30, s i The dimension of is set to 512, The dimension of is set to 384, and g i The dimension of is set to 200. The number of past utterances p and future utterances f is equal to 1. The application rate of the exit mechanism is 0.5. The learning rate is set to le-4, and the decay weight is set to le-8. In this embodiment, the number of iterative training epochs is set to 100, and the batch size is 32. In this embodiment, the method of the present invention is adopted, and the recognition results of this Japanese dialogue act data set are as Figure 3 shown.

[0076] In this embodiment, the accuracy rate and F1 value are used as evaluation indicators. When the number of past and future nodes is 1, the highest recognition accuracy rate reaches 72.37%, and the highest F1 score reaches 71.38%. The experimental results show that the method of the present invention takes into account the mutual connection between speakers and the importance of selecting significant context sentences for the dialogue act recognition task.

[0077] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope defined by the claims of the present invention.

Claims

1. A method for recognizing dialogue behaviors based on multi-level graph attention, characterized in that: It includes the following steps: Step 1: Construct a corpus dataset; Step 2: Perform sentence-level semantic encoding on the sentences in the corpus dataset; Step 3: Perform context-aware semantic encoding on the sentences in the dialogue; After obtaining the semantic encoding at the sentence level of each sentence, use a fixed window to select a continuous conversation for context-aware encoding, and the length of the window is n win Secondly, use a bidirectional long short-term memory network to capture the context information within the window, and obtain the context-aware semantic encoding features of the selected continuous conversation in the window; Step 4: Construct a local context graph for consecutive dialogues within the window; After obtaining the context-aware semantic encoding features, consider N nodes around the sentence for which the dialogue act needs to be predicted currently. node There are p past utterances and f future utterances within the window, and a directed graph is constructed. N node = p + f; where each statement is regarded as a node of the directed graph, and the node edge represents the connection between node v i and v j and the relationship type between nodes is represented by , t ∈ [0, 7] is the relationship index, and α ij ∈ w represents the weight of the edge , and the value range is 0 ≤ α ij ≤ 1;​​ Connection weight α ij It is obtained by designing a learnable adjacency matrix using the softmax function; first, define 8 connection methods between contexts, that is, the relationship types between nodes, to initialize the adjacency matrix, namely, the forward and backward directed connections of the speaker himself, the forward and backward directed connections of the interlocutor himself, the forward and backward directed connections from the speaker to the interlocutor, and the forward and backward directed connections from the interlocutor to the speaker; in the initialized adjacency matrix, the connection relationship between each two nodes is first filled with semantic similarity, and then the softmax function is used to process the similarity values of all nodes from node i and node i-p to node i+f into connection weight values with a sum of 1; Step 5: Perform multi-level semantic feature encoding on the local context graph within the window based on the attention mechanism, and fuse the final hidden semantic features for classification output at each level; Use the Relational Graph Convolutional Network (RGCN) and Graph Attention Network (GAT) to aggregate speaker-level dependencies; use the Gated Graph Convolutional Network (GatedGCN) to update semantic features at the context level, emphasizing significant context semantic information helpful for classification; For aggregating speaker-level dependencies, first use RGCN to consider the relationship between the speaker and the local context information within the window, and obtain semantic features with speaker dependencies As shown in the following formula: where s i and s j are the semantic encodings of the i-th and j-th statements in the dialogue respectively, and are both learnable weight matrices, α ij and α ii are the weights of edges r ij and edge r ii respectively, is the neighborhood index of node i under the relationship r t , and σ is the activation function; Semantic-related features based on speaker dependencies Then use GAT to update the weight α ij , and then use the updated weight to update the semantic features again based on speaker dependencies: where α′ ij is the updated weight, is the updated speaker-dependent semantic-related feature, is the learnable weight matrix, a is the parameterized weight vector, is the speaker-dependent semantic feature of statement j that is relevant to statement i; LeakyReLU is a neural network activation function; For the semantic feature update at the context level, use the Gated Graph Convolutional Network (GatedGCN) to select significant sentences helpful for classification, as shown in the following formula: η ij = σ(W cg2 s i + W cg3 s j ) Among them, g i is the updated context semantic feature, W cg1 , W cg2 and W cg3 are learnable weight matrices, ⊙ represents element-wise inner product, and η ij serves as the gate parameter on the connection weight between nodes i and j; Finally, connect the speaker-dependent semantic features and the context-level semantic features for feature fusion to obtain the hidden-layer semantic features of node i Step 6: Based on the hidden semantic features in Step 5, use a classifier to classify the dialogue; Step 7: Iteratively execute Steps 1-6, and use a loss function to calculate the loss between the predicted action label and the true action label, and update the weights in Steps 1-6 until the set termination condition is reached.

2. The method for identifying dialogue behaviors based on multi-level graph attention according to claim 1, wherein: The step 1 constructs a corpus dataset where Nc is the number of conversations in the dataset is the conversation data composed of a series of statements, u represents each specific utterance, and C m is the tag set of y represents the conversation act behind each sentence, and n cm is the conversation C m is the number of statements in the conversation, and u i represents the i-th statement that makes up the conversation C m , where i = 1, …, n cm .

3. The method for identifying dialogue behaviors based on multi-level graph attention according to claim 2, wherein: In step 2, a pre-trained language model is used to encode the words in each statement in the corpus dataset into vectors, where represents the i-th statement u i the semantic vectors of each word in represents the word w k in the i-th statement u i the dimension of the semantic vector is the i-th statement u i the number of words in; then mean pooling is performed on the semantic vectors of all words in the statement to obtain the sentence-level semantic encoding of the statement.

4. The method for recognizing dialogue behaviors based on multi-level graph attention according to claim 3, wherein: The specific method of Step 6 is as follows: After obtaining the context semantic features and speaker-related semantic features through Step 5 and concatenating them to obtain the hidden layer semantic features the process of performing the final dialogue act recognition is carried out; The recognition process is completed by a classifier, which consists of 2 fully connected layers to convert the hidden semantic features into action labels, as shown in the following formula: Among them, z′ i is the latent layer semantic feature after dimensionality reduction, p i is the predicted dialogue act label of utterance i, and are learnable weight matrices, and are both biases.

5. The method for recognizing dialogue behavior based on multi-level graph attention according to claim 4, characterized in that: In the iterative process of step 7, the loss function is set to categorical cross-entropy with L2 regularization, as shown in the following formula: Among them, P mi is the predicted dialogue act label of the i-th utterance in dialogue m, y mi is the true dialogue act label, λ is the weight of L2 regularization, and θ represents all trainable parameters.

Images