A multi-intent spoken language understanding method based on syntax analysis

Abstract

The application discloses a multi-intent spoken language understanding method based on syntax analysis, which comprises the following steps: firstly, obtaining an intent feature matrix and a slot feature matrix according to a user input sentence, and constructing a multi-level intent feature from the intent feature matrix; secondly, obtaining initial intent labels and initial slot prediction labels by using an intent decoding module and a slot decoding module respectively on the intent feature matrix and the slot feature matrix; then, inputting the multi-level intent feature and the initial slot prediction labels into a slot-intent interaction module to obtain an enhanced intent feature matrix, and inputting the slot feature matrix, the multi-level intent feature and the initial intent labels into an intent-slot interaction module to obtain an enhanced slot feature matrix; finally, inputting the enhanced intent feature matrix and the slot feature matrix into the intent decoding module and the slot decoding module respectively to obtain intent labels and slot sequence labels of the spoken language understanding task. The application improves the accuracy of intent recognition and slot sequence labeling and the accuracy of multi-intent recognition.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of natural language processing, intent recognition and slot filling, and particularly relates to a multi-intent spoken language understanding method based on syntax analysis. BACKGROUND

[0002] Spoken language understanding is a basic task in dialogue systems, and its goal is to capture the overall semantics of the user's utterance. This task usually includes two sub-tasks: intent recognition and slot filling. Intent recognition aims to analyze the semantic of user input to determine the intent contained behind it, and this task is usually regarded as a classification problem, which assigns a predetermined intent label to each user sentence to achieve accurate understanding of user needs. The purpose of the slot filling task is to extract structured information from the user's input sentence and map it to the attribute value of a specific domain keyword, which is essentially a sequence labeling task.

[0003] In real life, a user's sentence usually contains multiple intents, so it is increasingly important to understand multiple intents in a sentence. Most existing work mainly focuses on how to accurately identify the intent, and uses the predicted intent label to guide the slot filling task. And because of the close relationship between intent recognition and slot filling, a joint model is usually used to model the two tasks. However, most existing work focuses on using predicted intent labels to guide slot filling work, but ignores the fact that slot labels can also guide intent inference. Furthermore, existing research lacks the use of syntactic information of the sentence, which can help the model better capture multiple intents scattered in the sentence. At the same time, since a single intent in a sentence is usually expressed by a sub-sentence, the existing research usually uses word-level intent recognition methods and cannot well mine the hidden information of the sub-sentence. SUMMARY

[0004] In view of the problems in the prior art, the present application provides a multi-intent spoken language understanding method based on syntax analysis, which not only improves the accuracy of multi-intent recognition and the accuracy of slot prediction, but also effectively integrates syntax information and multi-level intent, and improves the overall performance and robustness of the model.

[0005] In order to solve the above technical problems, the technical scheme of the present application comprises the following steps:

[0006] S1: Obtain the user input sentence, use the input processing module proposed in the present application to obtain the intent feature matrix H I and the slot feature matrix H S for the intent recognition task and the slot filling task.

[0007] S2: According to the intent feature matrix H I, and the [CLS] token in the intent feature matrix H I obtained in S1 is taken as the sentence-level intent feature. I The word vector in the intent feature matrix H I obtained in S1 is taken as the word-level intent feature. The above sentence-level intent feature, window-level intent feature and word-level intent feature jointly constitute the multi-level intent feature.

[0008] S3: The intent feature matrix H I and the slot feature matrix H S obtained in S1 are respectively input into the intent decoding module and the slot decoding module to obtain the initial intent label and the initial slot prediction label.

[0009] S4: In the intent recognition task, the multi-level intent feature obtained in S2 and the initial slot prediction label obtained in S3 are input into the slot-intent interaction module to obtain the intent feature matrix H [I enhanced by the slot label. L] .

[0010] S5: In the slot filling task, the slot feature matrix obtained in S1, the multi-level intent feature obtained in S2 and the initial intent label obtained in S3 are input into the intent-slot interaction module to obtain the slot feature matrix H [S,L] enhanced by the intent label.

[0011] S6: The slot label enhanced intent feature matrix H [I,L] obtained in S4 is input into the intent decoding module, and the intent label enhanced slot feature matrix H [S,L] obtained in S5 is input into the slot decoding module, and then the two tasks are jointly trained to obtain the intent label and the slot sequence label required by the spoken language understanding task.

[0012] Further, the input processing module in S1 is as follows:

[0013] S101: Obtain the sentence input by the user, use the RoBert model as the encoder to produce the word vector matrix E = {e CLS ,e1,e2,...,e n}.

[0014] S102: Construct the syntax dependency tree for the sentence input by the user, encode the nodes of the dependency tree into word vectors using the RoBert model, and then use the graph attention mechanism to converge the semantic information to the root node of the dependency tree. Finally, the root node is taken as the syntax token [g], and this syntax token [g] is inserted after the [CLS] token in the input sequence E to construct the word vector matrix E' = {e CLS ,eg e1, e2,..., e n}.

[0015] S103: Perform forward and backward reading on the word vector matrix E' containing syntactic information obtained in S102 using two independent BiLSTM networks, respectively, to generate an intent feature matrix H I and a slot feature matrix H S for the intent recognition task and the slot filling task.

[0016] Further, the decoder module in step S3 includes two parts:

[0017] S301: Based on the intent feature matrix H I obtained in S103, use the sigmoid function to obtain the distribution of each word intent label, and use to perform word-level intent voting to obtain the preliminary sentence-level intent label.

[0018] S302: Based on the slot feature matrix H S obtained in S103, use the softmax classifier to obtain the slot label distribution of each word, and use the argmax function to obtain the preliminary slot prediction label.

[0019] Further, the slot-intent interaction module in step S4 includes:

[0020] S401: Use the gating mechanism to fuse the multi-level intent features obtained in S2 in the multi-level intent fusion layer to obtain the fused intent feature matrix H

[0021] S402: Use the intent feature matrix H IF , the intent feature matrix H IF in S401, and the syntax token in S3 to construct the intent semantic label graph, and strengthen the expression of the intent feature through the graph attention mechanism, to obtain the intent feature matrix H [I,L] enhanced using the syntax token and the slot label.

[0022] Further, the intent-slot interaction module in step S5 includes:

[0023] S501: Use the BiLSTM model to model the temporal dependency between the multi-level intent features obtained in S2 and the slot feature matrix obtained in S1 in the multi-level intent perception layer, and use the gating mechanism to fuse the multi-level intent enhanced slot features to obtain the fused slot feature matrix H

[0024] S502: using the slot feature matrix H obtained in S501 SF , slot feature matrix H SF in syntax tokens The initial intent label obtained in S3 constructs a slot semantic label graph, and the expression of the slot feature is strengthened through a graph attention mechanism, to obtain a slot feature matrix H enhanced using syntax tokens and intent labels [S,L] .

[0025] The present application has the following characteristics and benefits:

[0026] The present application proposes a new perspective to improve the spoken language understanding ability of the language model by fusing syntax information. The present application introduces a syntax dependency tree to enrich semantic information, and effectively utilizes the node information and edge information of the dependency tree to further mine the potential semantic relationship between words, thereby further improving the accuracy of intent recognition and slot sequence labeling.

[0027] The present application effectively makes up for the part of the slot label that is missing in the traditional method to guide the multi-intent recognition task, models the bidirectional interactive intent recognition and slot filling model, so that the multi-intent detection task and the slot filling task share basic semantics and guide each other, thereby improving the accuracy of multi-intent recognition.

[0028] The present application breaks through the inherent limitations of traditional methods and provides a method of fusing multi-level intent information in the slot filling task, which fully considers different levels of intent information: the discourse-level intent provides global semantic information, which can effectively reduce the information loss in the entire sentence; the word-level intent provides more fine-grained interaction, which can capture the intent at the word level and understand the semantics and target of the sentence in more detail; the window-level intent identifies the intent according to continuous fragments, which fully utilizes the relationship between the front and rear intents. Compared with the single discourse granularity method, fusing multi-level intent can provide better multi-intent recognition and slot prediction performance. BRIEF DESCRIPTION OF DRAWINGS

[0029] Figure 1 is a flowchart of the present application;

[0030] Figure 2 is a framework schematic diagram of the present application;

[0031] Figure 3 is a syntax information extraction module schematic diagram proposed by the present application;

[0032] Figure 4 is a semantic label graph for guiding intent recognition by slot labels proposed by the present application;

[0033] Figure 5 is a semantic label graph for guiding slot filling by intent labels proposed by the present application. DETAILED DESCRIPTION

[0034] In order to make the purposes, technical solutions and technical effects of the present application clearer, the following further describes in detail the multi-intent spoken language understanding method based on syntax analysis provided by the present application in combination with examples and the accompanying drawings.

[0035] The present application provides a multi-intent spoken language understanding method based on syntax analysis, as shown in the following steps: Figures 1-2

[0036] S1: using a semantic encoding module to perform text preprocessing on the user's utterance, obtaining an intent feature vector and a slot feature vector.

[0037] The preliminary processing and feature extraction of the data set data specifically include the following steps:

[0038] S101: using a word segmenter to segment the user's utterance, and using a RoBert model as an encoder to produce a word vector matrix E={e CLS ,e1,e2,...,e n} containing basic semantics.

[0039] Wherein e t (t∈1,2,...n) represents the vector representation of the tth word in the sentence.

[0040] S102: extracting syntax tokens [g] containing syntax information from the user's utterance through a syntax extraction module, and inserting them after the [CLS] token in the input sequence E, to construct an embedded representation E'={e CLS ,e g ,e1,e2,...,e n} that fuses syntax information.

[0041] S103: using two different BiLSTMs to read the input vector E' forward and backward, respectively generating context-sensitive feature vectors for specific tasks:

[0042]

[0043] Wherein represents the feature vector of the tth position in the intent detection task, represents the feature vector of the tth position in the slot filling task.

[0044] S2: the traditional intent recognition task usually directly uses word vectors for intent reasoning, but a single intent in a multi-intent utterance is usually composed of sub-sentences. In order to capture the sub-sentence information representing a single intent, the present method uses a multi-level intent construction method to capture deeper information, including sentence-level intent, window-level intent and word-level intent.​

[0045] The specific construction method is as follows:

[0046] S201: The sliding window self-attention mechanism is used to capture window-specific context information. A window with a size of win is used to slide on the sentence, and the intent feature matrix in the window is The self-attention calculation is performed to obtain the window-level intent feature matrix

[0047]

[0048] where w is the window number, size is the window size, is the input matrix in the sliding window, Q (query), K (key), and V (value) are matrices obtained by mapping through different linear parameters, A is the matrix obtained by self-attention calculation. t ={a1,...,a i ,...,a size} is the matrix obtained by self-attention calculation.

[0049] Further, the window-level intent feature can be represented as:

[0050]

[0051] where L represents the length of the sentence.

[0052] S202: The [CLS] token in the intent feature matrix H I is regarded as the sentence-level intent feature:

[0053]

[0054] S203: The feature of the intent recognition task is the word-level intent feature:

[0055]

[0056] S3: The intent feature and the slot feature are input into the intent decoding module and the slot decoding module, respectively, to obtain the initial intent label IL and the initial slot prediction label SL.

[0057] The specific steps include the following:

[0058] S301: The intent feature ​The intent distribution label of the t-th word input into the intent decoder can be obtained by the following formula:

[0059]

[0060] wherein are model training parameters, is the intent label corresponding to the t-th word.

[0061] Further, the intent label obtained by decoding each word element is voted: the intent prediction result at the word level is converted into a binary form, and greater than 0.5 is considered as a positive class, and less than or equal to 0.5 is considered as a negative class. Then, for each intent k, the sum of the number of word elements satisfying I ( t , k ) >0.5 is calculated, the votes of the word elements for the k-th intent are counted, and the intent with a vote number greater than n / 2 is added to the set of sentence intent results:

[0062]

[0063] wherein o I represents the set of sentence intent results, is an element in the set o I , represents a specific sentence intent, I ( i , k ) represents the classification result of the t-th word element for the k-th intent, and n represents the number of word elements.

[0064] S302: input the slot feature into a fully connected softmax layer to generate a slot label distribution for each word, and estimate a slot label for each word.

[0065]

[0066] wherein are model training parameters, is the slot label distribution corresponding to the t-th word, is the initial estimated slot label.

[0067] S4: fuse multi-level intent features using a multi-level intent fusion layer, and then use the intent semantic label graph to guide the intent label decoding using the preliminary estimated slot label. The existing method usually ignores the guidance of the slot label to the intent recognition. The present application fully excavates the semantic information of the slot label by constructing a slot label-enhanced graph attention mechanism for intent features, thereby improving the performance and robustness of the model.

[0068] The specific steps include the following:

[0069] S401: Obtain the sentence-level intent feature in S2 window-level intent feature word-level intent feature Fusion through a gating mechanism:

[0070] g t = σ(Wh t +b)

[0071]

[0072] where W, b are trainable variables, g t is a gating parameter controlling how many features of each granularity are kept, v is the intent-level feature after fusion.

[0073] S402: Project the slot labels {SL1, SL2,..., SL n} obtained in S302 into vector form using the slot label embedding matrix, to obtain the representation of the slot label corresponding to each word

[0074] S403: Input the fused intent feature and E sl into the intent semantic label graph, so that the slot semantic information can be fully utilized when performing intent recognition:

[0075] H [I,L] =GAT Intent ([H IF ,E sl ], G Intent , θ I )

[0076] where [H IF ,E sl ] is the input node representation, G Intent is the intent semantic label graph, θ I represents the parameters of the graph attention network, and H [I,L] is the intent feature enhanced by the L-layer graph attention network.

[0077] S5: Model and fuse the multi-level intent features and slot features using the multi-level intent perception layer, and then use the slot semantic label graph to guide the decoding of the slot labels based on the preliminary estimated intent labels. The present application also follows the existing research to guide the prediction of slot labels based on the preliminary estimated intent labels.

[0078] Specifically, the following steps are included:

[0079] S501: Since the B-I-O label of the slot tag has time sequence dependence, the application uses a BiLSTM network, introduces multi-level intent features, and models the time sequence dependence between slot features:

[0080]

[0081] wherein represents the slot feature after fusing the v-level intent information at the t position, represents the intent features of each level obtained in S2, and is a splicing operation.

[0082] Then the obtained slot features of each level are fused through a gating mechanism:

[0083]

[0084] wherein represents the slot feature representation obtained after fusing the multi-level intent.

[0085] S502: The intent label o I ={IL1,IL2,...,IL n} preliminarily recognized in S301 is projected into a vector form by using an intent label embedding matrix, to obtain the representation of the intent label corresponding to each word

[0086] S503: The fused slot feature and E il are input into the slot semantic label graph, so that the intent semantic information can be fully utilized when decoding the slot:

[0087] H [S,L] =GAT Slot ([H SF ,E il ],G Slot ,θ S )

[0088] wherein [H SF ,E il ] is the input node representation, G Slot is the slot semantic label graph, θ S represents the parameters of the graph attention network, and H [S,L] is the slot feature enhanced by the L-layer graph attention network.

[0089] S6: The intent feature H [I,L] obtained in S403 is input into the intent decoder by applying the method in S301, to obtain the final sentence-level intent label; the slot feature H [S,L]The input is input into the slot decoder to obtain the final output slot label. The present application adopts a joint training method to update the parameters of the two tasks of intent recognition and slot filling. The loss function of each task can be represented as:

[0090]

[0091] Wherein, L intent and L slot are the loss functions of the intent recognition and slot filling tasks, are the true labels of the intent and semantic slot respectively, N I is the number of single intent labels within a sentence, N S is the number of slot labels, and n is the number of words in a sentence.

[0092] Then the two tasks are jointly optimized by a joint loss function, which can be represented as:

[0093] L=αL intent +(1-α)L slot

[0094] Wherein, α is a hyperparameter.

[0095] Figure 3 The syntax information extraction module based on the syntax dependency tree proposed by the present application comprises:

[0096] Step one: use the en_core_web_sm model of spaCy to construct a syntax dependency tree for user input. The dependency tree can be represented by a graph G with n nodes, where each leaf node represents a word in the sentence. The edges of G represent the dependency relationship between the words, and the adjacent nodes of node i can be represented by . Then use the RoBert model to encode the leaf nodes of the dependency tree to obtain the feature h i of the leaf node i.

[0097] Step two: use the multi-head attention mechanism to aggregate the neighborhood node information:

[0098]

[0099] Wherein is the feature representation of the (l+1)th layer of node i, is the attention coefficient of the lth layer node i and node l, α ij represents the normalized attention weight of node j to node i at the lth layer, and || is the concatenation operation.

[0100] Step three: after applying the graph attention mechanism, the feature h rootAs a syntax token [g], and insert the syntax token [g] into the original embedding sequence E' = {e CLS ,e g ,e1,e2,...,e n}:

[0101] e g = h root

[0102] Figure 4 The schematic diagram of the intent semantic label graph proposed by the present application, which includes three nodes and five edges.

[0103] Among them, the three nodes are: syntax information node I g , intent feature node I t , and slot label mapping node SL t .

[0104] Among them, the five edges are: the edge of the syntax node I g pointing to the intent feature I t , the edge of the intent feature I i pointing to the intent feature I j , the edge of the intent feature I pointing to the slot SL, the edge of the slot label SL pointing to the intent feature I, and the edge of the slot label SL i pointing to the slot label SL j .

[0105] In order to match the window level intent capture interval, the I t node and the SL t node are connected based on the local window, taking the intent feature node I t as an example, which is connected with {I t-size ,...,I t+size} and {SL t-size ,...,SL t+size}. Among them, size is the window size, which is consistent with the sliding window size in S201.

[0106] Then calculate its multi-head attention feature:

[0107]

[0108] Among them, K represents the total number of heads, d is the feature dimension, and || is the concatenation operation.

[0109] Among them, represents the node set pointing to node i on the intent semantic label graph, represents the node set pointing to node i through the edge of type r in the semantic label graph, represents the edge of node j pointing to node i.

[0110] Among them is the weight matrix of the kth head with the edge type r.

[0111] Figure 5 The schematic diagram of the slot semantic label graph is proposed for the application, including 3 kinds of nodes and 5 kinds of edges:

[0112] The 3 kinds of nodes are respectively: syntax information node S g , slot feature node S t , and intent label mapping node IL t .

[0113] The 5 kinds of edges are respectively: the edge of the syntax node S g pointing to the slot feature S t , the edge of the slot label S i pointing to the intent feature S j , the edge of the slot feature S pointing to the intent label IL, the edge of the intent label IL pointing to the slot feature S, and the edge of the intent label IL i pointing to the intent label IL j .

[0114] The types of the nodes and the edges, and the calculation manner of the graph attention are consistent. Figure 4 Since the intent is at the sentence level, that is, the intent label can guide the prediction of all slot positions, when the graph is constructed, each intent label IL node is globally connected with all slot feature S t nodes. For the slot feature node S t , it is also connected in a local window manner, and the S t node is connected with {S t-size ,...,S t+size} and {IL1,...,IL m}. Wherein, size is the window size, which is consistent with the sliding window size in S201; and m is the number of initial predicted intents in S301.

[0115] According to the model training method, the training parameters of the embodiment of the application on the MixATIS dataset are as follows: the dimensions of the word embedding, the label embedding and the hidden state are all 256, the sliding window size of the window-level intent is 3, the number of all graph neural network layers is 2, the training is performed through the Adam optimizer, the batch size is 16, and the learning rate is 0.001.

[0116] The test results of the method of the present application on the MixATIS public dataset and comparison with other models are shown in Table 1. It should be noted that: the intent accuracy is used to measure the proportion of correct prediction of the model in the intent recognition task; the slot filling F1 score comprehensively considers the precision and recall of the slot, and is used to evaluate the comprehensive performance of the model in the slot filling task; the overall accuracy measures the proportion of samples that are correctly predicted by the model in both intent and all slots, i.e. only when the intent and all corresponding slots of a sample are correctly predicted is it considered as a positive example.

[0117] Table 1

[0118]

[0119] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; obviously, the drawings are only some examples or embodiments of the present application, and for those skilled in the art, the present application can also be applied to other similar situations according to these drawings, but without the need for creative labor. In addition, it can be understood that, although the work done in the development process may be complex and long, for those skilled in the art, some design, manufacture or production changes according to the technical content disclosed in the present application are only routine technical means, and should not be regarded as insufficient disclosure of the present application.

[0120] Although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: the technical solutions recorded in the foregoing embodiments can still be modified, or some technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1.A method for multi-intent spoken language understanding based on syntax analysis, characterized in that, The method comprises the following steps: S1: Obtain the sentence input by the user, obtain the intent feature matrix H for the intent recognition task through the input processing module I and the slot feature matrix H of the slot filling task S ; The input processing module specifically realizes the process as follows: S101: Obtain the sentence input by the user, use the RoBert model as the encoder to produce a word vector matrix E = {e CLS ,e1,e2,...,e n} containing semantics; S102: Construct a syntax dependency tree for the user input sentence, encode the nodes of the dependency tree into word vectors using the RoBert model, and use the graph attention mechanism to aggregate semantic information to the root node of the dependency tree. Finally, take the root node as the syntax token [g], and insert the syntax token [g] after the [CLS] token of the input sequence E to construct a word vector matrix E' containing syntax information = {e CLS ,e g ,e1,e2,...,e n}; S103: Perform forward and backward reading on the word vector matrix E' using two independent BiLSTM networks, respectively generate context-sensitive intent feature matrix H for the intent recognition task I and slot-filled slot feature matrix H S ; S2: obtaining the window-level intent feature, the sentence-level intent feature and the word-level intent feature according to the intent feature matrix H I The window-level intent feature, the sentence-level intent feature and the word-level intent feature are obtained respectively to form the multi-level intent feature, and the specific implementation process is as follows: According to the obtained intention feature matrix H I , a window self-attention mechanism is used to construct window-level intention features, and a [CLS] token in the intention feature matrix H I is taken as a sentence-level intention feature, and a word vector in the obtained intention feature matrix H I is a word-level intention feature; The sentence-level intent feature, the window-level intent feature and the word-level intent feature are combined to form a multi-level intent feature. S3: obtaining an initial intent label and an initial slot prediction label using an intent decoding module and a slot decoding module, respectively I and a slot feature matrix H S , respectively using an intent decoding module and a slot decoding module S4: In the intent recognition task, the multi-level intent feature and the initial slot prediction label are input into the slot-intent interaction module to obtain an intent feature matrix H enhanced by slot label guidance [I,L] ; The slot-intent interaction module specifically realizes the process as follows: S401: fuse the multi-level intention features obtained in S2 using a gating mechanism in a multi-level intention fusion layer to obtain a fused intention feature matrix S402: Use the obtained intent feature matrix H IF , the syntax token in the intent feature matrix H IF The initial slot prediction label constructs an intent semantic label graph, strengthens the expression of intent features through a graph attention mechanism, and obtains an intent feature matrix H[I , L ] ;​ S5: In the slot filling task, input the slot feature matrix, multi-level intent feature, and initial intent label into the intent-slot interaction module to obtain the slot feature matrix H[S , L ] ; The intent-slot interaction module specifically realizes the process as follows: S501: After the multi-level intent features and the slot feature matrix are spliced in the multi-level intent perception layer, the BiLSTM model is used, and the gate mechanism is used to obtain the fused slot feature matrix S502: Use slot feature matrix H SF , slot feature matrix H SF in the syntax token The initial intent label constructs the slot semantic label graph, and strengthens the expression of slot features through graph attention mechanism, to obtain the slot feature matrix H [S,L] enhanced by syntax tokens and intent labels; S6: Slot tag guided enhanced intent feature matrix H [I,L] Input to the intent decoding module, while guiding the enhanced slot feature matrix H with the intent tag [S,L] Input to the slot decoding module, jointly train two tasks to obtain the intent label and slot sequence label of the spoken language understanding task. 2.The syntax analysis based multi-intent spoken language understanding method of claim 1, wherein, The intent decoding module and the slot decoding module in the step S3 comprise the following two parts: S301: Obtain the distribution of each word intent label based on the intent feature matrix H I , and obtain the initial sentence-level intent label by using the sigmoid function to obtain the distribution of each word intent label and using to perform word-level intent voting. S302: Obtain the slot label distribution of each word based on the slot feature matrix H S , and obtain the preliminary slot prediction label using the argmax function.

Images