Multi-intention spoken language understanding method based on syntactic analysis

Abstract

The invention discloses a multi-intention spoken language understanding method based on syntactic analysis, and the method comprises the steps: firstly obtaining an intention feature matrix and a slot feature matrix according to a statement input by a user, and constructing a multi-level intention feature through the intention feature matrix; secondly, for the intention feature matrix and the slot feature matrix, obtaining an initial intention label and an initial slot prediction label by using an intention decoding module and a slot decoding module respectively; inputting the multi-level intention features and the initial slot prediction label into a slot-intention interaction module to obtain an enhanced intention feature matrix; and inputting the slot feature matrix, the multi-level intention features and the initial intention tag into an intention-slot interaction module to obtain an enhanced slot feature matrix. And finally, respectively inputting the enhanced intention feature matrix and the enhanced slot feature matrix into an intention decoding module and a slot decoding module to obtain an intention tag and a slot sequence tag of the spoken language understanding task. According to the method, the intention recognition and slot sequence labeling accuracy and the multi-intention recognition accuracy are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, intent recognition and slot filling, and in particular to a multi-intent spoken language understanding method based on syntactic analysis. Background Art

[0002] Spoken language understanding is a basic task in dialogue systems, and its goal is to capture the comprehensive semantics of user utterances. This task usually includes two subtasks: intent recognition and slot filling. Intent recognition aims to analyze the semantic words input by users to determine the intent behind them. This task is usually regarded as a classification problem, which achieves accurate understanding of user needs by assigning a predetermined intent label to each user statement. The purpose of the slot filling task is to extract the structured information in the user input utterance and map it to the attribute values ​​of specific domain keywords, which is essentially a sequence labeling task.

[0003] In real life, a user's utterance usually contains multiple intents, so research on understanding multiple intents in a sentence is becoming increasingly important. Most existing works focus on how to accurately identify intents and use predicted intent labels to guide slot filling tasks. Since intent identification and slot filling are closely related, a joint model is usually used to model the two tasks. However, most existing works focus on guiding slot filling work through predicted intent labels, but ignore that slot labels can also guide the inference of intent. Furthermore, existing research lacks the use of grammatical information in sentences. Syntactic information can help the model better capture multiple intents scattered in a sentence. At the same time, since a single intent in a sentence is usually expressed by sub-sentences, existing research usually uses word-level intent recognition methods and cannot well mine the hidden information of sub-sentences. Summary of the invention

[0004] In response to the problems existing in the prior art, the present invention provides a multi-intent spoken language understanding method based on syntactic analysis, which not only improves the accuracy of multi-intent recognition and slot prediction, but also more effectively integrates syntactic information and multi-level intents, thereby improving the overall performance and robustness of the model.

[0005] In order to solve the above technical problems, the technical solution of the present invention comprises the following steps:

[0006] S1: Obtain the sentence input by the user, and use the input processing module proposed in the present invention to obtain the intention feature matrix H for the intention recognition task I and the slot feature matrix H of the slot filling task S .

[0007] S2: Based on the intention feature matrix H obtained in S1 I, use the window self-attention mechanism to build window-level intent features, and transform the intent feature matrix H I The [CLS] token is used as the sentence-level intent feature, and the intent feature matrix H obtained in S1 I The word vector in is the word-level intent feature. The sentence-level intent feature, window-level intent feature, and word-level intent feature together constitute a multi-level intent feature.

[0008] S3: The intention feature matrix H in S1 I and slot feature matrix H S The intent decoding module and slot decoding module are used to obtain the initial intent label and the initial intent slot prediction label respectively.

[0009] S4: In the intent recognition task, the multi-level intent features obtained in S2 and the initial slot prediction labels obtained in S3 are input into the slot-intent interaction module to obtain the intent feature matrix H enhanced by slot label guidance. [I , L] .

[0010] S5: In the slot filling task, the slot feature matrix obtained in S1, the multi-level intent features obtained in S2, and the initial intent labels obtained in S3 are input into the intent-slot interaction module to obtain the slot feature matrix H enhanced by the intent label guidance. [S,L] .

[0011] S6: Use the slot labels obtained in S4 to guide the enhanced intent feature matrix H [I,L] Input to the intent decoding module, and use the intent label obtained in S5 to guide the enhanced slot feature matrix H [S,L] The input is sent to the slot decoding module, and then the two tasks are jointly trained to obtain the intent labels and slot sequence labels required for the spoken language understanding task.

[0012] Furthermore, the input processing module in step S1 is as follows:

[0013] S101: Get the sentence input by the user, use the RoBert model as the encoder to generate a word vector matrix E containing basic semantics = {e CLS ,e1,e2,...,e n}.

[0014] S102: Construct a syntactic dependency tree for the sentence input by the user, use the RoBert model to encode the nodes of the dependency tree into word vectors, and then use the graph attention mechanism to aggregate semantic information to the root node of the encoded dependency tree. Finally, the root node is used as the grammatical token [g], and this grammatical token [g] is inserted into the [CLS] token of the input sequence E to construct a word vector matrix E′={e CLS ,eg ,e1,e2,...,e n}.

[0015] S103: Use two independent BiLSTM networks to read the word vector matrix E′ containing syntactic information obtained in S102 forward and backward, respectively, to generate a context-sensitive intent feature matrix H for the intent recognition task I and the slot feature matrix H of the slot filling task S .

[0016] Furthermore, the decoder module in step S3 includes two parts:

[0017] S301: Based on the intention feature matrix H obtained in S103 I , use the sigmoid function to obtain the distribution of each word intent label, and use word-level intent voting to obtain preliminary sentence-level intent labels.

[0018] S302: Based on the slot feature matrix H obtained in S103 S , use the softmax classifier to obtain the slot label distribution of each word, and use the argmax function to obtain the preliminary slot prediction label.

[0019] Furthermore, the slot-intention interaction module in step S4 includes:

[0020] S401: Use the gating mechanism in the multi-level intent fusion layer to fuse the multi-level intent features obtained in S2 to obtain the fused intent feature matrix

[0021] S402: Use the intention feature matrix H obtained in S401 IF , intention feature matrix H IF Syntax tokens in The initial slot prediction labels are obtained in S3 to construct the intent semantic label graph, and the expression of intent features is strengthened through the graph attention mechanism to obtain the intent feature matrix H enhanced by grammatical tokens and slot labels. [I,L] .

[0022] Furthermore, the intention-slot interaction module in step S5 includes:

[0023] S501: Use the BiLSTM model in the multi-level intent perception layer to model the temporal dependency between the multi-level intent features obtained in S2 and the slot feature matrix obtained in S1, and use the gating mechanism to fuse the slot features enhanced by the multi-level intent to obtain the fused slot feature matrix

[0024] S502: Use the slot feature matrix H obtained in S501 SF , slot feature matrix H SF Syntax tokens in The initial intent labels obtained in S3 are used to construct the slot semantic label graph, and the expression of slot features is enhanced through the graph attention mechanism to obtain the slot feature matrix H enhanced with grammatical tokens and intent labels. [S,L] .

[0025] The present invention has the following characteristics and benefits:

[0026] The present invention proposes a new perspective to improve the spoken language comprehension ability of the language model by integrating syntactic information. The present invention enriches semantic information by introducing a syntactic dependency tree and effectively utilizes the node information and edge information of the dependency tree to further explore the potential semantic relationship between words, further improving the accuracy of intent recognition and slot sequence labeling.

[0027] The present invention effectively makes up for the lack of slot label guidance for multi-intent recognition tasks in traditional methods. By modeling a two-way interactive intent recognition and slot filling model, the multi-intent detection task and the slot filling task share basic semantics and guide each other, thereby improving the accuracy of multi-intent recognition.

[0028] This invention breaks through the inherent limitations of traditional methods and provides a method for fusing multi-level intent information in slot filling tasks, which fully considers intent information at different levels: utterance-level intent provides global semantic information, which can effectively reduce information loss in the entire sentence; word-level intent provides more fine-grained interactions, enabling it to capture intent at the word level and understand the semantics and goals of the sentence in more detail; window-level intent identifies intent by continuous fragments, making full use of the relationship between previous and subsequent intents. Compared with the single utterance granularity approach, fusing multi-level intent can provide better multi-intent recognition and slot prediction performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 is a flow chart of the present invention;

[0030] Figure 2 It is a schematic diagram of the framework of the present invention;

[0031] Figure 3 is a schematic diagram of a syntactic information extraction module proposed in the present invention;

[0032] Figure 4 It is a semantic label graph proposed by the present invention for guiding intent recognition through slot labels;

[0033] Figure 5 It is a semantic label graph proposed by the present invention that guides slot filling through intent labels. DETAILED DESCRIPTION

[0034] In order to make the purpose, technical scheme and technical effect of the present invention more clear, the multi-intention spoken language understanding method based on syntactic analysis provided by the present invention is further described in detail below in combination with the embodiments and the accompanying drawings of the specification.

[0035] The present invention provides a method for understanding multi-intention spoken language based on syntactic analysis. Figure 1-Figure 2 As shown, the following steps are included:

[0036] S1: The semantic encoding module is used to preprocess the user input to obtain the intent feature vector and slot feature vector.

[0037] The preliminary processing and feature extraction of the data set includes the following steps:

[0038] S101: Use a word segmenter to segment the user's speech, and use the RoBert model as an encoder to generate a word vector matrix E={e CLS ,e1,e2,...,e n}.

[0039] where e t (t∈1,2,...n) represents the vector representation of the tth word in the sentence.

[0040] S102: The syntax extraction module extracts the grammatical token [g] containing grammatical information from the user's speech and inserts it after the [CLS] token in the input sequence E to construct an embedded representation E′={e CLS ,e g ,e1,e2,...,e n}.

[0041] S103: Use two different BiLSTMs to read the input vector E′ forward and backward to generate feature vectors for specific context-sensitive tasks:

[0042]

[0043] in represents the feature vector at the tth position in the intention detection task, The feature vector representing the t-th position in the slot filling task.

[0044] S2: Traditional intent recognition tasks usually directly use word embeddings to reason about intent, but a single intent in a multi-intent utterance is usually composed of sub-sentences. In order to capture the sub-sentence information representing a single intent, this method uses a multi-level intent approach to capture deeper information, including sentence-level intent, window-level intent, and word-level intent.

[0045] The specific construction method is as follows:

[0046] S201: The present invention uses a sliding window self-attention mechanism to capture the contextual information of the window. A window of size win is used to slide on the sentence, and the intent feature matrix in the window is Perform self-attention calculation to obtain the window-level intention feature matrix

[0047]

[0048] Where w is the number of windows, size is the window size, is the input matrix in the sliding window, Q (query), K (key), and V (value) are The matrix A obtained by mapping different linear parameters t ={a1,...,a i ,...,a size} is the matrix obtained by self-attention calculation.

[0049] Furthermore, window-level intent features It can be expressed as:

[0050]

[0051] Where L represents the sentence length.

[0052] S202: This method converts the intention feature matrix H I The [CLS] token in Considered as sentence-level intent features:

[0053]

[0054] S203: Characteristics of the Intent Recognition Task That is the word-level intent feature:

[0055]

[0056] S3: The present invention respectively combines the intended features and slot features Input into the intent decoding module and the slot decoding module to obtain the initial intent label IL and the initial slot prediction label SL.

[0057] The specific steps include the following:

[0058] S301: Intent Features Input into the intent decoder, the intent distribution label of the tth word can be obtained by the following formula:

[0059]

[0060] in are model training parameters, is the intent label representation corresponding to the t-th word.

[0061] Furthermore, we vote for the intent labels obtained by decoding each word: the word-level intent prediction results are converted into binary form, and those greater than 0.5 are considered positive, and those less than or equal to 0.5 are considered negative. Then, for each intent k, we calculate the number of sentences that satisfy I ( t , k ) The sum of the number of tokens with a value greater than 0.5 is used to count the votes of the tokens that receive positive and negative predictions for the kth intent. Intents with a number of votes greater than n / 2 will be added to the set of sentence intent results:

[0062]

[0063] Among them I Represents a collection of sentence intent results, Yes I The elements in the set represent a specific sentence intention. ( i , k ) represents the classification result of the t-th word for the k-th intent, and n represents the number of words.

[0064] S302: Slot feature The input is fed into a fully connected softmax layer to generate the distribution of slot labels for each word and estimate the slot label for each word.

[0065]

[0066] in are model training parameters, is the slot label distribution corresponding to the t-th word, Slot labels for initial estimates.

[0067] S4: Use a multi-level intent fusion layer to fuse multi-level intent features, and then use the intent semantic label graph to guide intent label decoding through the initially estimated slot labels. Existing methods usually ignore the guidance of slot labels on intent recognition. This invention fully mines the semantic information of slot labels by constructing a graph attention mechanism that enhances intent features with slot labels, thereby improving the performance of intent recognition and the robustness of the model.

[0068] The specific steps include the following:

[0069] S401: Get the sentence-level intent features from S2 Window Intent Features Word-level intent features Fusion via gating mechanism:

[0070] g t =σ(Wh t +b)

[0071]

[0072] Where W and b are trainable variables, g t is the gating parameter that controls how many features of each granularity will be retained, v is the intent level, is the fused intention feature.

[0073] S402: Use the slot label embedding matrix to embed the slot labels {SL1, SL2, ..., SL n} Projected into a vector form, the slot label corresponding to each word is obtained

[0074] S403: The fused intention features and E sl Input into the intent semantic label graph so that the slot semantic information can be fully utilized when performing intent recognition:

[0075] H [I,L] =GAT Intent ([H IF ,E sl ],G Intent ,θ I )

[0076] Among them [H IF ,E sl ] is the input node representation, G Intent is the intention semantic label map, θ I represents the parameters of the graph attention network, H [I,L] is the intention feature enhanced by the L-layer graph attention network.

[0077] S5: Use a multi-level intent perception layer to model and fuse multi-level intent features and slot features, and then use the slot semantic label graph to guide the decoding of slot labels through the preliminary estimated intent labels. The present invention also follows the existing research to guide the prediction of slot labels through the preliminary predicted intent labels.

[0078] The specific steps include:

[0079] S501: Since the BIO tag of the slot tag has temporal dependency, the present invention uses a BiLSTM network and introduces multi-level intent features to model the temporal dependency between slot features:

[0080]

[0081] in It represents the slot feature after integrating the v-level intent information at position t. represents the intent features of each level obtained in S2, and ∥ is the concatenation operation.

[0082] Then the obtained slot features at each level are fused through a gating mechanism:

[0083]

[0084] in Represents the slot feature representation obtained after fusing multi-level intents.

[0085] S502: embed the intent label initially identified in S301 into the intent label o using the intent label embedding matrix I ={IL1,IL2,...,IL n} Projected into a vector form, the representation of the intent label corresponding to each word is obtained

[0086] S503: The merged slot features and E il Input into the slot semantic label graph so that the intent semantic information can be fully utilized when slot decoding:

[0087] H [S,L] =GAT Slot ([H SF ,E il ],G Slot ,θ S )

[0088] Among them [H SF ,E il ] is the input node representation, G Slot is the slot semantic label graph, θ S represents the parameters of the graph attention network, H [S,L] It is the slot feature enhanced by L-layer graph attention network.

[0089] S6: By applying the method in S301, the intention feature H obtained in S403 [I,L] Input into the intent decoder to obtain the final sentence-level intent label; by applying the method in S302, the slot feature H obtained in S503 is converted into [S,L]Input into the slot decoder to obtain the final output slot label. The present invention adopts a joint training method to update the parameters of the two tasks of intent recognition and slot filling. The loss function of each task can be expressed as:

[0090]

[0091] Among them, L intent and L slot are the loss functions for intent recognition and slot filling tasks, respectively. are the true labels of intent and semantic slot, respectively, I is the number of single intent labels in a sentence, N S is the number of slot tags and n is the number of words in the sentence.

[0092] Then the two tasks are jointly optimized through the joint loss function, which can be expressed as:

[0093] L=αL intent +(1-α)L slot

[0094] Where α is a hyperparameter.

[0095] Figure 3 The syntactic information extraction module based on the syntactic dependency tree proposed by the present invention includes:

[0096] Step 1: Use spaCy's en_core_web_sm model to build a syntactic dependency tree for user input. The dependency tree can be represented by a graph G with n nodes, where each leaf node represents a word in the sentence. The edges of G represent the dependency relationship between words, and the adjacent nodes of node i can be represented by Then the RoBert model is used to encode the leaf nodes of the dependency tree to obtain the feature h of leaf node i i .

[0097] Step 2: Use the multi-head attention mechanism to aggregate neighborhood node information:

[0098]

[0099] in is the feature representation of the l+1th layer of node i, is the attention coefficient of node i and node l in the lth layer, α ij represents the normalized attention weight of node j in layer l to node i, and || is the concatenation operation.

[0100] Step 3: After applying the graph attention mechanism, the root node feature h rootas a syntactic token [g], and insert the syntactic token [g] back into the original embedding sequence E′={e CLS ,e g ,e1,e2,...,e n}:

[0101] e g =h root

[0102] Figure 4 This is a schematic diagram of the intent semantic label graph proposed in the present invention, which includes 3 types of nodes and 5 types of edges.

[0103] The three types of nodes are: Syntactic information node I g 、Intent feature node I t , slot label mapping node SL t .

[0104] The five types of edges are: Syntactic Node I g Pointing Intention Feature I t The edge, intention feature I i Pointing Intention Feature I j The edge of the intention feature I points to the edge of the slot SL, and the slot label SL points to the edge of the intention feature I. i Point to slot label SL j edge.

[0105] To match the window-level intent capture interval, I t Node and SL t Nodes are connected based on local windows, with the intention feature node I t For example, it is similar to {I t-size ,...,I t+size} and {SL t-size ,...,SL t+size}. Where size is the window size, which is consistent with the sliding window size in S201.

[0106] Then calculate its multi-head attention features:

[0107]

[0108] Where K represents the total number of heads, d is the feature dimension, and || is the concatenation operation.

[0109] in Represents the set of nodes pointing to node i on the intent semantic label graph, represents the set of nodes in the semantic label graph that are pointed to i by edges of type r. Represents the edge from node j to node i.

[0110] in is the weight matrix of edge type r in the kth head.

[0111] Figure 5 The present invention proposes a schematic diagram of a slot semantic label graph, which includes three types of nodes and five types of edges:

[0112] The three types of nodes are: Syntactic information node S g , slot feature node S t , Intent signature mapping node IL t .

[0113] The five types of edges are: Syntactic node S g Pointing slot feature S t The edge and slot labels S i Pointing intention feature S j The slot feature S points to the edge of the intention label IL, and the intention label IL refers to the edge of the slot feature S. i Pointing to the intention label IL j edge.

[0114] The types of its nodes and edges and the way graph attention is calculated and Figure 4 Since the intent is at the sentence level, that is, the intent label can guide the prediction of all slots, when constructing the graph, each intent label IL node is associated with all slot features S t Nodes are globally connected. For slot feature nodes S t Also connected by local window, S t Node and {S t-size ,...,S t+size} and {IL1,...,IL m}. Where size is the window size, which is consistent with the sliding window size in S201; m is the number of intents initially predicted in S301.

[0115] According to the above-mentioned model training method, the training parameters of the embodiment of the present invention on the MixATIS dataset are as follows: the dimensions of word embedding, label embedding and hidden state are all 256, the sliding window size of window-level intent is 3, the number of layers of all graph neural networks is 2, and the training is performed through the Adam optimizer, the batch size is 16, and the learning rate is 0.001.

[0116] The test results of the method of the present invention on the MixATIS public dataset and the comparison with other models are shown in Table 1. It should be noted that: the intent accuracy is used to measure the proportion of correct predictions of the model in the intent recognition task; the slot filling F1 score comprehensively considers the precision and recall of the slot, and is used to evaluate the comprehensive performance of the model in the slot filling task; the overall accuracy measures the proportion of samples in which the model correctly predicts the intent and all slots at the same time, that is, only when the intent of a sample and all corresponding slots are correctly predicted is it considered as a positive example.

[0117] Table 1

[0118]

[0119] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; obviously, the drawings are only some examples or embodiments of the present application, and ordinary technicians in this field can also apply the present application to other similar situations based on these drawings without creative work. In addition, it is understandable that although the work done in this development process may be complicated and lengthy, for ordinary technicians in this field, certain changes in design, manufacturing or production based on the technical content disclosed in this application are only conventional technical means and should not be regarded as insufficient content disclosed in this application.

[0120] Although the present invention has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments can still be modified, or some of the technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and all of these belong to the protection scope of this application. Therefore, the protection scope of this application shall be based on the attached claims.

Claims

1. A multi-intention spoken language understanding method based on syntactic analysis, characterized in that: The following steps are involved: S1: Get the sentence input by the user, and obtain the intention feature matrix H for the intention recognition task through the input processing module I and the slot feature matrix H of the slot filling task S ; S2: According to the intention feature matrix H I , respectively obtain window-level intent features, sentence-level intent features, and word-level intent features to form a multi-level intent feature; S3: Intent feature matrix H I and slot feature matrix H S , use the intent decoding module and the slot decoding module to obtain the initial intent label and the initial slot prediction label respectively; S4: In the intent recognition task, the multi-level intent features and the initial slot prediction labels are input into the slot-intent interaction module to obtain the intent feature matrix H enhanced by slot label guidance. [I，L] ; S5: In the slot filling task, the slot feature matrix, multi-level intent features, and initial intent labels are input into the intent-slot interaction module to obtain the slot feature matrix H enhanced by the intent label guidance. [S，L] ; S6: Use slot labels to guide the enhanced intent feature matrix H [I，L] Input to the intent decoding module, and use the intent label to guide the enhanced slot feature matrix H [S，L] The input is sent to the slot decoding module, and the two tasks are jointly trained to obtain the intent label and slot sequence label of the spoken language understanding task.

2. The multi-intention spoken language understanding method based on syntactic analysis according to claim 1 is characterized in that: The specific implementation process of the input processing module in step S1 is as follows: S101: Get the sentence input by the user, use the RoBert model as the encoder to generate a word vector matrix E containing semantics = {e CLS ,e1,e2,..., e n}; S102: Construct a syntactic dependency tree for the sentence input by the user, use the RoBert model to encode the nodes of the dependency tree into word vectors, and use the graph attention mechanism to aggregate semantic information to the root node of the encoded dependency tree; finally, use the root node as the grammatical token [g], insert the grammatical token [g] into the [CLS] token of the input sequence E, and construct a word vector matrix E′={e CLS ,e g ,e1,e2,...,e n }; S103: Use two independent BiLSTM networks to read the word embedding matrix E′ forward and backward, respectively generating a context-sensitive intent feature matrix H for intent recognition tasks and a slot feature matrix H for slot filling. S .

3. The multi-intention spoken language understanding method based on syntactic analysis according to claim 2 is characterized in that: The specific implementation process of step S2 is: according to the obtained intention feature matrix H I , use the window self-attention mechanism to build window-level intent features, and transform the intent feature matrix H I The [CLS] token is used as the sentence-level intent feature, and the intent feature matrix H is obtained. I The word vector in is the word-level intent feature; Sentence-level intent features, window-level intent features, and word-level intent features are combined to form multi-level intent features.

4. The multi-intention spoken language understanding method based on syntactic analysis according to claim 3 is characterized in that: The decoder module in step S3 comprises the following two parts: S301: Based on the intention feature matrix H I , use the sigmoid function to obtain the distribution of each word intent label, and use word-level intent voting to obtain preliminary sentence-level intent labels; S302: Based on slot feature matrix H S , use the softmax classifier to obtain the slot label distribution of each word, and use the argmax function to obtain the preliminary slot prediction label.

5. The multi-intention spoken language understanding method based on syntactic analysis according to claim 4 is characterized in that: The specific implementation process of the slot-intention interaction module is as follows: S401: Use the gating mechanism in the multi-level intent fusion layer to fuse the multi-level intent features obtained in S2 to obtain the fused intent feature matrix S402: Using the obtained intention feature matrix H IF , intention feature matrix H IF Syntax tokens in The initial slot prediction label constructs the intent semantic label graph, and the expression of intent features is strengthened through the graph attention mechanism to obtain the intent feature matrix H enhanced with grammatical tokens and slot labels. [I，L] .

6. The multi-intention spoken language understanding method based on syntactic analysis according to claim 5 is characterized in that: The specific implementation process of the intention-slot interaction module is as follows: S501: After splicing the multi-level intent features and slot feature matrix in the multi-level intent perception layer, the BiLSTM model is used to fuse the slot vectors enhanced by the multi-level intent using the gating mechanism to obtain the fused slot feature matrix. S502: Use slot feature matrix H SF , slot feature matrix H SF Syntax tokens in The initial intent label constructs the slot semantic label graph, and strengthens the expression of slot features through the graph attention mechanism, obtaining the slot feature matrix H enhanced with grammatical tokens and intent labels. [S，L] .

Images