A multi-level intent recognition method for customer service based on knowledge fusion

Abstract

The application discloses a customer service multi-level intention recognition method based on knowledge fusion and belongs to the technical field of text classification of natural languages, and comprises the following steps: S1, knowledge base construction; S2, knowledge fusion; and S3, multi-intention recognition.The application firstly generates similar questions from standard questions to increase the difference between standard questions of the same category; then, the standard questions in the knowledge base are regarded as features and are integrated into user input questions to form a joint feature vector which is input into a knowledge fusion model to provide certain prior knowledge and make up for the user semantic ambiguity; finally, the knowledge fusion model performs progressive learning according to multi-level labels to recognize the multi-level intention of the user.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of text classification of natural language, and particularly relates to a customer service multistage intention recognition method based on knowledge fusion. BACKGROUND

[0002] The rapid development of the Internet also drives the rapid development of artificial intelligence. With the expansion and maturity of the scale of operators, the base number of users is increased, the number of businesses and problems is increased, limited customer service personnel cannot timely solve all user problems, and it is difficult to provide high-quality services. Using artificial intelligence related technology to solve the problem is the market trend, and digital customer service has become a key direction for future evolution and development. Meanwhile, intention recognition is essentially a classification problem. On the one hand, the data required for classification needs to be manually screened and labeled before it can be used, and there are certain requirements for data quality. On the other hand, for a task in which one piece of text corresponds to multiple categories and the categories have a cascading relationship, directly using the classification model result is not ideal. The fundamental reason is that in actual application scenarios, user colloquialisms are random, the context is complex, and there are a lot of invalid or interfering information in between. These cause user semantics to be ambiguous, resulting in low intention recognition accuracy. Therefore, a customer service multistage intention recognition method based on knowledge fusion is provided. SUMMARY

[0003] The technical problem to be solved by the application is how to solve the problem of semantic ambiguity caused by short text colloquialisms and non-standard grammar in the field of operator customer service. The application provides a customer service multistage intention recognition method based on knowledge fusion. In the method, standard questions in a standard question knowledge base are taken as features and are integrated into user input questions to form a joint feature vector, which is input into a knowledge fusion model together. Certain prior knowledge is provided, and more rich semantic information is obtained. The intention category of the user is predicted according to a knowledge fusion multistage intention recognition algorithm, and the accuracy of customer service multistage intention recognition is optimized.

[0004] The application solves the above technical problems by the following technical scheme. The application comprises the following steps:

[0005] The application comprises the following steps:

[0006] S1: Knowledge base construction

[0007] Standard questions are extracted from a corpus, similar questions are generated by data augmentation through multiple groups of Transformers according to the standard questions, and a standard question knowledge base is obtained;

[0008] S2: Knowledge fusion

[0009] The standard question knowledge base vector is integrated into the text vector, that is, the standard questions in the knowledge base are taken as features and are integrated into the user input questions, and then a joint feature vector is formed and is input into a knowledge fusion model;

[0010] S3: Multi-intent recognition

[0011] The knowledge fusion model learns the relationship between different layers of categories and texts, and predicts the final subcategory according to the prediction result of the parent category during prediction, performs top-down hierarchical classification, and obtains the final prediction result by aggregating the prediction results of different layers.

[0012] Further, in the step S1, the specific process is as follows:

[0013] S11: Extract a set of standard questions from the corpus, respectively denoted as sentence1 and sentence2, add a [CLS] symbol as the start position of the text, distinguish the two sentences with a [SEP] symbol, and add an end position at the same time. The data is converted into the form of [CLS]sentence1[SEP]sentence2[SEP];

[0014] S12: Then, according to sentence1, predict sentence2 to perform a similar sentence generation task;

[0015] S13: After training with multiple question pairs, the input sentence will generate similar questions with similar semantics, and then obtain the standard question knowledge base.

[0016] Further, in the step S2, the acquisition and processing process of the text vector is as follows:

[0017] S201: For user text Q = {q1, q2, …, qn}, its corresponding hierarchical label structure is Y = {y1, y2, …, yh}, where qn represents the nth sentence in the text Q, h represents the number of label levels, and yi represents the set of i-th level labels. n 1 2 h n i

[0018] S202: The user text is input into the embedding layer Embedding to obtain the word vector of the text , and then input into the Bi-GRU model to obtain the context representation at two different times through the Bi-GRU model:

[0019]

[0020]

[0021]

[0022] , where ​​​​​​Let represent the context representations obtained by GRU in the i-th different directions, respectively. This indicates that the contextual representations obtained from different directions are spliced ​​together. Represents the previous hidden state in different directions, and concat(·) concatenates the two vectors. and This means that the internal structure of the GRU model remains unchanged, but the direction of the input text is different.

[0023] Furthermore, in step S202, the GRU model function is:

[0024]

[0025]

[0026]

[0027]

[0028] Among them, z i The vector r represents the input gate neuron. i This represents the vector obtained by resetting the gate neuron. h represents the vector obtained by the memory gate neuron. i W represents the memory state of a vector, σ represents the sigmoid activation function, and W z W r W represents the parameters of each gate neuron.

[0029] Furthermore, in step S2, the acquisition and processing of the standard question are as follows:

[0030] S211: Randomly select k standard questions from the standard questions for each intent category to form a new standard question knowledge base T = {t1, t2, ..., t3}. m Word vectors are obtained through embedding layer processing. Among them, t m This represents the m-th sentence in the standard question knowledge base T;

[0031] S212: The word vectors are input into the Bi-LSTM model. Information from the entire sentence is fused into each hidden state of the Bi-LSTM model, and after encoding, the following is obtained:

[0032]

[0033]

[0034]

[0035] in, represents the context representation obtained by the Bi-LSTM in the jth different direction, represents the vector obtained by splicing the context representations obtained in different directions, represents the previous hidden layer state of different directions, and concat(·) represents splicing two vectors, and represents that the internal structure of the LSTM model is unchanged, and the direction of the input text is different.

[0036] Further, in the step S212, the LSTM model function is:

[0037]

[0038]

[0039]

[0040]

[0041]

[0042] h j =o j *tanh(C j )

[0043] wherein f j represents the vector obtained by the forgetting gate, r j represents the vector obtained by the input gate neuron, represents the vector obtained by the memory gate neuron, C j represents the cell state, o j represents the vector obtained by the output gate, h j represents the hidden state of the output gate, and σ represents the sigmoid activation function, W f , W r , W c , W o , b f , b j , b C , and b0 are parameters of each gate neuron.

[0044] Further, in the step S2, the text vector and the standard question vector are not directly spliced, but the text vector and the standard question knowledge base are mapped into the same dimension feature space, and the vectors are added to form a multi-feature fusion vector. The text features are extracted by using the convolutional neural network, the feature map calculated by convolution is processed by maximum pooling to retain the maximum feature value, and then spliced into a vector as the final knowledge representation:

[0045]

[0046]

[0047] wherein, represents a vector and a vector obtained by adding vectors represents a vector obtained after feature extraction, and avg represents an operation of retaining the maximum feature value by maximum pooling processing on a feature map calculated by convolution of a convolutional neural network.

[0048] Further, in the step S3, the knowledge fusion model includes multiple classification modules, can utilize more fine-grained hierarchical information for top-down hierarchical classification, has a local classifier under each parent class label, and predicts the final child class according to the prediction result of the parent class during prediction, and mines the potential spatial structure of the text.

[0049] Compared with the prior art, the customer multi-level intent recognition method based on knowledge fusion has the following advantages: the method generates similar questions from standard questions to increase the difference between standard questions of the same category; then, the standard questions in the knowledge base are taken as features and integrated into user input questions to form a joint feature vector, which provides certain prior knowledge for the knowledge fusion model and makes up for the fuzzy user semantic situation; finally, the knowledge fusion model is used for progressive learning of customer intent recognition to recognize the multi-level intent of the user. BRIEF DESCRIPTION OF DRAWINGS

[0050] Figure 1 is a flowchart of the customer multi-level intent recognition method based on knowledge fusion in the embodiment of the application. DETAILED DESCRIPTION

[0051] The embodiment of the application will be described in detail below, and the embodiment is implemented on the premise of the technical solution of the application, and detailed implementation modes and specific operation processes are given, but the protection scope of the application is not limited to the following embodiment.

[0052] As Figure 1 shown, the embodiment provides a technical solution: a customer multi-level intent recognition method based on knowledge fusion, which fills in the semantic features of the standard question knowledge base to solve the problem of poor intent recognition effect caused by fuzzy user semantics, captures the true intent behind the input sentence of the user, and searches based on the true intent, so as to return the search result that is most suitable for the needs of the user.

[0053] The application specifically includes the following main steps:

[0054] (1) Construction of a standard question knowledge base based on the operator customer service field

[0055] The standard question knowledge base is a powerful multifunctional tool, and the main goal is to help automate customer service by providing self-service options to users. Based on the existing intelligent customer service system, a large amount of dialogue text data is accumulated, and due to the uneven level of business personnel, it is difficult to provide high-quality dialogue for different problems, so it is particularly important to build a high-quality standard question knowledge base.

[0056] First, extract high-quality standard questions from the corpus, generate similar questions according to the standard questions, and based on the pre-training model, give the pre-training model the ability to retrieve and generate through multiple groups of Transformers. First, according to the input text and the corresponding similar text, such as sentence1 and sentence2, add a [CLS] symbol as the start position of the text, and use a [SEP] symbol to distinguish between the two sentences, and add a sentence end position. Then, according to sentence1, predict sentence2, and perform a similar sentence generation task. At the same time, some masks [MASK] are randomly added during input, which enhances the generation and understanding ability of the pre-training model. After a large number of question pairs are trained, the input sentence will generate similar questions with similar semantics. For example, "How much is my remaining balance?", This sentence generates "Help me check how much the balance is?", "How much is my monthly balance?" and other questions with the same semantics. This generated question will not be the same, and will be more diverse, and will generate some questions with summary properties and more specific questions, thereby solving the problem of difficulty in constructing a standard question knowledge base and low difference.

[0057] (2) User intent and standard question knowledge base knowledge fusion based on operator customer service field

[0058] Due to the shortness of the text in the operator customer service field, the non-standard grammar, and the semantic ambiguity, the user intent recognition accuracy is not high. The present application proposes a knowledge fusion representation research based on the operator customer service field, which enriches the text feature information and makes the user intent classification algorithm more discriminative.

[0059] First, for the user text Q = {q1, q2, …, q n}, q n represents the nth sentence in the text Q, and the corresponding hierarchical label structure Y = {y 1 , y 2 , …, y h}, h represents the number of label levels, and y irepresents the set of the i-th layer label; the vector of the text is obtained by embedding layer Embedding The word vector is input into a Bi-GRU model (a GRU model for bidirectional calculation of an input sequence), wherein the GRU model function is:

[0060]

[0061]

[0062]

[0063]

[0064] wherein z i represents the vector obtained by the input gate neuron, r i represents the vector obtained by the reset gate neuron, represents the vector obtained by the memory gate neuron, h i represents the memory state of the vector, and sigma represents a sigmoid activation function, W z , W r , and W are parameters of each gate neuron, which are obtained in the training process.

[0065] In order to facilitate the representation, the above process is directly represented by GRU as follows. Two context representations at different times are obtained by the Bi-GRU model respectively:

[0066]

[0067]

[0068]

[0069] wherein, represents the context representation obtained by the GRU in the i-th different direction, represents the context representation obtained by the GRU in the i-th different direction, represents the last hidden layer state in the different direction, and concat(·) represents the concatenation of two vectors, and represents that the internal structure of the GRU model is unchanged, and the direction of the input text is different.

[0070] On the basis of the text vector, the standard question knowledge base vector is integrated to form a joint feature vector, which is input into a knowledge fusion model. According to the prior knowledge, the semantic information of the user is enriched, k standard questions in each intent category are randomly extracted to form a new standard question knowledge base T = {t1, t2, …, tk}, and the standard question knowledge base is input into the knowledge fusion model. m} and t mThe mth sentence of the standard question knowledge base T is represented as a sentence, and a word vector is obtained through an embedding layer Embedding It is input into a Bi-LSTM model, where the LSTM model function is:

[0071]

[0072]

[0073]

[0074]

[0075]

[0076] h j =o j *tanh(C j )

[0077] where f j represents the vector obtained by the forget gate, r j represents the vector obtained by the input gate neuron, represents the vector obtained by the memory gate neuron, C j represents the cell state, o j represents the vector obtained by the output gate, h j represents the hidden state of the output gate, and σ represents the sigmoid activation function, W f , W r , W c , W o , b f , b j , b C , and b0 are parameters of each gate neuron, obtained during the training process.

[0078] For convenience, the above process is directly represented as LSTM below. In each hidden layer state of the Bi-LSTM model, the information of the entire sentence is fused, and the encoding is obtained as:

[0079]

[0080]

[0081]

[0082] where, represents the context representation obtained by Bi-LSTM in the jth different direction, represents the vector obtained by concatenating the context representations obtained in different directions, The last hidden layer state of different directions is represented, and concat(·) represents concatenating two vectors, With The internal structure of the LSTM model is unchanged, and the direction of the input text is different.

[0083] To prevent the obtained representation dimension from being too high, dimension reduction will cause the loss of semantic information, the application does not directly concatenate the text vector and the standard question vector, but maps the text vector and the standard question knowledge base into the same dimension feature space, adds the vectors to form a multi-feature fusion vector, extracts the text features by using a convolutional neural network (CNN), and the feature map calculated by convolution is subjected to maximum pooling processing (Max Pooling) to retain the maximum feature value, and then concatenated into a vector as the final knowledge representation:

[0084]

[0085]

[0086] Among them, The vector is added to the vector , The vector obtained after feature extraction is represented, and avg represents the operation of retaining the maximum feature value by maximum pooling processing (Max Pooling) on the feature map calculated by convolution of the convolutional neural network (CNN).

[0087] (3) Knowledge fusion multi-level intent recognition algorithm based on operator customer service field

[0088] The vector text and the vector label are subjected to hierarchical learning based on the attention mechanism-based hierarchical memory unit, and the internal classifier performs progressive learning according to the label hierarchical number corresponding to the text. Under this method, multiple classification modules are contained for training, and there is a local classifier under each classifier. When classifying, the classification of the sample is determined from the root node of the classification system top-down, and the final sub-category is predicted according to the prediction result of the parent category. The original text semantic representation is combined with the associated text category representation of the information introduced from the previous layer to generate the overall representation and predict the category of each layer.

[0089]

[0090]

[0091]

[0092] Among them, The text and the hth layer label interaction information are represented, Attention weight representing the h-th layer text and label interaction, w h-1 representing the last class level, representing the probability of the h-th layer label prediction, representing the information of the h-th layer knowledge fusion model overall representation, w h The information learned by the h-th layer is used as memory information for transmission learning. The label set is taken as a recall target set by the knowledge fusion-based intent recognition method, and the label most similar to the semantic of the input text is recalled as the label category system of the text.

[0093] Comparison of the performance of the existing model construction method and the model proposed in the application

[0094] In the field of operator technology, the application compares the performance of the existing similarity model and the knowledge fusion-based customer service multi-level intent recognition method proposed in the application, and the classification accuracy Acc is defined as follows:

[0095]

[0096] Among them, TP represents the number of corpus whose true label is True and whose predicted label is also True, FN represents the number of corpus whose true label is True and whose predicted label is False; TN represents the number of corpus whose true label is False and whose predicted label is also False, and FP represents the number of corpus whose true label is False and whose predicted label is True.

[0097] In summary, the knowledge fusion-based customer service multi-level intent recognition method of the above-mentioned embodiments first generates similar questions from standard questions to increase the difference between standard questions of the same category; then the standard questions in the knowledge base are taken as features and integrated into the user input question to form a joint feature vector, which is input into the knowledge fusion model to provide certain prior knowledge and make up for the user's semantic ambiguity; finally, progressive learning is performed according to the knowledge fusion customer service intent recognition to identify the multi-level intent of the user. The advantage of the application is that the knowledge fusion multi-level mining method is adopted to realize the intent recognition in the field of operator customer service, and the recognition accuracy is improved by 7.96%.

[0098] Although the embodiments of the application have been shown and described above, it should be understood that the above-mentioned embodiments are exemplary and cannot be understood as limiting the application, and those skilled in the art can make changes, modifications, replacements and variations to the above-mentioned embodiments within the scope of the application.

Claims

1. A method for multi-level intent recognition of customer service based on knowledge fusion, characterized in that, Comprise the following steps: S1: knowledge base construction The standard question is extracted from the corpus, and the similar question is generated by a group of Transformers according to the standard question, and then the standard question knowledge base is obtained; S2: knowledge fusion The standard question knowledge base vector is fused into the text vector, that is, the standard question in the knowledge base is taken as a feature and fused into the user input question, and then a joint feature vector is formed and input into the knowledge fusion model; S3: multi-intent recognition The knowledge fusion model learns the relationship between different layers and the text, and predicts the final subclass according to the prediction result of the parent class during prediction, performs top-down hierarchical classification, and obtains the final prediction result by aggregating the prediction results of different layers; In the step S2, the acquisition and processing process of the text vector is as follows: S201: For user text Q = {q1, q2, …, q n}, its corresponding hierarchical label structure is Y = {y 1 , y 2 , …, y h}, wherein q n represents the nth sentence in the text Q, h represents the number of label levels, and y i represents the set of i-th level labels; S202: The user text obtains the word vector of the text through the embedding layer Embedding Input into the Bi-GRU model, and obtain two different time context representations through the Bi-GRU model: wherein, denote the context representation obtained by GRU in the i-th different direction, denote the context representation obtained by GRU in the i-th different direction, denote the context representation obtained by GRU in the i-th different direction, and denote the context representation obtained by GRU in the i-th different direction.

2. The multi-level intent recognition method for customer service based on knowledge fusion according to claim 1, characterized in that: In the step S1, the specific process is as follows: S11: Extract a group of standard questions from the corpus, respectively marked as sentence1 and sentence2, add [CLS] symbol as the start position of the text, distinguish the two sentences with a [SEP] symbol, and add end position, convert the data into the form of [CLS]sentence1[SEP]sentence2[SEP]; S12: Then, according to sentence1, a similar sentence generation task is performed to predict sentence2; S13: After training of multiple question pairs, the input sentence will generate similar questions with similar semantics, and then the standard question knowledge base is obtained.

3. The multi-level intent recognition method for customer service based on knowledge fusion according to claim 2, characterized in that: In the step S202, the GRU model function is: where z i represents the vector obtained by the input gate neuron, r i represents the vector obtained by the reset gate neuron, represents the vector obtained by the memory gate neuron, h i represents the memory state of the vector, σ represents the sigmoid activation function, W z , W r , W are parameters of each gate neuron.

4. The multi-level intent recognition method for customer service based on knowledge fusion according to claim 2, characterized in that: In the step S2, the acquisition and processing process of the standard question is as follows: S211: Randomly select k standard questions in each intention category to form a new standard question knowledge base T={t1, t2, …, tk} m} and obtain word vectors through embedding layer processing wherein, tm m represents the mth sentence of the standard question knowledge base T; S212: Input the word vector into the Bi-LSTM model, and the information of the entire sentence is fused in each hidden layer state of the Bi-LSTM model, and the encoding is obtained: where, denotes the context representation obtained by Bi-LSTM in the jth different direction, denotes the vector obtained by splicing the context representations obtained in different directions, denotes the last hidden layer state of different directions, and concat(·) denotes splicing two vectors, and denotes that the internal structure of the LSTM model is unchanged, and the direction of the input text is different.

5. The multi-level intent recognition method for customer service based on knowledge fusion according to claim 4, characterized in that: In the step S212, the LSTM model function is: h j = o j tanh(C j ) where f j represents the vector obtained by the forget gate, r j represents the vector obtained by the input gate neuron, i represents the vector obtained by the memory gate neuron, C j represents the cell state, o j represents the vector obtained by the output gate, h j represents the hidden state of the output gate, σ represents the sigmoid activation function, W f , W r , W c , W o , b f , b j , b C , b0are parameters of respective gate neurons.

6. The multi-level intent recognition method for customer service based on knowledge fusion according to claim 5, characterized in that: In the step S2, the text vector and the standard question vector are not directly spliced, but the text vector and the standard question knowledge base are mapped to the same dimension feature space, and the vectors are added to form a multi-feature fusion vector. Convolutional neural network is used for text feature extraction, the feature map calculated by convolution is processed by maximum pooling to retain the maximum feature value, and then spliced into a vector as the final knowledge representation: wherein, a vector a vector a vector, denotes a vector obtained after feature extraction, avg denotes an operation of retaining the maximum feature value by maximum pooling processing on a feature map calculated by convolution of the convolutional neural network.

7. The multi-level intent recognition method for customer service based on knowledge fusion according to claim 6, characterized in that: In the step S3, the knowledge fusion model includes multiple classification modules and can perform top-down hierarchical classification. Each parent class label has a local classifier, and the final subclass is predicted according to the prediction result of the parent class during prediction.

Images