An intent recognition method and device for a dialogue system with insufficient training data

Abstract

The application discloses an intent recognition method and device for a dialogue system with insufficient training data. The method comprises the following steps: recognizing, by an intent classification module, whether a user input text is an inquiry intent or a greeting intent; if the user input text is recognized as a greeting intent, the user input text is delivered to a robot of a greeting module for reply; if the user input text is recognized as an inquiry intent, the user input text is delivered to an intent detection module for recognition; if the intent detection module recognizes the user input text as a certain known intent category, the user input text is delivered to a corresponding question and answer robot of the intent category for reply; if the user input text is recognized as an unknown intent category, a rejection reply is performed; and clustering and analyzing data in the unknown intent category in a certain time period by an intent exploration module, so that a management and maintenance personnel of an intelligent customer service iteratively updates the intent category. The application can cope with insufficient training data or categories of intents by constructing a three-layer intent recognition architecture, and provides a direction for improving the quality of text data used for subsequent training.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an intent recognition method and apparatus for dialogue systems with insufficient training data. Background Technology

[0002] Dialogue systems have always been an important topic in the field of artificial intelligence applications. They can help companies that require frequent user interaction reduce labor costs. Excellent dialogue systems can also improve the efficiency of responding to users, thereby increasing user satisfaction. Intent recognition is a crucial component of task-oriented dialogue systems, aiding in the subsequent analysis and response to user questions. In mainstream research, it is typically viewed as a fixed-category text classification task.

[0003] To identify the intent behind user questions, researchers have recently proposed methods using deep learning models, including Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), and attention-based RNNs (RNN+attention). These intent classification methods have achieved good accuracy, but they can only identify predefined intent categories and cannot be freely extended. Since user intent is random and endless, the intent of user questions may frequently increase or change over time and with changing events, making the initial definition of intent categories insufficient to meet later user needs. Furthermore, in practical applications, summarizing user intent and collecting related text presents difficulties, resulting in a limited number of categories and data in the early stages of system implementation.

[0004] Some existing technologies classify unknown intent categories by introducing training sets from other domains as the (k+1)th category, but this requires additional training sets, which are difficult to obtain in practice. Other existing technologies distinguish intents by setting fixed thresholds, but this method requires extensive experimentation after each data update, consuming significant resources. Still other existing technologies identify known and unknown intents based on the geometric distribution of text generation features, but text vectors generated by pre-trained models are densely distributed in the vector space, making it difficult to accurately separate known and unknown intents based solely on geometric distribution. Summary of the Invention

[0005] This invention provides an intent recognition method and apparatus for dialogue systems with insufficient training data, in order to solve the problem of insufficient intent recognition data when applying question-and-answer systems. When data is missing in a known intent category, the method can supplement the data in the relevant intents according to actual use; when intent categories are missing, the method can expand the intent categories according to actual use, thereby better solving the problem of incorrect recognition of unknown intent categories.

[0006] In a first aspect, embodiments of the present invention provide an intent recognition method for dialogue systems with insufficient training data, comprising:

[0007] S1. The intent classification module identifies the user's input text. If it is identified as a greeting intent, it is passed to the chatbot in the greeting module for a reply; if it is identified as a question intent, it is passed to the intent detection module for identification.

[0008] S2. If the intent detection module identifies the user's input text as a known intent category, it will pass it to the corresponding question-answering robot for a response; if it identifies it as an unknown intent category, it will reject the response and pass it to the intent exploration module for identification.

[0009] S3. The intent exploration module performs clustering and identification on the input text corresponding to the unknown intent category within a certain time period, and performs simple filtering on the identified data and categories. If the filtering results in a new intent category, the new intent category and the corresponding text data are added to the dataset of known intent categories; otherwise, the filtered text data is added to the dataset of known intent categories.

[0010] Optionally, following S3, the following may also be included:

[0011] The intent classification and intent detection modules were retrained using the expanded dataset of known intent categories.

[0012] Optionally, S2 specifically includes:

[0013] The user input text is converted into an intent feature vector and the corresponding intent probability distribution is extracted through a pre-trained model.

[0014] A threshold loss function is used to learn the threshold of the intent probability distribution to obtain the learned probability threshold.

[0015] During inference, the probability distribution of intent corresponding to the user input text is compared with the learned probability threshold to determine whether the user input text belongs to a known intent category or an unknown intent category.

[0016] Optionally, the pre-trained model includes two different dropout parameters, and during training, the training set is passed through two pre-trained models with different dropout parameters.

[0017] Optionally, the formula for the threshold loss function is:

[0018]

[0019] Where N represents the total number of intent categories, c i Δ represents the intent category of the i-th sample. ci For the cthi The probability threshold learned in each category, σ i The formula is:

[0020]

[0021] x i Let be the probability distribution of intent corresponding to the i-th sample.

[0022] Optionally, S3 specifically includes:

[0023] The pre-trained model converts user input text with unknown intent into vectors and extracts the corresponding intent feature vectors.

[0024] By using contrastive learning in the neighborhood, the intention feature vectors of unknown intentions are clustered together;

[0025] A clustering algorithm is used to cluster the feature vectors of unknown intentions based on the distribution of known intention data.

[0026] Secondly, embodiments of the present invention also provide an intent recognition device for dialogue systems with insufficient training data, comprising:

[0027] The intent classification module is used to identify user-input text. If it is identified as a greeting intent, it is passed to the chatbot in the greeting module for a response; if it is identified as a question intent, it is passed to the intent detection module for identification.

[0028] The intent detection module identifies user input text as belonging to a known intent category and then passes it to the corresponding question-answering robot for a response; if it is identified as an unknown intent category, it rejects the input and passes it to the intent exploration module for identification.

[0029] The intent exploration module is used to cluster and identify input text corresponding to unknown intent categories within a certain time period, and to perform simple filtering on the identified data and categories. If the filtering results in a new intent category, the new intent category and its corresponding text data are added to the dataset of known intent categories; otherwise, the filtered text data is added to the dataset of known intent categories.

[0030] This invention employs an intent classification module to categorize user text corpora into greeting intents and inquiry intents; an intent detection module further categorizes inquiry intents into various fine-grained categories and can detect unknown intent categories not previously defined, facilitating appropriate responses or rejections from different chatbots; and an intent exploration module performs cluster analysis on unknown intents, allowing administrators to iteratively update intent categories for improved intent recognition. This three-layer intent recognition architecture addresses situations such as insufficient intent training data or categories, enabling the system to more accurately understand user needs, better respond to user questions, improve user experience, and provide direction for improving the quality of text data used for training. Attached Figure Description

[0031] Figure 1 A flowchart of an intent recognition method for dialogue systems with insufficient training data, provided as an embodiment of the present invention;

[0032] Figure 2 This invention provides an overall architecture diagram of an intent recognition method for dialogue systems with insufficient training data, as provided in an embodiment of the invention.

[0033] Figure 3 This is an architecture diagram of the intent detection module provided in an embodiment of the present invention;

[0034] Figure 4 The intent exploration module architecture provided for embodiments of the present invention. Detailed Implementation

[0035] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, the accompanying drawings show only the parts relevant to the present invention, and not all of the structures.

[0036] Example

[0037] See Figure 1 and Figure 2 The flowchart of an intent recognition method for dialogue systems with insufficient training data provided by this invention includes the following steps:

[0038] S1. The intent classification module identifies the user's input text. If it is identified as a greeting intent, it is passed to the chatbot in the greeting module for a reply; if it is identified as a question intent, it is passed to the intent detection module for identification.

[0039] S2. If the intent detection module identifies the user's input text as a known intent category, it passes it to the corresponding question-and-answer robot for a response. If it identifies it as an unknown intent category, it rejects the input and guides the user to describe the problem in more detail to express a clearer intent, or it reports the situation to human customer service for further understanding and detailed answers. User input text identified as an unknown intent category is further passed to the intent exploration module.

[0040] Furthermore, S2 specifically includes the following steps: converting user input text into intent feature vectors and extracting the corresponding intent probability distribution through a pre-trained model; using a threshold loss function to learn the threshold of the intent probability distribution to obtain the learned probability threshold; during inference, comparing the intent probability distribution corresponding to the user input text with the learned probability threshold to determine whether the user input text belongs to a known intent category or an unknown intent category.

[0041] The main architecture of the intent detection module is as follows: Figure 3 As shown. First, a pre-trained model (such as BERT) is used to convert the user's input question into a text vector and extract the intent features from the text. In this embodiment, the pre-trained model is fine-tuned using a known intent category dataset to make the model more adaptable to the current task scenario, enabling the pre-trained model to accurately extract the intent features in this application scenario.

[0042] The pre-trained model in this embodiment includes two different dropout parameters. When training the model, the training set is passed through two pre-trained models with different dropout parameters. This method can obtain slightly different but similar semantic text vectors, thereby making the trained model more generalizable.

[0043] Furthermore, each threshold used to distinguish between known and unknown intent categories is learned, balancing the positive and negative losses for each category, making the threshold a suitable boundary between known and unknown intents. The formulas for calculating the loss function for the external and internal cases of the intent category threshold are shown below:

[0044]

[0045] Where N represents the total number of intent categories, c i Δ represents the intent category of the i-th sample. ci For the cth i The probability threshold learned in each category, while σ i The definition is shown in the following formula:

[0046]

[0047] Finally, by using the learned probability threshold, the user's question is identified as either a known intent category or an unknown intent category.

[0048] S3. The intent exploration module clusters and identifies the input text corresponding to unknown intent categories within a certain time period, and performs simple filtering on the identified data and categories. If the filtering results in a new intent category, the new intent category and its corresponding text data are added to the dataset of known intent categories; otherwise, the filtered text data is added to the dataset of known intent categories.

[0049] In this embodiment, the recognition results of the intent exploration module can be shown to the backend maintenance personnel. After simple screening, the backend maintenance personnel can add the new intent category and the corresponding user text to the known intent database and retrain the intent classification and intent detection modules to improve the accuracy of subsequent responses.

[0050] The above S3 specifically includes the following steps: converting user input text with unknown intent into vectors and extracting the corresponding intent feature vectors through a pre-trained model;

[0051] By using contrastive learning in the neighborhood, the intention feature vectors of unknown intentions are clustered together;

[0052] A clustering algorithm is used to cluster the feature vectors of unknown intentions based on the distribution of known intention data.

[0053] Specifically, the intention is to explore the basic architecture of the algorithm, such as Figure 4 As shown, firstly, intent features are extracted from the input text, transforming the user-input text into vectors. The pre-trained model used for extracting intent features is fine-tuned using data from known intent categories, enabling the model to better extract intent features specific to the domain.

[0054] Secondly, the k nearest neighbors of the training instances in the vector space are identified, and then the nearest neighbor vectors are compared with the original vectors for learning. This step allows data vectors of the same category to cluster together more effectively. Finally, a simple clustering method (such as DBSCAN) is used to cluster the vectors. Using data with known intent categories, the centroid of each known category is calculated, and the distance from each data vector to its category centroid is calculated. The average distance for each category is set as the eps value. At the same time, the number of vectors in each category is calculated, and the minimum value is used as the min_samples value. This method uses the distribution of known intent data to guide the clustering of unknown intents, ultimately obtaining the classification results of user input data with unknown intents.

[0055] This embodiment, through the intent exploration module, can promptly expand the categories of known intents, making it convenient to understand and analyze the user's current main intent.

[0056] Implement verification

[0057] This invention can accurately detect known user intents and distinguish unknown intents when there are few intent categories and a small amount of data, and facilitates the collection of new intent category data to improve user experience.

[0058] To better train and test the model's recognition performance, this invention collects and cleans question-and-answer data from real-world scenarios, creating a Chinese question-and-answer intent domain dataset. This dataset includes coarse intent classification and fine intent classification datasets. The coarse intent classification dataset contains 37,866 training samples and 5,248 test samples, divided into two categories: greetings and inquiries. The fine intent classification dataset contains 18,893 user texts in the training set and 4,838 user texts in the test set, categorized into 19 known intent categories.

[0059] For each module, multiple experimental results demonstrate that the method of this invention performs well in both intent detection and intent exploration. Table 1 quantitatively compares the performance of current state-of-the-art methods and the intent detection module proposed in this invention on relevant Chinese question-answering datasets in terms of accuracy and F1 evaluation metrics. It can be seen that the method of this invention performs excellently.

[0060]

[0061] Table 2 quantitatively compares the performance of current state-of-the-art methods and the intent exploration module proposed in this invention on relevant Chinese question-answering datasets in terms of accuracy, normalized mutual information (NMI), and adjusted Land coefficient (ARI). It can be seen that the method of this invention performs excellently.

[0062]

[0063] The present invention also provides an intent recognition device for dialogue systems with insufficient training data, comprising:

[0064] The intent classification module is used to identify user-input text. If it is identified as a greeting intent, it is passed to the chatbot in the greeting module for a response; if it is identified as a question intent, it is passed to the intent detection module for identification.

[0065] The intent detection module identifies user input text as belonging to a known intent category and then passes it to the corresponding question-answering robot for a response; if it is identified as an unknown intent category, it rejects the input and passes it to the intent exploration module for identification.

[0066] The intent exploration module is used to cluster and identify input text corresponding to unknown intent categories within a certain time period, and to perform simple filtering on the identified data and categories. If the filtering results in a new intent category, the new intent category and its corresponding text data are added to the dataset of known intent categories; otherwise, the filtered text data is added to the dataset of known intent categories.

[0067] Optionally, the device may also include: retraining the intent classification module and the intent detection module with an expanded dataset of known intent categories.

[0068] The aforementioned intent detection module is specifically used for:

[0069] The user input text is converted into an intent feature vector and the corresponding intent probability distribution is extracted through a pre-trained model.

[0070] A threshold loss function is used to learn the threshold of the intent probability distribution to obtain the learned probability threshold.

[0071] During inference, the probability distribution of intent corresponding to the user input text is compared with the learned probability threshold to determine whether the user input text belongs to a known intent category or an unknown intent category.

[0072] The pre-trained model includes two different dropout parameters. During training, the training set is passed through the two pre-trained models with different dropout parameters.

[0073] Specifically, the formula for the threshold loss function is:

[0074]

[0075] Where N represents the total number of intent categories, c i Δ represents the intent category of the i-th sample. ci For the cth i The probability threshold learned in each category, σ i The formula is:

[0076]

[0077] x i Let be the probability distribution of intent corresponding to the i-th sample.

[0078] Furthermore, the intent exploration module is specifically used for:

[0079] The pre-trained model converts user input text with unknown intent into vectors and extracts the corresponding intent feature vectors.

[0080] By using contrastive learning in the neighborhood, the intention feature vectors of unknown intentions are clustered together;

[0081] A clustering algorithm is used to cluster the feature vectors of unknown intentions based on the distribution of known intention data.

[0082] The intent recognition device for dialogue systems with insufficient training data provided in this embodiment of the invention can execute the intent recognition method for dialogue systems with insufficient training data provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0083] Note that the above description is merely a preferred embodiment of the present invention and the technical principles employed. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of the present invention, the scope of which is determined by the scope of the appended claims.

Claims

1. A method for intent recognition in dialogue systems with insufficient training data, characterized in that, include: S1. The intent classification module identifies the user's input text. If it is identified as a greeting intent, it is passed to the chatbot in the greeting module for a reply; if it is identified as a question intent, it is passed to the intent detection module for identification. S2. If the intent detection module identifies the user's input text as a known intent category, it will pass it to the corresponding question-answering robot for a response; if it identifies it as an unknown intent category, it will reject the response and pass it to the intent exploration module for identification. S3. The intent exploration module clusters and identifies the input text corresponding to unknown intent categories within a certain time period, and filters the identified data and categories. If a new intent category is selected, the new intent category and its corresponding text data are added to the dataset of known intent categories. Otherwise, add the filtered text data to the dataset with known intent categories; S2 specifically includes: The user input text is converted into an intent feature vector and the corresponding intent probability distribution is extracted through a pre-trained model. A threshold loss function is used to learn the threshold of the intent probability distribution to obtain the learned probability threshold. During inference, the probability distribution of intent corresponding to the user input text is compared with the learned probability threshold to determine whether the user input text belongs to a known intent category or an unknown intent category. The formula for the threshold loss function is: Where N represents the total number of intent categories. It is the intent category of the i-th sample. For the first The probability threshold learned in each category, The formula is: Let i be the probability distribution of intent corresponding to the i-th sample; S3 specifically includes: The pre-trained model converts user input text with unknown intent into vectors and extracts the corresponding intent feature vectors. By using contrastive learning in the neighborhood, the intention feature vectors of unknown intentions are clustered together; A clustering algorithm is used to cluster the feature vectors of unknown intentions based on the distribution of known intention data.

2. The method according to claim 1, characterized in that, Following S3 are: The intent classification and intent detection modules were retrained using the expanded dataset of known intent categories.

3. The method according to claim 2, characterized in that, The pre-trained model includes two different dropout parameters. During training, the training set is passed through the two pre-trained models with different dropout parameters.

4. An intent recognition device for dialogue systems with insufficient training data, characterized in that, include: The intent classification module is used to identify the text input by the user. If it is identified as a greeting intent, it is passed to the chatbot in the greeting module for a response. If the intent is identified as a query, it is passed to the intent detection module for further processing. The intent detection module identifies user input text as belonging to a known intent category and then passes it to the corresponding question-answering robot for a response; if it is identified as an unknown intent category, it rejects the input and passes it to the intent exploration module for identification. The intent detection module converts user input text into intent feature vectors and extracts the corresponding intent probability distributions through a pre-trained model. A threshold loss function is used to learn the threshold of the intent probability distribution to obtain the learned probability threshold. During inference, the probability distribution of intent corresponding to the user input text is compared with the learned probability threshold to determine whether the user input text belongs to a known intent category or an unknown intent category. The formula for the threshold loss function is: Where N represents the total number of intent categories. It is the intent category of the i-th sample. For the first The probability threshold learned in each category, The formula is: Let i be the probability distribution of intent corresponding to the i-th sample; The intent exploration module is used to cluster and identify input text corresponding to unknown intent categories within a certain time period, and to filter the identified data and categories. If a new intent category is selected, the new intent category and its corresponding text data are added to the dataset of known intent categories; otherwise, the filtered text data is added to the dataset of known intent categories. The intent exploration module converts user input text with unknown intent into vectors and extracts the corresponding intent feature vectors through a pre-trained model. By using contrastive learning in the neighborhood, the intention feature vectors of unknown intentions are clustered together; A clustering algorithm is used to cluster the feature vectors of unknown intentions based on the distribution of known intention data.

Images