Intention recognition method and device, equipment and medium

Through the meta-comparative learning method, the target intention scorer is constructed using visible and non-visible class datasets, which solves the problem of low efficiency and high cost of new intention recognition in the dialogue system, and achieves more efficient and accurate new intention recognition.

CN119004113APending Publication Date: 2024-11-22PURPLE MOUNTAIN LAB
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202411204914.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-29
Publication Date
2024-11-22

AI Technical Summary

Technical Problem

In dialogue systems, traditional supervised learning methods are difficult to identify new user intentions under zero samples, resulting in low recognition efficiency, high cost and low accuracy.

Method used

Using the meta-contrast learning method, the visible class data set and the non-visible class data set are used for comparison learning, and the target intention scorer is built, and new intentions are identified by embedding vectors and probability distribution functions to avoid introducing additional information.

Benefits of technology

Improve the cost efficiency and accuracy of new intention recognition, allowing the model to learn new category features faster, accurately identify new intentions, and reduce the recognition cost and time.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119004113A_ABST
    Figure CN119004113A_ABST
Patent Text Reader

Abstract

The invention discloses an intention recognition method and device, equipment and a medium, and relates to the technical field of computers, and the method comprises the steps: inputting an embedded vector of a current to-be-recognized text into a target intention scoring device; wherein the target intention scoring device is a comparative learning network model obtained by performing meta comparative learning by using a visible data set and an invisible data set, and the category of the visible data set and the category of the invisible data set are not associated; obtaining each prediction intention score of the current to-be-recognized text output by the target intention scoring device; and converting each prediction intention score into the probability of each prediction intention by utilizing a probability distribution function, and selecting the prediction intention with the maximum probability value from the probabilities of the prediction intentions as the final intention of the current to-be-recognized text. According to the scheme, the new intention recognition cost is lower, the efficiency is higher, and the precision is higher.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to an intent recognition method, device, equipment and medium. Background Art

[0002] In recent years, with the rapid development of artificial intelligence technology, conversational artificial intelligence applications have grown rapidly. In these dialogue application systems, correctly identifying user intent is the most basic and important step, which directly affects downstream tasks and decisions. In the traditional supervised learning scenario, classic classification neural networks can be used to train models through a large amount of labeled data, so that the model can learn the category knowledge of the training data well and achieve the purpose of high-precision recognition of user intent. However, in actual dialogue system applications, users often have new intents. Since there is no training data as prior knowledge for new intents, the model can only recognize intent based on the original category knowledge. Therefore, the model has the problem of unclear intent recognition, which affects the user experience of the dialogue system. Therefore, how to identify new user intents under zero samples is an urgent problem that the current dialogue system needs to solve.

[0003] Currently, the same dataset is used for training and evaluation. However, in real life, data often show a high Kullback-Leibler divergence (KL) divergence. New intents are likely to have low similarity with the features that the model has already learned, resulting in the model being unable to recognize new intents well. Alternatively, the relationship between visible and invisible class labels is introduced into the model, but this requires a lot of manpower and time, which also poses a great challenge to intent recognition.

[0004] In summary, how to make new intent recognition lower in cost, higher in efficiency and higher in accuracy is a problem to be solved in this field. Summary of the invention

[0005] In view of this, the purpose of the present invention is to provide an intent recognition method, device, equipment and medium, so that the cost of new intent recognition is lower, the efficiency is higher and the accuracy is higher. The specific scheme is as follows:

[0006] In a first aspect, the present application discloses an intention recognition method, comprising:

[0007] Inputting the embedding vector of the current text to be recognized into a target intent scorer; wherein the target intent scorer is a contrastive learning network model obtained by performing meta-contrastive learning using a visible class dataset and an invisible class dataset respectively, and there is no correlation between the categories of the visible class dataset and the categories of the invisible class dataset;

[0008] Obtaining each predicted intent score of the current text to be recognized output by the target intent scorer;

[0009] The probability distribution function is used to convert each of the predicted intention scores into the probability of each predicted intention, and the predicted intention with the largest probability value is selected from the probabilities of each of the predicted intentions as the final intention of the current text to be recognized.

[0010] Optionally, get the target intent scorer, including:

[0011] Constructing a training task including a visible class data set and a non-visible class data set, and constructing a first training sample of the visible class data set and a second training sample of the non-visible class data set;

[0012] Obtaining a first embedding vector of the first training sample and a second embedding vector of the second training sample respectively;

[0013] Meta-training an initial intent scorer using the first embedding vector to obtain a first trained intent scorer;

[0014] The first trained intent scorer is meta-tested by using the second embedding vector to obtain a second trained intent scorer, and the second trained intent scorer is determined as a target intent scorer.

[0015] Optionally, the first training sample and the second training sample each contain multiple positive and negative sample pairs, and each of the positive and negative sample pairs includes a single positive sample and multiple negative samples corresponding to the positive sample, wherein the positive sample includes a sampled text and a true intent label corresponding to the sampled text, and the negative sample includes the sampled text and other intent labels of the sampled text, wherein the other intent label of the sampled text is any intent label other than the true intent label of the sampled text.

[0016] Optionally, the obtaining a first embedding vector of the first training sample and a second embedding vector of the second training sample respectively includes:

[0017] Using a text embedding model, the first training sample is converted into a first positive sample embedding vector and a first negative sample embedding vector; wherein the first positive sample embedding vector includes a first text embedding vector and a first positive sample intention embedding vector, and the first negative sample embedding vector includes the first text embedding vector and a first negative sample intention embedding vector;

[0018] The second training sample is converted into a second positive sample embedding vector and a second negative sample embedding vector by using the text embedding model; wherein the second positive sample embedding vector includes a second text embedding vector and a second positive sample intention embedding vector, and the second negative sample embedding vector includes the second text embedding vector and a second negative sample intention embedding vector.

[0019] Optionally, the iteratively training the initial intent scorer using the first embedding vector includes:

[0020] Determine an initial intent scorer as a current intent scorer, and input the first positive sample embedding vector and the first negative sample embedding vector into a target loss function to obtain a first loss value of the current intent scorer;

[0021] Based on a first gradient descent algorithm, and using the first loss value, updating a target parameter of the current intent scorer;

[0022] Accordingly, the iterative training of the first trained intent scorer using the second embedding vector includes:

[0023] Determine the first trained intent scorer as the current trained intent scorer, and input the second positive sample embedding vector and the second negative sample embedding vector into the target loss function to obtain a second loss value of the current intent scorer;

[0024] Based on a second gradient descent algorithm, target parameters of the currently trained intent scorer are updated using the second loss value.

[0025] Optionally, the objective loss function is:

[0026] ;

[0027] in, is the text embedding vector, is the positive sample intent embedding vector, is the negative sample intent embedding vector, is the temperature parameter, and K is the number of negative samples corresponding to the text embedding vector.

[0028] Optionally, the first gradient descent algorithm is:

[0029] ;

[0030] in, is the target parameter after updating the current intention scorer, is the target parameter of the current intent scorer, is the learning rate of the first gradient descent algorithm, is the visible class dataset, is the i-th training task, is the distribution of the training tasks, is the sum of the first loss values;

[0031] Accordingly, the second gradient descent algorithm is:

[0032] ;

[0033] in, is the target parameter after updating the currently trained intent scorer, is the learning rate of the second gradient descent algorithm, is the non-visible class dataset, is the sum of the second loss values.

[0034] In a second aspect, the present application discloses an intention recognition device, comprising:

[0035] A text input module, used to input the embedding vector of the current text to be recognized into a target intent scorer; wherein the target intent scorer is a contrastive learning network model obtained by performing meta-contrastive learning using a visible class data set and an invisible class data set respectively, and there is no correlation between the categories of the visible class data set and the categories of the invisible class data set;

[0036] A score acquisition module, used to obtain each predicted intention score of the current text to be recognized output by the target intention scorer;

[0037] A probability conversion module, used to convert each of the predicted intention scores into the probability of each predicted intention using a probability distribution function;

[0038] The intention determination module is used to select the predicted intention with the largest probability value from the probabilities of each predicted intention as the final intention of the current text to be recognized.

[0039] In a third aspect, the present application discloses an electronic device, comprising:

[0040] Memory, used to store computer programs;

[0041] A processor is used to execute the computer program to implement the steps of the aforementioned disclosed intention recognition method.

[0042] In a fourth aspect, the present application discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, the steps of the aforementioned disclosed intent recognition method are implemented.

[0043] The beneficial effects of the present application are as follows: the present application inputs the embedding vector of the current text to be recognized into a target intent scorer; wherein the target intent scorer is a contrastive learning network model obtained by performing meta-contrastive learning using a visible class data set and an invisible class data set respectively, and there is no correlation between the categories of the visible class data set and the categories of the invisible class data set; the predicted intent scores of the current text to be recognized output by the target intent scorer are obtained; the probability distribution function is used to convert each of the predicted intent scores into the probability of each predicted intent, and the predicted intent with the largest probability value is selected from the probabilities of each of the predicted intents as the final intent of the current text to be recognized. It can be seen that the target intent scorer of the present application is a contrastive learning network model obtained by meta-contrastive learning using visible class data sets and invisible class data sets respectively, and there is no correlation between the categories of the visible class data sets and the categories of the invisible class data sets. That is to say, unlike the traditional meta-learning setting, the present application uses visible class data sets and invisible class data sets with no correlation between categories to perform meta-contrastive learning on the model during the model training process, which can enable the model to learn the characteristics of new categories faster, that is, the model parameters can be generalized from visible classes to invisible classes, so the final target intent scorer can recognize new intents more accurately; further, the various predicted intent scores of the current text to be recognized obtained by using the target intent scorer are converted into probabilities of each predicted intent, and the predicted intent with the largest probability value is selected from the probabilities of each predicted intent as the final intent of the current text to be recognized. Since the target intent scorer can accurately recognize new intents, it provides a strong guarantee for the subsequent determination of the final intent of the current text to be recognized, and there is no need to introduce additional information, which makes the cost of new intent recognition lower, the efficiency higher and the accuracy higher. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0045] Figure 1 A flow chart of an intention recognition method disclosed in this application;

[0046] Figure 2 A specific intention scorer training diagram disclosed in this application;

[0047] Figure 3 This is a schematic diagram of the structure of an intention recognition device disclosed in this application;

[0048] Figure 4A structural diagram of an electronic device disclosed in this application;

[0049] Figure 5 A schematic diagram of the implementation principle of a computer program product disclosed in this application. DETAILED DESCRIPTION

[0050] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0051] In recent years, with the rapid development of artificial intelligence technology, conversational artificial intelligence applications have grown rapidly. In these dialogue application systems, correctly identifying user intent is the most basic and important step, which directly affects downstream tasks and decisions. In the traditional supervised learning scenario, classic classification neural networks can be used to train models through a large amount of labeled data, so that the model can learn the category knowledge of the training data well and achieve the purpose of high-precision recognition of user intent. However, in actual dialogue system applications, users often have new intents. Since there is no training data as prior knowledge for new intents, the model can only recognize intent based on the original category knowledge. Therefore, the model has the problem of unclear intent recognition, which affects the user experience of the dialogue system. Therefore, how to identify new user intents under zero samples is an urgent problem that the current dialogue system needs to solve.

[0052] Currently, the same dataset is used for training and evaluation. However, in real life, the data often show a high divergence. The new intent is likely to have a low similarity with the features already learned by the model, resulting in the model being unable to recognize the new intent well. Alternatively, the relationship between the visible and invisible class labels is introduced into the model, but this requires a lot of manpower and time, which also poses a great challenge to intent recognition.

[0053] To this end, the present application provides a corresponding intent recognition solution, which makes new intent recognition less costly, more efficient and more accurate.

[0054] See also Figure 1 As shown, the embodiment of the present application discloses a method for identifying intent, including:

[0055] Step S11: inputting the embedding vector of the current text to be recognized into a target intent scorer; wherein the target intent scorer is a contrastive learning network model obtained by performing meta-contrastive learning using a visible class data set and an invisible class data set respectively, and there is no correlation between the categories of the visible class data set and the categories of the invisible class data set.

[0056] The current text to be recognized is converted into a corresponding embedding vector. The text embedding model (Graph Convolutional Network, i.e., text level GCN) can be used to convert the current text to be recognized into a corresponding embedding vector. Specifically, a text embedding model based on a text-level graph neural network (Text Level Graph NeuralNetwork, i.e., text-level GNN), Bert (Bidirectional Encoder Representations from Transformers) and other text embedding vector models can be used to convert the current text to be recognized into a corresponding embedding vector.

[0057] In this embodiment, obtaining a target intent scorer includes: constructing a training task including a visible class data set and an invisible class data set, and constructing a first training sample of the visible class data set and a second training sample of the invisible class data set; respectively obtaining a first embedding vector of the first training sample and a second embedding vector of the second training sample; meta-training the initial intent scorer using the first embedding vector to obtain a first trained intent scorer; meta-testing the first trained intent scorer using the second embedding vector to obtain a second trained intent scorer, and determining the second trained intent scorer as the target intent scorer. The meta-learning framework only requires a small number of gradient updates to quickly adapt to new tasks. Its rapid adaptability is due to the fact that the model learns the optimal parameters in the parameter space that are not biased towards the training data. Inspired by the powerful generalization ability of meta-learning, an intent recognition method for generalized zero-sample learning is proposed. A contrastive learning network model is trained using a meta-learning strategy. The intent scorer is obtained by training the positive and negative samples of the constructed training samples. The scorer is used to score the relationship between discourse and intent. The purpose is to enable the model to learn the general features of the data, rather than the distribution and features specific to the data set. It can be understood that before the embedding vector of the current text to be recognized is input into the target intent scorer, the target intent scorer needs to be obtained. In the process of obtaining the target intent scorer, a training task including a visible class data set and an invisible class data set is constructed to obtain a first embedding vector and a second embedding vector. The initial intent scorer is iteratively trained using the first embedding vector, and the first trained intent scorer is iteratively trained using the second embedding vector. The process of obtaining the first embedding vector includes constructing a first training sample of the visible class data set and obtaining a first embedding vector of the first training sample. The process of obtaining the second embedding vector includes constructing a second training sample of the invisible class data set and obtaining a second embedding vector of the second training sample. It should be noted that the process of obtaining the first embedding vector and the process of obtaining the second embedding vector can be executed synchronously or asynchronously. In the zero-shot learning problem, the labeled data class for training is called the visible class data set, and the unlabeled data class for testing is called the invisible class data set.

[0058] In this embodiment, the first training sample and the second training sample both contain multiple positive and negative sample pairs, and each of the positive and negative sample pairs includes a single positive sample and multiple negative samples corresponding to the positive sample, wherein the positive sample includes the sampled text and the true intent label corresponding to the sampled text, and the negative sample includes the sampled text and other intent labels of the sampled text, wherein the other intent label of the sampled text is any intent label other than the true intent label of the sampled text. Every task , which is different from the traditional few-shot learning method and The categories are the same, and in this embodiment, and The categories are disjoint, unrelated, and non-overlapping, among which the visible class dataset The first training sample and the non-visible class dataset The second training sample contains multiple positive and negative sample pairs, each of which includes a single positive sample And multiple negative samples corresponding to the positive samples , K represents the number of negative samples, the jth positive sample Includes sample text And the true intent label corresponding to the sampled text ,in, represents the training sample text set, Represents the training sample intention set, the Kth negative sample corresponding to the jth positive sample Includes sample text and other intent tags for the sampled text , other intent labels for the sampled text is a label other than the true intent label of the sampled text Any intent tag in .

[0059] In a specific embodiment, the step of respectively obtaining the first embedding vector of the first training sample and the second embedding vector of the second training sample includes: using a text embedding model to convert the first training sample into a first positive sample embedding vector and a first negative sample embedding vector; wherein the first positive sample embedding vector includes a first text embedding vector and a first positive sample intention embedding vector, and the first negative sample embedding vector includes the first text embedding vector and a first negative sample intention embedding vector; using the text embedding model to convert the second training sample into a second positive sample embedding vector and a second negative sample embedding vector; wherein the second positive sample embedding vector includes a second text embedding vector and a second positive sample intention embedding vector, and the second negative sample embedding vector includes the second text embedding vector and a second negative sample intention embedding vector. Text Embedding Model Applying graph neural networks to text data and representing text as a graph structure enables GNN to use the information in the graph structure to improve the learning effect of text features. Therefore, this embodiment uses a text embedding model. The first training sample is converted into a first positive sample embedding vector and a first negative sample embedding vector, wherein the first positive sample embedding vector includes a first text embedding vector and a first positive sample intention embedding vector, and the first negative sample embedding vector includes a first text embedding vector and a first negative sample intention embedding vector. Similarly, using the text embedding model The second training sample is converted into a second positive sample embedding vector and a second negative sample embedding vector; wherein the second positive sample embedding vector includes a second text embedding vector and a second positive sample intention embedding vector, and the second negative sample embedding vector includes a second text embedding vector and a second negative sample intention embedding vector. Taking the positive sample of any training sample as an example, its conversion into the first positive sample embedding vector is as follows:

[0060] ;

[0061] ;

[0062] In the formula, represents the text embedding vector, represents the positive sample intention embedding vector, Represents the parameters of the text embedding model.

[0063] In another specific embodiment, other embedding methods may be used to respectively obtain the first embedding vector of the first training sample and the second embedding vector of the second training sample, for example, the first embedding vector of the first training sample and the second embedding vector of the second training sample may be respectively obtained by using a Bert model.

[0064] In this embodiment, the iterative training of the initial intent scorer using the first embedding vector includes: determining the initial intent scorer as the current intent scorer, and inputting the first positive sample embedding vector and the first negative sample embedding vector into the target loss function to obtain a first loss value of the current intent scorer; based on a first gradient descent algorithm, and using the first loss value to update the target parameters of the current intent scorer. The label is 1, negative sample The label is 0. In the embedding space, contrastive learning converts the positive sample The distance is shortened, and the negative samples The distance is increased, and further, the first positive sample embedding vector and the first negative sample embedding vector are input into the target loss function to obtain the first loss values ​​of the current intent scorer, and then based on the first gradient descent algorithm, the target parameters of the current intent scorer are updated using the first loss values; wherein the target parameters can specifically be the weights of the embedding vectors; the process of minimizing the loss value is the process of feature learning of the intent scorer, that is, increasing the distance between positive samples and reducing the distance between negative samples, and learning the common features of the data by comparing a large number of sample pairs. These features can capture the basic structure and pattern of the data, rather than data features specific to a certain class. Therefore, when encountering new unseen classes, these common features can still effectively represent the new classes, thereby improving the generalization performance.

[0065] In this embodiment, the iterative training of the first trained intent scorer using the second embedding vector includes: determining the first trained intent scorer as the current trained intent scorer, inputting the second positive sample embedding vector and the second negative sample embedding vector into the target loss function to obtain the second loss value of the current intent scorer; based on the second gradient descent algorithm, the target parameter of the current trained intent scorer is updated using the second loss value. Similarly, when the first trained intent scorer is iteratively trained using the second embedding vector, the second positive sample embedding vector and the second negative sample embedding vector are respectively input into the target loss function to obtain the second loss value of the current intent scorer; based on the second gradient descent algorithm, the target parameter of the current trained intent scorer is updated using the second loss value; in contrastive learning, the proposed intent scorer is trained to distinguish similar and dissimilar samples in the feature space, and the representation ability of the training scorer is significantly enhanced, which can well capture the subtle differences and complex structures of the data. This enhanced representation ability enables the model to still have strong discrimination ability when processing unseen classes. The model can use the support set (i.e., the visible class dataset) to train model parameters in the meta-contrastive learning process, and use the query dataset (i.e., the invisible class dataset) to update parameters on the invisible class. This meta-learning process enables the model to quickly adapt to new intent scenarios, effectively solving the problem of the model mistakenly identifying the invisible class as the visible class during the intent recognition process, thereby improving the accuracy of the model in simultaneously identifying the visible and invisible classes.

[0066] In this embodiment, the objective loss function is:

[0067] ;

[0068] in, is the text embedding vector, is the positive sample intent embedding vector, is the negative sample intent embedding vector, is the temperature parameter, and K is the number of negative samples corresponding to the text embedding vector.

[0069] Understandably, is the temperature parameter, which is used to adjust the smoothness of the distribution and the target loss function The minimization process is the feature learning process of the intent scorer, that is, increasing the distance between positive samples and reducing the distance between negative samples.

[0070] In this embodiment, the first gradient descent algorithm is:

[0071] ;

[0072] in, is the target parameter after updating the current intention scorer, is the target parameter of the current intent scorer, is the learning rate of the first gradient descent algorithm, is the visible class dataset, is the i-th training task, is the distribution of the training tasks, is the sum of the first loss values.

[0073] For the training set , based on the first gradient descent method to update the model parameters of the intent scorer, different from the traditional MAML (Model-Agnostic Meta-Learning) algorithm, the meta-adaptation process (i.e., meta-training process) is targeted at the task set (Visible classes) are optimized instead of each task.

[0074] In this embodiment, the second gradient descent algorithm is:

[0075] ;

[0076] in, is the target parameter after updating the currently trained intent scorer, is the learning rate of the second gradient descent algorithm, is the non-visible class dataset, is the sum of the second loss values.

[0077] For the test set , based on the second gradient descent method to update the model parameters of the intent scorer, in this stage, on the test set Update parameters ,It should be noted that the types of the training set and the test set are ,disjoint, so in the meta-optimization phase (i.e., the meta-testing phase), the model parameters ,will generalize from the seen class to the unseen class.

[0078] It should be noted that the meta-contrastive learning objective is to learn the association between utterances and intentions. Through model training, the intent recognition scorer will assign a score between 0 and 1 to "utterance-intent". The learning process model is more oriented towards labels (intents) rather than the distribution of specific data sets. Therefore, under the target loss function, the model learns the ability to distinguish new intents from corresponding utterances.

[0079] Step S12: Obtain each predicted intent score of the current text to be recognized output by the target intent scorer.

[0080] It can be understood that the embedding vector of the current text to be recognized is input into the trained target intent scorer, and the target intent scorer outputs the predicted intent scores of the current text to be recognized. Even if the current text to be recognized belongs to a new intent, the target intent scorer can predict the intent and score of the current text to be recognized.

[0081] Step S13: using a probability distribution function to convert each predicted intention score into the probability of each predicted intention, and selecting the predicted intention with the largest probability value from the probabilities of each predicted intention as the final intention of the current text to be recognized.

[0082] The probability distribution function is used to convert each predicted intent score into the probability of each predicted intent. Specifically, the softmax function can be used to convert each predicted intent score into the probability of each predicted intent, so as to select the predicted intent with the largest probability value from the probabilities of various predicted intents as the final intent of the current text to be recognized.

[0083] The beneficial effects of the present application are as follows: the present application inputs the embedding vector of the current text to be recognized into a target intent scorer; wherein the target intent scorer is a contrastive learning network model obtained by performing meta-contrastive learning using a visible class data set and an invisible class data set respectively, and there is no correlation between the categories of the visible class data set and the categories of the invisible class data set; the predicted intent scores of the current text to be recognized output by the target intent scorer are obtained; the probability distribution function is used to convert each of the predicted intent scores into the probability of each predicted intent, and the predicted intent with the largest probability value is selected from the probabilities of each of the predicted intents as the final intent of the current text to be recognized. It can be seen that the target intent scorer of the present application is a contrastive learning network model obtained by meta-contrastive learning using visible class data sets and invisible class data sets respectively, and there is no correlation between the categories of the visible class data sets and the categories of the invisible class data sets. That is to say, unlike the traditional meta-learning setting, the present application uses visible class data sets and invisible class data sets with no correlation between categories to perform meta-contrastive learning on the model during the model training process, which can enable the model to learn the characteristics of new categories faster, that is, the model parameters can be generalized from visible classes to invisible classes, so the final target intent scorer can recognize new intents more accurately; further, the various predicted intent scores of the current text to be recognized obtained by using the target intent scorer are converted into probabilities of each predicted intent, and the predicted intent with the largest probability value is selected from the probabilities of each predicted intent as the final intent of the current text to be recognized. Since the target intent scorer can accurately recognize new intents, it provides a strong guarantee for the subsequent determination of the final intent of the current text to be recognized, and there is no need to introduce additional information, which makes the cost of new intent recognition lower, the efficiency higher and the accuracy higher.

[0084] Below Figure 2 Taking the specific intent scorer training schematic diagram shown as an example, the acquisition process of the target intent scorer in this application is explained accordingly.

[0085] 1) Construct a visible class dataset and an invisible class dataset; wherein the categories of the visible class dataset and the invisible class dataset are disjoint, unrelated, and non-overlapping.

[0086] 2) Use the visible class dataset to train the intent scorer stage:

[0087] 2.1) Construct the first training sample of the visible class data set, the first training sample includes multiple positive and negative sample pairs, and each positive and negative sample pair includes a single positive sample and multiple negative samples corresponding to the positive sample, wherein the positive sample includes the sampled text and the true intent label corresponding to the sampled text, and the negative sample includes the sampled text and other intent labels of the sampled text; in this way, the target loss function can be made to perform contrastive learning for the intent label for the contrastive learning target, and learn the association between "utterance-intent" instead of the relationship between "utterance-utterance", and subsequently through contrastive learning, the distance between "utterance-intent" (positive sample pairs) with high KL divergence is shortened, and the distance between "utterance-intent" (negative sample pairs) with low KL divergence is extended;

[0088] 2.2) Obtaining a first embedding vector of a first training sample, that is, using a text embedding model to convert the first training sample into a first positive sample embedding vector and a first negative sample embedding vector;

[0089] 2.3) Meta-training the initial intent scorer using the first embedding vector to obtain a first trained intent scorer, specifically including:

[0090] 2.3.1) Determine the initial intent scorer as the current intent scorer, and input the first positive sample embedding vector and the first negative sample embedding vector into the target loss function to obtain a first loss value of the current intent scorer;

[0091] 2.3.2) Based on the first gradient descent algorithm, the target parameters of the current intent scorer are updated using the first loss value to perform a new round of iterative training.

[0092] 3) Use the unseen dataset to train the intent scorer:

[0093] 3.1) Constructing a second training sample of the non-visible class data set, the second training sample includes multiple positive and negative sample pairs, and each positive and negative sample pair includes a single positive sample and multiple negative samples corresponding to the positive sample, wherein the positive sample includes the sampled text and the true intent label corresponding to the sampled text, and the negative sample includes the sampled text and other intent labels of the sampled text;

[0094] 3.2) Obtaining a second embedding vector of the second training sample, that is, using the text embedding model to convert the second training sample into a second positive sample embedding vector and a second negative sample embedding vector;

[0095] 3.3) Meta-test the first trained intent scorer using the second embedding vector to obtain the second trained intent scorer. This step uses the meta-learning optimization strategy to adjust the model parameters so that the model performs well on the new task set. (unseen classes) to achieve good performance, including:

[0096] 3.3.1) Determine the first trained intent scorer as the current trained intent scorer, and input the second positive sample embedding vector and the second negative sample embedding vector into the target loss function to obtain a second loss value of the current trained intent scorer;

[0097] 3.3.2) Based on the second gradient descent algorithm, the target parameters of the currently trained intent scorer are updated using the second loss value to perform a new round of iterative training. By simulating the process of generalized zero-shot learning, that is, setting the data set and Disjoint, the model can use the support set to train the model parameters in the meta-contrastive learning process, and use the query data set to update the parameters on the unseen class. This meta-learning process enables the model to quickly adapt to new intent scenarios, effectively solving the problem of the model mistakenly identifying the unseen class as the visible class during the intent recognition process, thereby improving the accuracy of the model in simultaneously identifying the visible and unseen classes.

[0098] 4) Determine the second trained intent scorer as the target intent scorer.

[0099] First, the proposed meta-contrastive learning enables the intent recognition scorer to learn more general features for label intent rather than the distribution and features of a certain data set; secondly, the model further optimizes learning through a meta-learning framework. Different from the traditional meta-learning setting, this embodiment simulates the real generalized zero-shot intent recognition process by setting the training set and the test set categories to be disjoint, so that the model can learn the features of new categories faster, optimize model parameters, and generalize the features to unseen (new) classes. Therefore, the proposed generalized zero-shot intent recognition model training method can improve the versatility and generalization of the model in the generalized zero-shot scenario, effectively solve the intent recognition bias problem in this scenario, and improve the accuracy of the model's unknown intent recognition.

[0100] See also Figure 3 As shown, the embodiment of the present application discloses an intention recognition device, including:

[0101] A text input module 11 is used to input the embedding vector of the current text to be recognized into a target intent scorer; wherein the target intent scorer is a contrastive learning network model obtained by performing meta-contrastive learning using a visible class data set and an invisible class data set respectively, and there is no correlation between the categories of the visible class data set and the categories of the invisible class data set;

[0102] A score acquisition module 12 is used to obtain each predicted intention score of the current text to be recognized output by the target intention scorer;

[0103] A probability conversion module 13, used to convert each of the predicted intention scores into a probability of each predicted intention using a probability distribution function;

[0104] The intention determination module 14 is used to select the predicted intention with the largest probability value from the probabilities of the various predicted intentions as the final intention of the current text to be recognized.

[0105] The beneficial effects of the present application are as follows: the present application inputs the embedding vector of the current text to be recognized into a target intent scorer; wherein the target intent scorer is a contrastive learning network model obtained by performing meta-contrastive learning using a visible class data set and an invisible class data set respectively, and there is no correlation between the categories of the visible class data set and the categories of the invisible class data set; the predicted intent scores of the current text to be recognized output by the target intent scorer are obtained; the probability distribution function is used to convert each of the predicted intent scores into the probability of each predicted intent, and the predicted intent with the largest probability value is selected from the probabilities of each of the predicted intents as the final intent of the current text to be recognized. It can be seen that the target intent scorer of the present application is a contrastive learning network model obtained by meta-contrastive learning using visible class data sets and invisible class data sets respectively, and there is no correlation between the categories of the visible class data sets and the categories of the invisible class data sets. That is to say, unlike the traditional meta-learning setting, the present application uses visible class data sets and invisible class data sets with no correlation between categories to perform meta-contrastive learning on the model during the model training process, which can enable the model to learn the characteristics of new categories faster, that is, the model parameters can be generalized from visible classes to invisible classes, so the final target intent scorer can recognize new intents more accurately; further, the various predicted intent scores of the current text to be recognized obtained by using the target intent scorer are converted into probabilities of each predicted intent, and the predicted intent with the largest probability value is selected from the probabilities of each predicted intent as the final intent of the current text to be recognized. Since the target intent scorer can accurately recognize new intents, it provides a strong guarantee for the subsequent determination of the final intent of the current text to be recognized, and there is no need to introduce additional information, which makes the cost of new intent recognition lower, the efficiency higher and the accuracy higher.

[0106] Furthermore, an embodiment of the present application also provides an electronic device. Figure 4 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content in the diagram cannot be regarded as any limitation on the scope of use of the present application.

[0107] Figure 4A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Specifically, it may include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 is used to store a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the intent recognition method performed by the electronic device disclosed in any of the aforementioned embodiments.

[0108] In this embodiment, the power supply 23 is used to provide working voltage for each hardware device on the electronic device; the communication interface 24 can create a data transmission channel between the electronic device and the external device, and the communication protocol it follows is any communication protocol that can be applied to the technical solution of the present application, and is not specifically limited here; the input and output interface 25 is used to obtain external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs and is not specifically limited here.

[0109] Among them, the processor 21 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 21 can be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 21 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 21 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 21 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0110] In addition, the memory 22, as a carrier for storing resources, can be a read-only memory, a random access memory, a disk or an optical disk, etc. The resources stored thereon include an operating system 221, a computer program 222 and data 223, etc. The storage method can be temporary storage or permanent storage.

[0111] Among them, the operating system 221 is used to manage and control the hardware devices and computer programs 222 on the electronic device to realize the operation and processing of the massive data 223 in the memory 22 by the processor 21, which can be Windows, Unix, Linux, etc. In addition to including a computer program that can be used to complete the intention recognition method performed by the electronic device disclosed in any of the aforementioned embodiments, the computer program 222 can further include a computer program that can be used to complete other specific tasks. In addition to data received by the electronic device and transmitted from an external device, the data 223 can also include data collected by its own input and output interface 25, etc.

[0112] Furthermore, the present application also discloses a computer-readable storage medium for storing a computer program; wherein the computer program, when executed by a processor, implements the aforementioned disclosed intention recognition method. The specific steps of the method can refer to the corresponding contents disclosed in the aforementioned embodiments, and will not be repeated here.

[0113] The present application also provides a computer program product, including a computer program and / or computer instructions, which implement the steps of the above SRv6 message transmission method when executed by a processor. Figure 5 , Figure 5 A schematic diagram of the implementation principle of a computer program product provided in an embodiment of the present application, the computer program product includes a text input program a, a score acquisition program b, a probability conversion program c, and an intention determination program d. When the text input program a is executed by the processor, the operation implemented is: inputting the embedding vector of the current text to be recognized into the target intention scorer; wherein the target intention scorer is a contrastive learning network model obtained by performing meta-contrast learning using a visible class data set and an invisible class data set respectively, and there is no correlation between the categories of the visible class data set and the categories of the invisible class data set. When the score acquisition program b is executed by the processor, the operation implemented is: obtaining the predicted intention scores of the current text to be recognized output by the target intention scorer. When the probability conversion program c is executed by the processor, the operation implemented is: using a probability distribution function to convert each of the predicted intention scores into the probability of each predicted intention. When the intention determination program d is executed by the processor, the operation implemented is: selecting the predicted intention with the largest probability value from the probabilities of each of the predicted intentions as the final intention of the current text to be recognized.

[0114] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0115] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented with electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application. The steps of the method or algorithm described in conjunction with the embodiments disclosed herein can be implemented directly with hardware, a software module executed by a processor, or a combination of the two. The software module can be placed in a random access memory (RAM), memory, read-only memory (ROM), electrically programmable EPROM (Erasable Programmable Read Only Memory), electrically erasable programmable EEPROM (Electrically Erasable Programmable read only memory), register, hard disk, removable disk, CD-ROM (Compact Disc Read-Only Memory), or any other form of storage medium known in the technical field.

[0116] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0117] The above is a detailed introduction to the intent recognition method, device, equipment and medium provided by the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for general technical personnel in the field, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A method for intention recognition, characterized in that: include: Inputting the embedding vector of the current text to be recognized into a target intent scorer; wherein the target intent scorer is a contrastive learning network model obtained by performing meta-contrastive learning using a visible class dataset and an invisible class dataset respectively, and there is no correlation between the categories of the visible class dataset and the categories of the invisible class dataset; Obtaining each predicted intent score of the current text to be recognized output by the target intent scorer; The probability distribution function is used to convert each of the predicted intention scores into the probability of each predicted intention, and the predicted intention with the largest probability value is selected from the probabilities of each of the predicted intentions as the final intention of the current text to be recognized.

2. The method for intention recognition according to claim 1, characterized in that: Get the target intent scorer, including: Constructing a training task including a visible class data set and a non-visible class data set, and constructing a first training sample of the visible class data set and a second training sample of the non-visible class data set; Obtaining a first embedding vector of the first training sample and a second embedding vector of the second training sample respectively; Meta-training an initial intent scorer using the first embedding vector to obtain a first trained intent scorer; The first trained intent scorer is meta-tested by using the second embedding vector to obtain a second trained intent scorer, and the second trained intent scorer is determined as a target intent scorer.

3. The method for identifying intention according to claim 2, characterized in that: The first training sample and the second training sample both contain multiple positive and negative sample pairs, and each of the positive and negative sample pairs includes a single positive sample and multiple negative samples corresponding to the positive sample, wherein the positive sample includes a sampled text and a true intent label corresponding to the sampled text, and the negative sample includes the sampled text and other intent labels of the sampled text, and the other intent label of the sampled text is any intent label other than the true intent label of the sampled text.

4. The method for intention recognition according to claim 3, characterized in that: The obtaining a first embedding vector of the first training sample and a second embedding vector of the second training sample respectively includes: Using a text embedding model, the first training sample is converted into a first positive sample embedding vector and a first negative sample embedding vector; wherein the first positive sample embedding vector includes a first text embedding vector and a first positive sample intention embedding vector, and the first negative sample embedding vector includes the first text embedding vector and a first negative sample intention embedding vector; The second training sample is converted into a second positive sample embedding vector and a second negative sample embedding vector by using the text embedding model; wherein the second positive sample embedding vector includes a second text embedding vector and a second positive sample intention embedding vector, and the second negative sample embedding vector includes the second text embedding vector and a second negative sample intention embedding vector.

5. The method for intention recognition according to claim 4, characterized in that: The meta-training of the initial intent scorer using the first embedding vector includes: Determine an initial intent scorer as a current intent scorer, and input the first positive sample embedding vector and the first negative sample embedding vector into a target loss function to obtain a first loss value of the current intent scorer; Based on a first gradient descent algorithm, and using the first loss value, updating a target parameter of the current intent scorer; Accordingly, the meta-testing the first trained intent scorer using the second embedding vector includes: Determine the first trained intent scorer as the current trained intent scorer, and input the second positive sample embedding vector and the second negative sample embedding vector into the target loss function to obtain a second loss value of the current intent scorer; Based on a second gradient descent algorithm, target parameters of the currently trained intent scorer are updated using the second loss value.

6. The method for intention recognition according to claim 5, characterized in that: The objective loss function is: ; in, is the text embedding vector, is the positive sample intent embedding vector, is the negative sample intent embedding vector, is the temperature parameter, and K is the number of negative samples corresponding to the text embedding vector.

7. The method for intention recognition according to claim 5, characterized in that: The first gradient descent algorithm is: ; in, is the target parameter after updating the current intention scorer, is the target parameter of the current intent scorer, is the learning rate of the first gradient descent algorithm, is the visible class dataset, is the i-th training task, is the distribution of the training tasks, is the sum of the first loss values; Accordingly, the second gradient descent algorithm is: ; in, is the target parameter after updating the currently trained intent scorer, is the learning rate of the second gradient descent algorithm, is the non-visible class dataset, is the sum of the second loss values.

8. An intention recognition device, characterized in that: include: A text input module, used to input the embedding vector of the current text to be recognized into a target intent scorer; wherein the target intent scorer is a contrastive learning network model obtained by performing meta-contrastive learning using a visible class data set and an invisible class data set respectively, and there is no correlation between the categories of the visible class data set and the categories of the invisible class data set; A score acquisition module, used to obtain each predicted intention score of the current text to be recognized output by the target intention scorer; A probability conversion module, used to convert each of the predicted intention scores into the probability of each predicted intention using a probability distribution function; The intention determination module is used to select the predicted intention with the largest probability value from the probabilities of each predicted intention as the final intention of the current text to be recognized.

9. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor, configured to execute the computer program to implement the steps of the intent recognition method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: Used to store computer programs; wherein, when the computer program is executed by a processor, the steps of the intention recognition method as described in any one of claims 1 to 7 are implemented.

11. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the intention recognition method as claimed in any one of claims 1 to 7 are implemented.

Citation Information

Patent Citations

  • Contrast task adaptive learning method for few-sample intention recognition

    CN113516209A

  • Multi-modal intention recognition method and system based on comparative learning

    CN116304984A