Intention recognition method and device, and electronic device

Abstract

The application discloses an intention recognition method, belonging to the technical field of computers, and used for improving intention recognition accuracy. The intention recognition method comprises the following steps: performing encoding processing on a to-be-recognized query input through a preset encoder to determine feature encoding of the to-be-recognized query input; performing classification mapping on the feature encoding of the to-be-recognized query input through a pre-trained intention classification model to determine a probability that the to-be-recognized query input matches a preset intention category; wherein the intention classification model shares model parameters with an intention classification task in a pre-trained multi-task model, the multi-task model is trained in combination with association between information of intention categories and query inputs and semantic matching relationship between intention categories and query inputs; and according to the probability that the to-be-recognized query input matches the preset intention category, the preset intention category matched by the to-be-recognized query input is determined, so that the accuracy of intention recognition is improved.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of computer, in particular to an intent recognition method and device, electronic equipment and computer readable storage medium. BACKGROUND

[0002] With the rapid development of artificial intelligence and deep learning technology, intent recognition based on deep learning model has become a current research trend, and its application scenarios are also increasing. Taking a task-oriented dialogue system as an example, generally speaking, a task-oriented dialogue system includes multiple modules such as speech recognition, intent recognition, dialogue management, natural language generation, and speech synthesis. As an important link in the dialogue system, the accuracy of the recognition result of intent recognition directly affects the performance and user experience of the dialogue system. In the prior art, the common process of intent recognition is: after encoding the user input to obtain the encoding vector of the user input, the encoding vector is classified and processed by an intent classification module to obtain an intent category matched with the user input. In the prior art, generally, the intent classification model is a single-task classification model trained based on training samples constructed from user input and intent categories, and the accuracy of intent recognition needs to be improved.

[0003] The applicant has noticed that there is a neural network model including two tasks of intent recognition and entity recognition in the prior art, and the intent recognition task and the entity recognition task can share part of the model parameters, the purpose of which is to improve the accuracy of intent recognition. However, at least the following defects exist in the intent recognition method in the prior art: there is no necessary connection between intent and entity, and the effect of improving the accuracy of intent recognition by entity training data is unstable.

[0004] It can be seen that the intent recognition method in the prior art still needs to be improved. SUMMARY

[0005] Embodiments of the present application provide an intent recognition method, which helps to improve the accuracy of intent recognition.

[0006] In a first aspect, the embodiments of the present application provide an intent recognition method, comprising:

[0007] encoding the to-be-recognized query input by a preset encoder to determine the feature encoding of the to-be-recognized query input;

[0008] The feature encoding of the to-be-recognized query input is classified and mapped by a pre-trained intent classification model to determine a probability that the to-be-recognized query input matches a preset intent category; wherein the intent classification model shares model parameters with an intent classification task in a pre-trained multi-task model, and the multi-task model is trained in combination of an association between an intent category and information of a query input and a semantic matching relationship between an intent category and a query input.

[0009] The preset intent category matched by the to-be-recognized query input is determined according to the probability that the to-be-recognized query input matches the preset intent category.

[0010] In a second aspect, an embodiment of the present application provides an intent recognition device, comprising:

[0011] An encoding module is configured to perform encoding processing on a to-be-recognized query input by a preset encoder to determine feature encoding of the to-be-recognized query input.

[0012] An intent category matching probability determination module is configured to perform classification mapping on the feature encoding of the to-be-recognized query input by a pre-trained intent classification model to determine a probability that the to-be-recognized query input matches a preset intent category; wherein the intent classification model shares model parameters with an intent classification task in a pre-trained multi-task model, and the multi-task model is trained in combination of an association between an intent category and information of a query input and a semantic matching relationship between an intent category and a query input.

[0013] An intent category determination module is configured to determine the preset intent category matched by the to-be-recognized query input according to the probability that the to-be-recognized query input matches the preset intent category.

[0014] In a third aspect, an embodiment of the present application further discloses an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the intent recognition method of the embodiment of the present application.

[0015] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium having a computer program stored thereon, wherein the program is executable on a processor to implement the steps of the intent recognition method disclosed in the embodiment of the present application.

[0016] The intention recognition method disclosed by the embodiments of the present application encodes the to-be-recognized query input through a preset encoder to determine the feature encoding of the to-be-recognized query input; classifies and maps the feature encoding of the to-be-recognized query input through a pre-trained intention classification model to determine the probability that the to-be-recognized query input matches a preset intention category; wherein the intention classification model shares model parameters with an intention classification task in a pre-trained multi-task model, the multi-task model is trained in combination of the association between the intention category and the information of the query input, and the semantic matching relationship between the intention category and the query input; and according to the probability that the to-be-recognized query input matches the preset intention category, the preset intention category matched by the to-be-recognized query input is determined, which helps to improve the accuracy of intention recognition.

[0017] The above description is only a summary of the technical solutions of the present application. In order to more clearly understand the technical means of the present application, the embodiments can be implemented according to the content of the description, and in order to make the above and other purposes, characteristics and advantages of the present application more obvious and easy to understand, the following specific embodiments of the present application are described. BRIEF DESCRIPTION OF DRAWINGS

[0018] In order to make the purposes, technical solutions and advantages of the embodiments of the present application more clear, the technical solutions in the embodiments of the present application will be described clearly and completely below in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0019] Figure 1 is one of the intention recognition method flowcharts of the first embodiment of the present application;

[0020] Figure 2 is the second intention recognition method flowchart of the first embodiment of the present application;

[0021] Figure 3 is a multi-task model structure diagram in the intention recognition method of the first embodiment of the present application;

[0022] Figure 4 is one of the intention recognition device structure diagrams of the second embodiment of the present application;

[0023] Figure 5 is the second intention recognition device structure diagram of the second embodiment of the present application;

[0024] Figure 6 a block diagram of an electronic device for performing the method according to the present application is schematically shown; and

[0025] Figure 7A storage unit for holding or carrying program code implementing the method according to the application is schematically shown. DETAILED DESCRIPTION

[0026] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some of the embodiments of the present application, but not all of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of the present application.

[0027] Embodiment one

[0028] The intent recognition method disclosed in the embodiments of the present application is as shown in the following steps. Figure 1 The method comprises steps 110 to 130.

[0029] In step 110, a pre-set encoder is used to encode the input query to be recognized, so as to determine the feature encoding of the input query to be recognized.

[0030] The pre-set encoder in the embodiments of the present application can be a general text encoding network model in the prior art. For example, the pre-set encoder can be a BERT model in the prior art, or a Word2Vec (word vector) model, a CNN (Convolutional Neural Network, convolutional neural network model), an RNN (Recurrent Neural Network, recurrent neural network model), an LSTM (long-short term memory, long-short term memory model), etc.

[0031] In some embodiments of the present application, the pre-set encoder is an encoding network module in the backbone network structure of a pre-trained multi-task model. For example, the base network part of the multi-task model can be implemented based on the BERT model structure or other neural network model structure to realize the pre-set encoder.

[0032] In the model training stage and the online prediction stage, the encoder is used to encode the input text and output the vector representation obtained after the encoding processing. That is, the pre-set encoder encodes the query input text and outputs the feature encoding of the query input text, and the pre-set encoder encodes the intent category text and outputs the feature encoding of the intent category text.

[0033] In step 120, a pre-trained intent classification model is used to classify and map the feature encoding of the input query to be recognized, so as to determine the probability that the input query to be recognized matches a pre-set intent category.

[0034] The intent classification model shares model parameters with an intent classification task in a pre-trained multi-task model, the multi-task model being trained in combination with an association between intent categories and information of query inputs and a semantic matching relationship between intent categories and query inputs.

[0035] As shown in Figure 2 The method further includes, before the step of encoding the to-be-identified query input by using the preset encoder to determine the feature encoding of the to-be-identified query input, the step 100.

[0036] In the step 100, the multi-task model is trained according to a plurality of matching combinations of query inputs and intent categories.

[0037] In some embodiments of the present application, a plurality of query input texts and a plurality of intent category texts are defined in advance, and a matching relationship between the query input texts and the intent category texts is determined according to business requirements; or, the matching relationship between the query input texts and the intent category texts is determined according to historical query and click behavior data of users; and then, a plurality of matching combinations of query inputs and intent categories are established according to the matching relationship between the query input texts and the intent category texts.

[0038] Further, the matching relationship between the query inputs and the intent categories in the plurality of matching combinations and the association degree of the semantic information of the query inputs and the intent categories can be learned by training the multi-task model, so as to learn a mapping relationship from the query inputs to the intent categories, which is used to determine an intent category matched by a specified query input according to the mapping relationship. The multi-task model includes two tasks: an intent classification task and an intent matching task, each task has a different learning objective, and the parameter update process of each task is performed according to the learning results of the two tasks.

[0039] In some embodiments of the present application, the pre-trained intent classification model is constructed based on a pre-trained multi-task model. As shown in Figure 3 The multi-task model structure is taken as an example, Figure 3 The multi-task model structure includes an encoder 310, an intent classification network branch 320 and an intent matching network branch 330, and a multi-task fusion layer 340. In the embodiments of the present application, the intent classification model and the intent classification network branch 320 adopt the same network structure and share network parameters.

[0040] The training process of the intent classification network branch 320 will be described below by taking the network structure of the multi-task model shown in Figure 3 The training process of the intent classification model is essentially the training process of the intent classification network branch 320.

[0041] As Figure 3 shown in the network structure of the multi-task model, the multi-task model includes two branch networks, i.e., an intent classification network branch 320 and an intent matching network branch 330, each of which is used to implement different tasks, wherein the intent classification network branch 320 is used to implement an intent classification task, and the intent matching network branch 330 is used to implement an intent matching task. The multi-task fusion layer 340 calculates the loss value of the multi-task model by fusing the loss values of the intent classification network branch 320 and the intent matching network branch 330. In the embodiment of the present application, by jointly training the intent classification task and the intent matching task in the multi-task model, in the training process of the intent classification task, in addition to learning the semantic matching relationship between the query input and the intent category, the association between the information of the intent category and the query input is further combined for parameter optimization.

[0042] In some embodiments of the present application, training the multi-task model according to a plurality of matching combinations of query inputs and intent categories includes: respectively encoding the query input and the intent category in each matching combination by the preset encoder to determine the feature encoding of the query input and the feature encoding of the intent category; estimating the probability distribution of matching the feature encoding of the query input in the matching combination with a preset intent category by the intent classification task to determine a first loss value; estimating the semantic similarity between the feature encoding of the query input in the matching combination and the feature encoding of the intent category by the intent matching task to determine a second loss value; fusing the first loss value and the second loss value to obtain the loss value of the multi-task model; optimizing the loss value of the multi-task model by optimizing the network parameters of the intent classification task and the intent matching task to iteratively train the multi-task model.

[0043] In the training process of the multi-task model, first, the encoder 310 is used to respectively encode the query input and the intent category in each matching combination to respectively determine the feature encoding of the query input and the feature encoding of the intent category in each matching combination.

[0044] Then, the output of the encoder 310 is respectively processed by the intent classification task and the intent matching task for feature mapping to determine the loss value corresponding to the respective task.

[0045] For example, the feature encoding of the query input in each matching combination is input to the intent classification network branch 320, each feature encoding of the input is classified and mapped respectively by performing the intent classification task, the probability distribution of each feature encoding matching the preset intent category is estimated, and then the first loss value of the intent classification task is determined according to the error between the estimated probability distribution of each feature encoding matching the preset intent category and the true probability distribution of the preset intent category matching the query input.

[0046] For example, the feature encoding of the query input in each matching combination and the feature encoding of the intent category are input to the intent matching network branch 330, similarity calculation is performed on the feature encoding of the query input and the intent category that have a matching relationship, and similarity calculation is performed on the feature encoding of the query input and the intent category that do not have a matching relationship, and the second loss value of the intent matching task is determined according to the similarity calculation result related to each matching combination.

[0047] Finally, the first loss value and the second loss value are fused to obtain the loss value of the multi-task model. The network parameters of the intent classification task and the intent matching task are optimized to optimize the loss value as the target, and the multi-task model is iteratively trained.

[0048] In some embodiments of the present application, the network parameters of the encoder 310 are also optimized in the model parameter optimization process. For example, the encoder 310 can adopt the BERT model commonly used in the prior art, and by optimizing the parameters of the encoder 310, the encoder 310 can be more suitable for text encoding in the intent recognition scenario during the multi-task model training process.

[0049] In some embodiments of the present application, the probability distribution of the feature encoding of the query input in the matching combination matching the preset intent category is estimated by the intent classification task to determine the first loss value, comprising: the probability estimation value of each query input matching the preset intent category is determined by classifying and mapping the feature encoding of the query input in the matching combination by the intent classification task; the first loss value is determined according to the probability estimation value of each query input matching the preset intent category and the probability true value of the query input matching the preset intent category, wherein the probability true value of the query input matching the preset intent category is determined according to the intent category matching the query input.

[0050] In the embodiments of the present application, the intent classification network branch 320 adopts a classification network structure, and the intent classification task determines the probability value of the input feature encoding matching each intent category by performing multi-class mapping on the input feature encoding.

[0051] Taking an example of preset intent categories including C standard intent categories, a C-dimensional vector can be used to identify a probability true value of each query input matching the preset intent categories, wherein each dimension of the vector represents a probability of the query input matching the intent category corresponding to the dimension. If it is determined according to the above matching combination that a query input matches intent category 1, the value of the vector corresponding to intent category 1 in the above C-dimensional vector is set to 1, and the values of the vectors corresponding to other intent categories in the above C-dimensional vector are set to 0, and the obtained C-dimensional vector can be used to identify the probability true value of the query input matching the preset intent categories.

[0052] In the process of training the multi-task model, after the encoding features of each query input in the matching combination are input to the intent classification network branch 320, the intent classification task will further perform classification mapping on the feature encoding to obtain a C-dimensional vector, which is used to indicate the probability estimate value of the input feature encoding matching the preset C intent categories.

[0053] After obtaining the probability estimate value of each query input matching different intent categories respectively, the probability estimate value of each query input matching the preset intent categories can be converted into a first loss value of the intent classification task using cross entropy. In some embodiments of the present application, the first loss value loss1 of the intent classification task can be determined, for example, by the following formula:

[0054] Wherein, C represents the number of intent categories, N represents the number of matching combinations, i and j represent the identification of query input and intent category respectively, represents the probability true value of the ith query input matching the jth intent category, represents the probability estimate value of the ith query input matching the jth intent category.

[0055] In another embodiment of the present application, the cumulative error between the probability estimate value of each query input matching the preset intent categories and the probability true value of the query input matching the preset intent categories can also be determined as the first loss value of the intent classification task.

[0056] The present application does not limit the specific implementation of determining the first loss value of the intent classification task, and does not enumerate them one by one here.

[0057] In some embodiments of this application, the step of estimating the semantic similarity between the feature code of the query input and the feature code of the intent category in the matching combination through the intent matching task, and determining a second loss value, includes: selecting at least some of the matching combinations, and performing the following similarity estimation operations respectively: using the query input and the intent category in the same matching combination as positive examples, and the query input and the intent category in different matching combinations as negative examples, determining the semantic similarity between the feature code of the query input and the feature code of the intent category in each positive and negative example through the intent matching task; and determining a second loss value based on the determined semantic similarity.

[0058] For example, by downsampling the several matching combinations, multiple batches of matching combinations are determined. Then, for each batch of matching combinations, each matching combination in that batch is used as a positive example, and the query input in one of the matching combinations in that batch is combined with the intent category in other matching combinations to form negative examples, thus obtaining several negative examples. Then, the similarity between the feature code of the query input and the feature code of the intent category in each positive example is calculated by the intent matching task, and the semantic similarity between the feature code of the query input and the feature code of the intent category in each negative example is also calculated. The semantic similarity between the feature code of the query input and the feature code of the intent category reflects the semantic similarity between the corresponding query input and the intent category. Finally, based on the semantic similarity between the query input and the intent category in each positive example and the semantic similarity between the query input and the intent category in each negative example, the second loss value of the intent matching task is determined.

[0059] In some embodiments of this application, cosine similarity can be used to represent the semantic similarity between the feature encoding of the query input and the feature encoding of the intent category. For example, through formula s ij =cos(u i ,l j Calculate the feature code u of the i-th query input. i and the feature encoding of the j-th intention category l j semantic similarity between ij In some other embodiments of this application, the feature encoding u of the i-th query input can be first performed. i and the feature encoding of the j-th intention category l j The concatenated vectors are then mapped using a multilayer perceptron (MLP) to obtain the feature encoding u of the i-th query input. i and the feature encoding of the j-th intention category l j The semantic similarity score.

[0060] In other embodiments of this application, the feature code u of the i-th query input can also be calculated in other ways. i and the feature encoding of the j-th intention category l j The semantic similarity between the query input feature code and the intent category feature code will not be listed in detail in this application. This application does not limit the specific implementation method for calculating the semantic similarity between the feature code of the query input and the feature code of the intent category.

[0061] Since the goal of the intent matching task is to determine the semantic similarity between the query input and the intent category, the optimization objective of the intent matching network branch 330 should be to maximize the semantic similarity between the query input and the intent category in positive matching examples, or to maximize the semantic similarity between the query input and the intent category in positive matching examples and minimize the semantic similarity between the query input and the intent category in negative matching examples. Therefore, in some embodiments of this application, when determining the second loss value based on the determined semantic similarities, the sum of the semantic similarities between the feature codes of the query input and the feature codes of the intent category in each positive matching example can be used as the second loss value. In other embodiments of this application, when determining the second loss value based on the determined semantic similarities, the second loss value can also be calculated using the Margin loss function, based on the semantic similarity between the feature codes of the query input and the feature codes of the intent category in each positive matching example, and the semantic similarity between the feature codes of the query input and the feature codes of the intent category in each negative matching example.

[0062] In some embodiments of this application, determining the second loss value based on the determined semantic similarities includes: generating a similarity matrix based on the semantic similarity between the feature encoding of the query input and the feature encoding of the intent category in each of the positive and negative matching examples; wherein the rows and columns of the similarity matrix correspond to the query input and the intent category, respectively, and the matrix elements in specified rows and columns of the similarity matrix represent the semantic similarity between the feature encodings of the corresponding query input and the intent category; the matrix elements on the diagonal of the similarity matrix represent the semantic similarity between the feature encodings of the query input and the intent category in the positive matching examples; generating an identity matrix corresponding to the similarity matrix; and determining the second loss value by performing a preset matrix operation on the similarity matrix and the identity matrix.

[0063] Taking the selection of at least X matching combinations, including X query inputs and X intent categories, as an example, the query input in the i-th matching combination is represented as Q. i Let L represent the intent category in the i-th matching combination. i Then the i-th matching combination is represented as (Qi , i ), wherein 1≤i≤X, and the X matching combinations are matching positives. Then, for each matching positive, the query input in the matching positive is respectively matched with the intent categories in the other X-1 matching combinations to form X-1 matching negatives, for example, represented as (Q1, L2), (Q1, L3), …, (Q1, L X ), (Q2, L1), (Q2, L3), …, and then, according to the above method, the semantic similarity between the feature encoding of the query input and the feature encoding of the intent category in each matching positive can be determined, and the semantic similarity between the feature encoding of the query input and the feature encoding of the intent category in each matching negative can be determined. In an embodiment of the present application, the semantic similarity between the feature encoding u i of the query input Q i and the feature encoding l i of the intent category L i is represented as S ii .

[0064] Next, a similarity matrix S[i][j] is constructed, wherein the i-th row corresponds to the semantic similarity between the feature encoding of the query input Q i and the feature encoding of each intent category, the j-th column corresponds to the semantic similarity between the feature encoding of the intent category L j and the feature encoding of each query input, and the matrix element in the i-th row and the j-th column of the similarity matrix S[i][j] represents the semantic similarity between the feature encoding u i of the query input Q j and the feature encoding l i of the intent category L i . As can be seen, the matrix elements on the diagonal of the similarity matrix represent the semantic similarity between the feature encoding of the query input and the intent category in the matching positive.

[0065] Since the optimization goal of the intent matching task is to maximize the semantic similarity corresponding to the matching positive, in some embodiments of the present application, a unit matrix with the same shape as the similarity matrix can be generated as the label of the intent matching task, and finally, the second loss value is calculated by performing a preset matrix operation on the similarity matrix and the unit matrix. For example, the second loss value is calculated by multiplying the similarity matrix and the unit matrix, and calculating the accumulation sum of each matrix element in the matrix obtained after multiplication, and taking the accumulation sum as the second loss value. In other embodiments of the present application, the second loss value can also be calculated by performing other matrix operations on the similarity matrix and the unit matrix, so that the second loss value calculated can reflect the goal of maximizing the sum of the elements on the diagonal of the similarity matrix.

[0066] After the training of the multi-task model is completed, the network part including the encoder 310 and the intent classification network branch 320 in the multi-task model is used for the intent recognition stage, and the intent classification network branch 320 serves as an intent recognition model.

[0067] In the intent recognition stage, after an input query to be recognized is input into the encoder, the encoder outputs a feature encoding of the input query to be recognized, and then the feature encoding of the input query to be recognized is input into the intent recognition model, which outputs a C-dimensional vector corresponding to the feature encoding. The value of each dimension of the C-dimensional vector is used to indicate the matching probability of the input query to be recognized and the intent category corresponding to the dimension.

[0068] In step 130, the preset intent category matched by the input query to be recognized is determined according to the probability that the input query to be recognized matches the preset intent category.

[0069] As described above, the value of each dimension of the C-dimensional vector output by the intent recognition model is used to indicate the matching probability of the input query to be recognized and the intent category corresponding to the dimension. In some embodiments of the present application, the intent category corresponding to the maximum matching probability can be taken as the intent category matched by the input query to be recognized.

[0070] The intent recognition method disclosed in the embodiments of the present application determines the feature encoding of the input query to be recognized by encoding the input query to be recognized through a preset encoder. The probability that the input query to be recognized matches a preset intent category is determined by classifying and mapping the feature encoding of the input query to be recognized through a pre-trained intent classification model. The intent classification model shares model parameters with an intent classification task in a pre-trained multi-task model. The multi-task model is trained in combination with the association between the information of intent categories and query inputs and the semantic matching relationship between intent categories and query inputs. The preset intent category matched by the input query to be recognized is determined according to the probability that the input query to be recognized matches the preset intent category, which helps to improve the accuracy of intent recognition.

[0071] The intent recognition method disclosed in the embodiments of the present application learns the association between the information of intent categories and query inputs and the semantic matching relationship between intent categories and query inputs through the multi-task model in the model training stage. When learning the mapping relationship between query inputs and intent categories, the label semantic information of the query input is fully utilized, and the accuracy of identifying the intent category matched by the query input by using the learned mapping relationship between the query input and the intent category is effectively improved.

[0072] Further, the training data used in the multi-task model training process is a matching combination of query input and intent category, that is, only the intent category labeling is needed for the query input, and no additional data labeling is needed for other tasks in the multi-task model, reducing the labeling workload of the multi-task model training samples and improving the training efficiency of the multi-task model. That is, the intent recognition method disclosed in the embodiments of the present application not only improves the intent recognition accuracy, but also reduces the training workload of the intent recognition model.

[0073] Embodiment two

[0074] The intent recognition device disclosed in the embodiments of the present application comprises Figure 4 as shown in the figure, the device comprises:

[0075] The encoding module 410 is configured to encode the query input to be recognized by using a preset encoder to determine the feature encoding of the query input to be recognized.

[0076] The intent category matching probability determination module 420 is configured to classify and map the feature encoding of the query input to be recognized by using a pre-trained intent classification model to determine the probability that the query input to be recognized matches a preset intent category; wherein the intent classification model shares model parameters with an intent classification task in a pre-trained multi-task model, and the multi-task model is trained in combination with the association between the intent category and the information of the query input, and the semantic matching relationship between the intent category and the query input.

[0077] The intent category determination module 430 is configured to determine the preset intent category matched by the query input to be recognized according to the probability that the query input to be recognized matches the preset intent category.

[0078] In some embodiments of the present application, the multi-task model further comprises an intent matching task, as shown in the figure, the device further comprises: Figure 5

[0079] The multi-task model training module 400 is configured to train the multi-task model according to a plurality of matching combinations of query input and intent category.

[0080] In some embodiments of the present application, the multi-task model training module 400 is further configured to:

[0081] The preset encoder is used to encode the query input and the intent category in each matching combination respectively to determine the feature encoding of the query input and the feature encoding of the intent category.

[0082] The intent classification task is used to estimate the probability distribution of the feature encoding of the query input in the matching combination matching a preset intent category to determine a first loss value.​

[0083] estimate semantic similarity between the feature encoding of the query input in the matching combination and the feature encoding of the intent category through the intent matching task, to determine a second loss value;

[0084] fuse the first loss value and the second loss value to obtain a loss value of the multi-task model;

[0085] optimize the loss value of the multi-task model by optimizing network parameters of the intent classification task and the intent matching task, and iteratively train the multi-task model.

[0086] In some embodiments of the present application, the estimating, through the intent classification task, a probability distribution of the feature encoding of the query input in the matching combination matching a preset intent category to determine a first loss value comprises:

[0087] classifying and mapping, through the intent classification task, the feature encoding of the query input in the matching combination to determine a probability estimation value of each query input matching a preset intent category;

[0088] determining the first loss value according to the probability estimation value of each query input matching a preset intent category and a probability true value of the query input matching a preset intent category, wherein the probability true value of the query input matching a preset intent category is determined according to the intent category matched by the query input.

[0089] In some embodiments of the present application, the estimating, through the intent matching task, semantic similarity between the feature encoding of the query input in the matching combination and the feature encoding of the intent category to determine a second loss value comprises:

[0090] selecting at least part of the matching combinations from the plurality of matching combinations, and respectively performing the following similarity estimation operations: taking the query input and the intent category in the same matching combination as matching positive examples, taking the query input and the intent category in different matching combinations as matching negative examples, and respectively determining, through the intent matching task, semantic similarity between the feature encoding of the query input and the feature encoding of the intent category in each matching positive example and each matching negative example;

[0091] determining the second loss value according to the determined semantic similarity.

[0092] In some embodiments of the present application, the determining the second loss value according to the determined semantic similarity comprises:

[0093] generate a similarity matrix according to semantic similarities between the feature encoding of the query input and the feature encoding of the intent category in each of the matching positive examples and each of the matching negative examples; wherein rows and columns of the similarity matrix correspond to query inputs and intent categories respectively, and a matrix element of a specified row and a specified column of the similarity matrix represents a semantic similarity between the feature encoding of the corresponding query input and the intent category; and a matrix element on a diagonal line of the similarity matrix represents a semantic similarity between the feature encoding of the query input and the intent category in the matching positive example;

[0094] generate a unit matrix corresponding to the similarity matrix;

[0095] determine a second loss value by performing a preset matrix operation on the similarity matrix and the unit matrix.

[0096] The intent recognition device disclosed in the embodiments of the present application is used to implement the intent recognition method described in Embodiment One, and the specific implementation of each module of the device will not be described again, and the specific implementation of the corresponding steps of the method embodiments can be referred to.

[0097] The intent recognition device disclosed in the embodiments of the present application encodes the query input to be recognized by using a preset encoder to determine the feature encoding of the query input to be recognized; and classifies and maps the feature encoding of the query input to be recognized by using a pre-trained intent classification model to determine the probability that the query input to be recognized matches a preset intent category; wherein the intent classification model shares model parameters with an intent classification task in a pre-trained multi-task model, and the multi-task model is trained in combination with the association between the information of the intent category and the query input, and the semantic matching relationship between the intent category and the query input; and according to the probability that the query input to be recognized matches the preset intent category, the preset intent category matched by the query input to be recognized is determined, which helps to improve the accuracy of intent recognition.

[0098] The intent recognition device disclosed in the embodiments of the present application learns, by using the task model, the association between the information of the intent category and the query input, and the semantic matching relationship between the intent category and the query input in the model training stage, so that the label semantic information of the query input is fully utilized when learning the mapping relationship between the query input and the intent category, and the accuracy of identifying the intent category matched by the query input by using the learned mapping relationship between the query input and the intent category is effectively improved.

[0099] In addition, the training data used in the multi-task model training process is a matching combination of query input and intent category, that is, only the intent category labeling needs to be performed for the query input, and no additional data labeling needs to be performed for other tasks in the multi-task model, thereby reducing the labeling workload of the multi-task model training sample and improving the training efficiency of the multi-task model. That is, the intent recognition method disclosed in the embodiment of the present application not only improves the intent recognition accuracy, but also reduces the training workload of the intent recognition model.

[0100] Each of the embodiments in the specification is described in a progressive manner, and each embodiment focuses on the difference from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the part of the method embodiment.

[0101] The above describes the method and device for intent recognition provided by the present application in detail. The principle and implementation of the present application are described by using specific examples. The above embodiment is only used to help understand the method and the core idea of the present application; meanwhile, for the general technical personnel in the art, according to the idea of the present application, the specific implementation and application range can be changed. Therefore, the content of the specification should not be understood as the limitation of the present application.

[0102] The device embodiments described above are only schematic, and the units described as separate components can or can not be physically separate, and the components displayed as units can or can not be physical units, that is, they can be located in one place, or distributed on multiple network units. According to the actual needs, some or all of the modules can be selected to achieve the purpose of the embodiment. Those skilled in the art can understand and implement without creative labor.

[0103] The various component embodiments of the present application can be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. Those skilled in the art should understand that microprocessors or digital signal processors (DSP) can be used in practice to implement some or all of the functions of some or all of the components in the electronic device according to the embodiments of the present application. The present application can also be implemented as a device or apparatus program (for example, a computer program and a computer program product) for executing part or all of the methods described herein. Such program implementing the present application can be stored on a computer readable medium, or can have one or more signals in the form. Such signals can be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form.

[0104] For example, Figure 6 An electronic device is shown, which can implement the method according to the present application. The electronic device can be a PC, a mobile terminal, a personal digital assistant, a tablet computer, etc. The electronic device traditionally comprises a processor 610 and a memory 620 and program code 630 stored on the memory 620 and executable on the processor 610, which, when executed by the processor 610, implements the method described in the above embodiments. The memory 620 can be a computer program product or a computer readable medium. The memory 620 can be an electronic memory such as a flash memory, an EEPROM (electrically erasable programmable read-only memory), an EPROM, a hard disk, or a ROM. The memory 620 has a storage space 6201 for the program code 630 of the computer program for executing any of the method steps described above. For example, the storage space 6201 for the program code 630 can comprise individual computer programs for implementing the various steps in the above methods, respectively. The program code 630 is computer readable code. The computer programs can be read from or written to one or more computer program products. The computer program products comprise program code carriers such as a hard disk, a compact disk (CD), a memory card, or a floppy disk. The computer programs comprise computer readable code which, when executed on the electronic device, causes the electronic device to perform the method according to the above embodiments.

[0105] The embodiments of the present application further disclose a computer readable storage medium, having stored thereon a computer program, which, when executed by a processor, implements the steps of the intent recognition method according to the embodiment one of the present application.

[0106] Such a computer program product can be a computer readable storage medium having stored thereon the computer program. A computer readable storage medium can be a Figure 6 memory 620 in the electronic device shown. The program code can be stored in the computer readable storage medium, for example, in a compressed form. The computer readable storage medium is typically a portable or stationary storage unit as described with reference to Figure 7 The computer readable storage medium typically comprises computer readable code 630' which is code that is readable by a processor, which, when executed by the processor, implements the various steps in the above described methods.

[0107] The terms "one embodiment", "an embodiment” or "one or more embodiments” as used herein mean that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the application. The appearances of the phrase "in one embodiment” in various places in this specification are not necessarily all referring to the same embodiment.

[0108] In the description provided herein, numerous specific details are set forth. However, it is understood that embodiments of the application can be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.

[0109] In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word 'comprising' does not exclude the presence of elements or steps other than those listed in a claim. The word 'a' or 'an' preceding an element does not exclude the presence of a plurality of such elements. The application can be implemented by means of both hardware and software, and any combination thereof. In a unitary claim, several devices or sub-claims can be joined by means of the word 'or'. The word 'first','second', 'third', etc. do not imply any order. The terms 'first','second', 'third', etc. are to be interpreted according to their meaning in the context and are not to be interpreted as a ranking.

[0110] It has to be noted that, while the above describes example embodiments of the application, these are merely to illustrate the technical solutions of the application, and are not to limit the same. Even though the above has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still make modifications to the technical solutions recorded in the foregoing embodiments, or make equivalent replacements to some of the technical features therein; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the application.

Claims

1. An intention recognition method characterized by, The method comprises the following steps: encoding the to-be-recognized query input by a preset encoder to determine the feature encoding of the to-be-recognized query input; classifying and mapping the feature encoding of the to-be-recognized query input by a pre-trained intent classification model to determine the probability that the to-be-recognized query input matches a preset intent category; wherein the intent classification model shares model parameters with an intent classification task in a pre-trained multi-task model, and the multi-task model is trained based on the association between the intent category and the information of the query input and the semantic matching relationship between the intent category and the query input; determining the preset intent category matched by the to-be-recognized query input according to the probability that the to-be-recognized query input matches the preset intent category; the multi-task model further comprises an intent matching task, and before the step of encoding the to-be-recognized query input by a preset encoder to determine the feature encoding of the to-be-recognized query input, the method further comprises the following steps: encoding the query input and the intent category in each of the matching combinations by the preset encoder to determine the feature encoding of the query input and the feature encoding of the intent category; estimating the probability distribution of the feature encoding of the query input in the matching combination matching a preset intent category by the intent classification task to determine a first loss value; estimating the semantic similarity between the feature encoding of the query input and the feature encoding of the intent category in the matching combination by the intent matching task to determine a second loss value; fusing the first loss value and the second loss value to obtain the loss value of the multi-task model; optimizing the loss value of the multi-task model by optimizing the network parameters of the intent classification task and the intent matching task to iteratively train the multi-task model.

2. The method of claim 1, wherein, The step of estimating the probability distribution of the feature encoding of the query input in the matching combination matching a preset intent category by the intent classification task to determine a first loss value comprises the following steps: classifying and mapping the feature encoding of the query input in the matching combination by the intent classification task to determine the probability estimation value of each query input matching a preset intent category; determining the first loss value according to the probability estimation value of each query input matching a preset intent category and the real probability value of the query input matching a preset intent category, wherein the real probability value of the query input matching a preset intent category is determined according to the intent category matched by the query input.

3. The method of claim 1, wherein, The step of estimating the semantic similarity between the feature encoding of the query input and the feature encoding of the intent category in the matching combination by the intent matching task to determine a second loss value comprises the following steps: The similarity estimation operations are performed on at least part of the matching combinations, respectively, as follows: taking the query input and the intent category in the same matching combination as a matching positive example, taking the query input and the intent category in different matching combinations as a matching negative example, and determining the semantic similarity between the feature encoding of the query input and the feature encoding of the intent category in each matching positive example and each matching negative example through the intent matching task; A second loss value is determined according to the determined semantic similarities.

4. The method of claim 3, wherein, The step of determining the second loss value according to the determined semantic similarities comprises: A similarity matrix is generated according to the semantic similarity between the feature encoding of the query input and the feature encoding of the intent category in each matching positive example and each matching negative example; the rows and columns of the similarity matrix correspond to the query input and the intent category, respectively; the matrix element of a specified row and a specified column of the similarity matrix represents the semantic similarity between the feature encoding of the corresponding query input and the intent category; and the matrix elements on the diagonal line of the similarity matrix represent the semantic similarity between the feature encoding of the query input and the intent category in the matching positive example. A unit matrix corresponding to the similarity matrix is generated. A second loss value is determined by performing a preset matrix operation on the similarity matrix and the unit matrix.

5. An intention recognition apparatus characterized by comprising: The method comprises: An encoding module is configured to encode a to-be-identified query input through a preset encoder to determine the feature encoding of the to-be-identified query input. An intent category matching probability determination module is configured to classify and map the feature encoding of the to-be-identified query input through a pre-trained intent classification model to determine the probability that the to-be-identified query input matches a preset intent category; the intent classification model shares model parameters with an intent classification task in a pre-trained multi-task model, and the multi-task model is trained in combination with the association between the information of the intent category and the query input and the semantic matching relationship between the intent category and the query input. An intent category determination module is configured to determine the preset intent category matched by the to-be-identified query input according to the probability that the to-be-identified query input matches the preset intent category; the multi-task model further comprises an intent matching task, and the device further comprises: A multi-task model training module is configured to train the multi-task model according to a plurality of matching combinations of query input and intent category; the multi-task model training module is further configured to encode the query input and the intent category in each matching combination through the preset encoder to determine the feature encoding of the query input and the feature encoding of the intent category. The probability distribution of the feature encoding of the query input in the matching combination matching a preset intent category is estimated through the intent classification task to determine a first loss value. The semantic similarity between the feature code of the query input in the matching combination and the feature code of the intent category is estimated through the intent matching task, and a second loss value is determined; The first loss value and the second loss value are fused to obtain a loss value of the multi-task model; The loss value of the multi-task model is optimized by optimizing the network parameters of the intent classification task and the intent matching task, and the multi-task model is iteratively trained.

6. An electronic device comprising a memory, a processor, and program code stored on the memory and executable on the processor, wherein, The processor executes the program code to implement the intent recognition method of any one of claims 1 to 4.

7. A computer-readable storage medium having stored thereon a program code, characterized in that, The program code is executed by the processor to implement the steps of the intent recognition method of any one of claims 1 to 4. The program code is executed by the processor to implement the steps of the intent recognition method of any one of claims 1 to 4.

Images