Intention recognition method and device, computer device, and storage medium

Abstract

The application discloses an intention recognition method, which comprises the following steps: obtaining a question and answer correlation label by performing correlation prediction on historical dialogue data through a preset training model; obtaining an intention recognition model by iteratively training the preset training model through a historical question, a historical answer and the question and answer correlation label corresponding to the same historical dialogue data; obtaining a target intention recognition model by fine-tuning the intention recognition model through a target scene question, a target scene answer and a target scene label; and performing intention recognition on a to-be-recognized question through the target intention recognition model to obtain an intention recognition result. The preset training model is pre-trained through a large amount of unlabeled historical dialogue data, and can be applied to a target scene through target scene data fine-tuning, so that the intention recognition model has strong generalization. The intention recognition accuracy of the target intention recognition model is improved through the target intention recognition model.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of natural language, in particular to an intent recognition method and device, computer equipment and a storage medium. BACKGROUND

[0002] With the development of artificial intelligence, intelligent technologies such as big data perception and artificial intelligence customer service are constantly penetrating into people's daily life. Whether it is intent recognition in big data perception or multi-turn dialogue scenario in artificial intelligence customer service, input needs to be recognized for intent and then relevant answers are given. Therefore, it is very important to accurately recognize the intent of the input question.

[0003] In the prior art, artificial online customer service platforms are usually used to solve customer consultation problems by relying on manual services, and customer service personnel obtain corresponding customer service answers for problem solving by querying an internal knowledge base. Or an automatic customer service system is realized by an intelligent question and answer system, which matches the user consultation content with the knowledge base content by performing word segmentation or semantic analysis. Since the relevance between the answers and the questions in the scenario is not considered, the accuracy of intent recognition is not high, which leads to low query efficiency and the situation of answering irrelevant questions, seriously affecting the user experience. SUMMARY

[0004] The embodiments of the present application provide an intent recognition method, device, computer equipment and storage medium to solve the problem that the relevance between the answers and the questions in the scenario is not considered in the prior art, resulting in low accuracy of intent recognition.

[0005] An intent recognition method comprises:

[0006] Obtaining a historical unlabeled data set, the historical unlabeled data set comprising at least one historical dialogue data, the historical dialogue data comprising a historical question and a historical answer corresponding to the historical question;

[0007] Obtaining a preset training model, predicting the relevance of all the historical questions and all the historical answers by the preset training model to obtain a question and answer relevance label; the question and answer relevance label is used to represent whether the historical question and the historical answer are relevant;

[0008] Iteratively training the preset training model by the historical question, the historical answer and the question and answer relevance label corresponding to the same historical dialogue data to obtain an intent recognition model;

[0009] Obtaining a target scene data set, the target scene data set comprising at least one target scene data and a target scene label corresponding to the target scene data; the target scene data comprising a target scene question and a target scene answer;

[0010] Fine-tune the intent recognition model through all the target scene questions, all the target scene answers, and the target scene labels, to obtain a target intent recognition model;

[0011] Obtain at least one to-be-recognized question, perform intent recognition on all the to-be-recognized questions based on the target intent recognition model, and obtain an intent recognition result corresponding to each to-be-recognized question.

[0012] An intent recognition device comprises:

[0013] A data acquisition module is configured to acquire a historical unlabeled data set, wherein the historical unlabeled data set comprises at least one historical dialogue data, and the historical dialogue data comprises a historical question and a historical answer corresponding to the historical question;

[0014] A label prediction module is configured to acquire a preset training model, perform relevance prediction on all the historical questions and all the historical answers through the preset training model, and obtain a question-answer relevance label; the question-answer relevance label is used to represent whether the historical question and the historical answer are relevant;

[0015] A model training module is configured to perform iterative training on the preset training model through the historical question, the historical answer, and the question-answer relevance label corresponding to the same historical dialogue data, to obtain an intent recognition model;

[0016] A target data acquisition module is configured to acquire a target scene data set, wherein the target scene data set comprises at least one target scene data and a target scene label corresponding to the target scene data; the target scene data comprises a target scene question and a target scene answer;

[0017] A model fine-tuning module is configured to fine-tune the intent recognition model through all the target scene questions, all the target scene answers, and the target scene labels, to obtain a target intent recognition model;

[0018] An identification result module is configured to obtain at least one to-be-recognized question, perform intent recognition on all the to-be-recognized questions based on the target intent recognition model, and obtain an intent recognition result corresponding to each to-be-recognized question.

[0019] A computer device comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the above-mentioned intent recognition method when executing the computer program.

[0020] A computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement the above-mentioned intent recognition method.

[0021] The application provides an intention recognition method, device, computer equipment and storage medium. The method predicts all historical dialogue data through a preset training model, thereby realizing acquisition of question and answer related labels, determining whether historical questions and historical answers in the historical dialogue data are related, fully utilizing unannotated historical dialogue data, and improving the effect of intention recognition of the preset training model. The preset training model is iteratively trained through historical questions, historical answers and question and answer related labels corresponding to the same historical dialogue data, thereby realizing determination of an intention recognition model, improving the intention recognition accuracy of the intention recognition model through iterative training, and applying the intention recognition model to a target scene through fine-tuning of target scene data, thereby making the intention recognition model have strong generalization, and improving the recognition accuracy of the target intention recognition model. The target intention recognition model is used to recognize the intention of a to-be-recognized question in the target scene, thereby realizing determination of an intention recognition result, and ensuring the accuracy of the intention recognition result. Further, the target scene data and the pre-training data are set to be consistent, i.e., including target scene questions, target scene answers and target scene labels, thereby reducing the gap between the pre-training stage and the fine-tuning stage, and improving the recognition effect of the target intention recognition model. BRIEF DESCRIPTION OF DRAWINGS

[0022] In order to more clearly illustrate the technical solutions of the embodiments of the application, the following will briefly introduce the drawings needed to be used in the description of the embodiments of the application. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor.

[0023] Figure 1 is an application environment schematic diagram of the intention recognition method in an embodiment of the application;

[0024] Figure 2 is a flowchart of the intention recognition method in an embodiment of the application;

[0025] Figure 3 is a flowchart of step S20 of the intention recognition method in an embodiment of the application;

[0026] Figure 4 is a flowchart of step S50 of the intention recognition method in an embodiment of the application;

[0027] Figure 5 is a principle block diagram of the intention recognition device in an embodiment of the application;

[0028] Figure 6 is a schematic diagram of the computer equipment in an embodiment of the application. DETAILED DESCRIPTION

[0029] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of the present application.

[0030] The intent recognition method provided by the embodiments of the present application can be applied in an application environment as shown in Figure 1 . Specifically, the intent recognition method is applied in an intent recognition device, which includes a client and a server as shown in Figure 1 . The client and the server communicate through a network, and are used to solve the problem that the correlation between answers and questions in a scene is not considered in the prior art, resulting in a low accuracy of intent recognition. The server can be an independent server, or a cloud server providing basic cloud computing services such as cloud service, cloud database, cloud computing, cloud function, cloud storage, network service, cloud communication, middleware service, domain name service, security service, content delivery network (CDN), and big data and artificial intelligence platform. The client, also known as the user end, is a program that provides classification services for clients and corresponds to the server. The client can be installed on, but is not limited to, various personal computers, notebook computers, smart phones, tablet computers, and portable wearable devices.

[0031] In an embodiment, as shown in Figure 2 , an intent recognition method is provided. Taking the server in Figure 1 as an example, the method includes the following steps:

[0032] S10: Obtain a historical unlabeled data set, the historical unlabeled data set including at least one historical dialogue data, the historical dialogue data including a historical question and a historical answer corresponding to the historical question.

[0033] Understandably, the historical dialogue data can be obtained from different clients or prepared in advance from different databases. Then, a historical unlabeled data set is constructed according to all the obtained historical dialogue data. One historical dialogue data includes a historical question and a historical answer corresponding to the historical question, and the content of the historical question and the historical answer is different in different application scenarios. For example, in the application scenario of intelligent question answering, the historical question and the historical answer can be: Are you satisfied with the company's training? The answer is: satisfied. At this time, the unlabeled historical dialogue data can be subjected to intent recognition, and the semantics is extracted from the historical dialogue data and classified to obtain an intent recognition result. In the application scenario of insurance, the historical question and the historical answer can be: How much is this insurance? The answer is: 3000 yuan for this car insurance. At this time, the unlabeled historical dialogue data can be subjected to intent recognition, and the semantics is extracted from the historical dialogue data and classified to obtain an intent recognition result.

[0034] S20: Obtain a preset training model, and perform relevance prediction on all the historical questions and all the historical answers through the preset training model to obtain a question and answer relevance label; the question and answer relevance label is used to represent whether the historical question and the historical answer are relevant.

[0035] Understandably, the question and answer relevance label is used to represent whether the historical question and the historical answer are relevant, for example, Are you satisfied with the company's training? The answer is: 3000 yuan for this car insurance, and a question and answer relevance label representing that the historical question and the historical answer are not relevant is obtained. The preset training model is a model set in advance for intent recognition.

[0036] Specifically, after obtaining the historical unlabeled data set, a preset training model is called from the database, and all the historical dialogue data is input into the preset training model, that is, a binary tuple composed of the historical question and the historical answer corresponding to the same historical dialogue data is input into the preset training model. The historical question and the historical answer in all binary tuples are predicted for relevance by the preset training model, that is, the historical question and the historical answer are encoded by the input layer of the preset training model, that is, the historical question and the historical answer are connected in the order of question first and answer second to obtain a set of historical texts, and the historical texts are encoded by using one-hot encoding to obtain an input vector. The semantic layer of the preset training model extracts the semantics of the input vector, that is, the main content is extracted from the input vector by the attention mechanism to obtain a semantic vector. The classification layer of the preset training model predicts the semantic vector, that is, the semantic vector and the input vector of the historical answer are compared to obtain a question and answer relevance label corresponding to the historical dialogue data.

[0037] S30, iteratively training the preset training model by the historical question, the historical answer and the question-answer related label corresponding to the same historical dialogue data, to obtain an intent recognition model.

[0038] Understandably, the intent recognition model is obtained by training a large amount of unlabeled historical dialogue data. The convergence condition can be a condition that the predicted loss value is less than a set threshold, or a condition that the predicted loss value is very small and cannot be further reduced after 500 calculations, and the training is stopped.

[0039] Specifically, after obtaining all the question-answer related labels, the historical question, the historical answer and the question-answer related label corresponding to the same historical dialogue data are associated to construct a triple. The historical question in the binary tuple in the historical dialogue data is randomly replaced with the historical question in another historical dialogue data, and the historical answer in the binary tuple is added, and the question-answer related label for representing irrelevant is added to construct another triple. At least two triples (one group of historical questions and historical answers related, and one group of historical questions and historical answers unrelated) are constructed. All the constructed triples are input into the preset training model, that is, the intent recognition of the historical question and the historical answer in each triple is performed to obtain the predicted label. The loss value of the preset training model is calculated by the predicted label and the question-answer related label corresponding to the same triple, and the predicted loss value of the preset training model is obtained. Further, it is judged whether the predicted loss value reaches the convergence condition. When the predicted loss value does not reach the preset convergence condition, the initial parameters of the preset training model are adjusted according to the predicted loss value, and all the triples are input into the preset training model with adjusted initial parameters. The preset training model with adjusted initial parameters is iteratively trained by the triples to obtain the predicted loss value corresponding to the preset training model with adjusted initial parameters. In this way, the result output by the preset training model constantly approaches the accurate result, and the prediction accuracy is higher and higher until the converged preset training model is recorded as the intent recognition model.

[0040] S40, obtaining a target scene data set, the target scene data set including at least one target scene data and a target scene label corresponding to the target scene data; the target scene data including a target scene question and a target scene answer.

[0041] Understandably, the target scene data is dialogue data in a specific scene, for example, in the application scenario of intelligent question answering, the target scene data can be the dialogue between the staff and the customer in a specific scene. The target scene data can be sent to the server from different clients, or can be collected from different databases. Then, a target scene data set is constructed according to all the obtained target dialogue data, wherein the target scene data set is labeled data. The target scene data includes a target scene question, a target scene answer, and a target scene label corresponding to the target scene data, wherein the target scene question, the target scene label, and the target scene answer form a triple, so as to fine-tune the intent recognition model, and reduce the gap between the pre-training stage and the fine-tuning stage. The target scene label is used to represent the intent of the target scene data corresponding to the target scene label.

[0042] S50, fine-tune the intent recognition model through all the target scene questions, all the target scene answers, and the target scene labels, to obtain a target intent recognition model.

[0043] Understandably, the target intent recognition model is obtained by fine-tuning the intent recognition model, and is used in a specific scene. The difference between the target intent recognition model and the intent recognition model is that the classification layers are different.

[0044] Specifically, after obtaining the target scene data set, all the target scene questions, all the target scene answers, and the target scene labels are input into the intent recognition model to adjust the parameters of the intent recognition model, so that the intent recognition model is more consistent with the target scene. First, the target scene question and the target scene answer of the same target scene data are spliced to obtain a group of target texts. And the target text is encoded by the input layer of the intent recognition model, that is, the one-hot encoding method is used for encoding, so as to obtain a target input vector. Then, the semantics in the target input vector are extracted through the semantic layer, that is, the semantic extraction of the target input vector is performed through the attention mechanism, to obtain a target semantic vector. And the target semantic vector is subjected to intent recognition through the classification layer, so as to obtain a target prediction label. Further, the target prediction label corresponding to the same target scene data is compared with the target scene label through a loss function, so as to determine the model loss value of the intent recognition model. It is judged whether the model loss value reaches the convergence condition, and when the model loss value does not reach the preset convergence condition, the initial parameters of the intent recognition model are adjusted according to the model loss value, so that the model loss value of the intent recognition model with adjusted initial parameters reaches the preset convergence condition, and the converged intent recognition model is recorded as the target intent recognition model.

[0045] S60, obtaining at least one to-be-identified question, performing intent recognition on all the to-be-identified questions based on the target intent recognition model to obtain an intent recognition result corresponding to each to-be-identified question

[0046] It can be understood that the to-be-identified question is a question that needs to be subjected to intent recognition in a target scenario. The to-be-identified data can be collected from different databases or sent from a client to a server. The intent recognition result is a result obtained by performing intent recognition on the to-be-identified question.

[0047] Specifically, the fine-tuned target entity recognition model is invoked, and all the to-be-identified questions are input into the target intent recognition model. The input layer of the target intent recognition model is used to respectively perform encoding processing on all the to-be-identified questions, that is, the to-be-identified questions are encoded in a one-hot encoding manner, so as to obtain an input vector corresponding to each to-be-identified question. The semantic layer of the target intent recognition model is used to perform semantic extraction on the input vector, that is, important content in the input vector is extracted through an attention mechanism, so as to obtain a semantic vector corresponding to each to-be-identified question. Finally, the update classification layer of the target intent recognition model is used to perform intent recognition on the semantic vector, so as to obtain an intent recognition result. In this way, the intent recognition result of all the to-be-identified questions can be obtained through the above manner.

[0048] In an embodiment of the present application, a method for intent recognition is provided. The method performs prediction on all historical dialogue data through a preset training model, so as to realize acquisition of a question and answer related label, and further determine whether a historical question and a historical answer in the historical dialogue data are related, and fully utilize the unlabeled historical dialogue data, thereby improving the effect of intent recognition of the preset training model. The preset training model is iteratively trained through the historical question, the historical answer and the question and answer related label corresponding to the same historical dialogue data, so as to realize determination of an intent recognition model, and further improve the intent recognition accuracy of the intent recognition model through iterative training. The target scene data is fine-tuned and applied to the target scene, so that the intent recognition model has strong generalization, and the recognition accuracy of the target intent recognition model is improved. The target intent recognition model is used to perform intent recognition on a to-be-identified question in a target scenario, so as to realize determination of an intent recognition result, and further ensure the accuracy of the intent recognition result. Further, the target scene data and the pre-training data are set to be consistent, that is, the target scene question, the target scene answer and the target scene label are included, so as to reduce the gap between the pre-training stage and the fine-tuning stage, and further improve the recognition effect of the target intent recognition model.

[0049] In an embodiment, as shown in FIG. 6, the method comprises the following steps. Figure 3As shown, in step S20, the relevance prediction of all the historical questions and all the historical answers by the preset training model is performed to obtain a question-answer relevance label, including:

[0050] S201, the historical question and the historical answer are encoded by the input layer of the preset training model to obtain an input vector corresponding to the historical dialogue data.

[0051] Understandably, the input vector is obtained by the input layer of the preset training model processing the historical dialogue data.

[0052] Specifically, after obtaining the historical unlabeled data set, the historical question and the historical answer in all the historical dialogue data are connected in the order of question first and answer later to obtain historical text corresponding to each historical dialogue data. The input layer of the preset training model is used to perform word segmentation on each historical text respectively, that is, the historical text is divided into one character by one character, and the punctuation symbol is segmented as a character. A CLS flag is added before the first character of the historical text as a start flag, a SEP flag is added between the historical question and the historical answer for distinguishing the historical question and the historical answer, and a SEP flag is added after the last character of the historical answer as an end flag. Further, the position of each character and flag is labeled, and the position labeling function is obtained by learning. After vectorizing each character and flag, that is, using one-hot encoding method for encoding, and splicing all vectors according to the positions corresponding to the characters and flags, an input vector corresponding to each historical dialogue data is obtained.

[0053] S202, the semantic layer of the preset training model is used to perform semantic extraction on all the input vectors to obtain a semantic vector corresponding to the historical dialogue data.

[0054] Understandably, the semantic vector is a feature for representing the historical dialogue data.

[0055] Specifically, after obtaining all input vectors, semantic extraction is performed on all input vectors by a semantic layer of a preset training model, that is, semantic extraction is performed by multiple sets of attention mechanisms, that is, Q vectors, K vectors, and V vectors in the input vectors are calculated by the multiple sets of attention mechanisms respectively, that is, a correlation score between the Q vectors and the K vectors in the input vectors is calculated by using a dot product method, that is, a dot product is calculated between each vector in Q and each vector in K, and the correlation score between the Q vectors and the K vectors is normalized to obtain a score vector. Then, the score vector corresponding to the input vector is converted into a probability distribution between 0 and 1 by using a softmax function, and then multiplied by a corresponding Values value to obtain an attention result. Finally, the attention results of different groups are spliced, and average pooling is performed on the attention vectors, so as to obtain a semantic vector corresponding to the historical dialogue data.

[0056] In S203, the semantic vectors are predicted by a classification layer of the preset training model to obtain a question-answer related label corresponding to the historical dialogue data.

[0057] Specifically, the semantic vectors are predicted by the classification layer of the preset training model, that is, the semantic vectors are nonlinearly transformed, that is, the semantic vectors are predicted by hidden units in the classification layer with different weights, and an intent recognition vector is output. By calculating the similarity between the intent recognition vector and the historical answer, the question-answer related label corresponding to the historical dialogue data can be obtained by comparing the similarity value with a preset similarity threshold.

[0058] In the embodiment of the application, the historical dialogue data is encoded by the preset training model, the input vectors are determined, the semantic vectors are extracted by performing semantic extraction on all input vectors, the question-answer related label representing whether the historical question and the historical answer are related is predicted by predicting all semantic vectors, and the preset training model is iteratively trained by using unlabeled data.

[0059] In an embodiment, in S203, that is, the semantic vectors are predicted by the classification layer of the preset training model to obtain the question-answer related label corresponding to the historical dialogue data, including:

[0060] In S2031, similarity processing is performed on the semantic vectors and an input vector corresponding to the historical answer to obtain a similarity value.

[0061] It can be understood that the similarity value is an Euclidean distance or a cosine similarity between the semantic vectors and the input vector corresponding to the historical answer, or is obtained by using other methods for calculating similarity.

[0062] Specifically, after obtaining the semantic vector, all semantic vectors are input into a classification layer for prediction, that is, all semantic vectors are predicted by hidden units in the classification layer with different weights, and an intent recognition vector is output, that is, the intent recognition vector corresponding to the same historical dialogue data and the input vector corresponding to the historical dialogue data are subjected to similarity processing, that is, the cosine similarity between the intent recognition vector corresponding to the historical dialogue data and the input vector corresponding to the historical dialogue data is calculated, and the calculated cosine similarity between the intent recognition vector and the input vector corresponding to the historical dialogue data is determined as a similarity value.

[0063] S2032, when the similarity value is greater than or equal to a preset similarity threshold, the question and answer related label represents that the historical question and the historical answer are related.

[0064] S2033, when the similarity value is less than the preset similarity threshold, the question and answer related label represents that the historical question and the historical answer are not related.

[0065] Specifically, the preset similarity threshold is called, and all similarity values corresponding to the historical dialogue data are compared with the preset similarity threshold in turn, when the similarity value is greater than or equal to the preset similarity threshold, that is, when the cosine similarity between the semantic vector and the input vector corresponding to the historical dialogue data is greater than or equal to the preset threshold, the question and answer related label representing that the historical question and the historical answer are related is obtained. When the similarity value is less than the preset similarity threshold, that is, when the cosine similarity between the semantic vector and the input vector corresponding to the historical dialogue data is less than the preset threshold, the question and answer related label representing that the historical question and the historical answer are not related is obtained. In this way, all question and answer related labels corresponding to the historical dialogue data can be obtained by the above method.

[0066] The embodiment of the application realizes the determination of the similarity value between two vectors by calculating the similarity between the semantic vector and the input vector of the historical answer, when the similarity value is greater than or equal to the preset similarity threshold, the question and answer related label representing that the historical question and the historical answer are related is determined, and when the similarity value is less than the preset similarity threshold, the question and answer related label representing that the historical question and the historical answer are not related is determined, thereby ensuring that the subsequent intent recognition model has a higher prediction accuracy.

[0067] In an embodiment, in step S30, that is, the preset training model is iteratively trained by the historical question corresponding to the same historical dialogue data, the historical answer and the question and answer related label, and an intent recognition model is obtained, comprising:

[0068] S301, predicting, by the preset training model, all the historical dialogue data to obtain a predicted label.

[0069] It can be understood that the predicted label is a relevance prediction of the historical question, the historical answer and the question and answer related label, that is, the predicted label is a probability predicted by the preset training model.

[0070] Specifically, after obtaining the question and answer related label, the historical dialogue data and the question and answer related label corresponding to the historical dialogue data are input into the preset training model, and a triple (question, answer and label) is constructed according to the historical dialogue data and the question and answer related label corresponding to the historical dialogue data. All triples are encoded by the input layer of the preset training model, that is, the CLS flag bit and the SEP flag bit are added, and the triple and the flag bit are represented in the form of a vector, that is, the one-hot encoding method is used for encoding, so as to obtain an input vector corresponding to each historical dialogue data. The semantic layer of the preset training model is used to extract the intention in the historical dialogue data, that is, the intention in the historical dialogue data is extracted by the attention mechanism, so as to obtain a semantic vector. Then, the classification layer of the preset training model is used to identify the intention of the semantic vector, so as to obtain a predicted label corresponding to the historical dialogue data. In this way, all the predicted labels corresponding to the historical dialogue data can be obtained by the above method.

[0071] S302, determining, according to the historical question, the historical answer, the question and answer related label and the predicted label corresponding to the same historical dialogue data, a prediction loss value of the preset training model.

[0072] It can be understood that the prediction loss value is generated by the preset training model in the process of predicting the historical question and the historical answer. The prediction loss calculation formula is: y represents the vector of the label, the category is C, the pre-training C=2, when y j =1, it means that the label is the jth category, P j is the prediction probability.

[0073] Specifically, after obtaining the predicted label, the predicted label corresponding to each historical dialogue data is arranged according to the order of the historical dialogue data in the historical unlabeled data set, and then the question and answer related label corresponding to the historical dialogue data is compared with the predicted label of the historical dialogue data with the same sequence; that is, according to the historical dialogue data sorting, the question and answer related label corresponding to the historical dialogue data located at the first is compared with the predicted label corresponding to the historical dialogue data located at the first, and the loss value between the question and answer related label and the predicted label is calculated through the loss function; then the question and answer related label corresponding to the historical dialogue data located at the second is compared with the predicted label corresponding to the historical dialogue data located at the second, until the comparison of the predicted label and the question and answer related label is completed, that is, the prediction loss value of the preset training model can be determined.

[0074] S303, when the prediction loss value does not reach the preset convergence condition, iteratively update the initial parameters in the preset training model until the prediction loss value reaches the convergence condition, and record the preset training model after convergence as an intention recognition model.

[0075] Understandably, the convergence condition can be a condition that the prediction loss value is less than a set threshold, and can also be a condition that the prediction loss value is very small and cannot be further reduced after 500 calculations, and the training is stopped.

[0076] Specifically, after determining the prediction loss value of the preset training model, it is judged whether the prediction loss value reaches the convergence condition, when the prediction loss value does not reach the preset convergence condition, the initial parameters of the preset training model are adjusted through the prediction loss value, and all triplets are re-input into the preset training model with adjusted initial parameters, and the preset training model with adjusted initial parameters is retrained through the triplets, to obtain the prediction loss value corresponding to the preset training model with adjusted initial parameters. And when the prediction loss value does not reach the preset convergence condition, the initial parameters of the preset recognition model are adjusted again according to the prediction loss value, so that the prediction loss value of the preset training model with adjusted initial parameters reaches the preset convergence condition. In this way, the result output by the preset training model can be constantly close to the accurate result, and the prediction accuracy of the model is higher and higher, until the prediction loss value of the preset training model reaches the preset convergence condition, and the preset training model after convergence is recorded as an intention recognition model.

[0077] The embodiment of the application trains the preset training model through a large amount of historical dialogue data, calculates the prediction loss value between the predicted label and the question and answer related label through the loss function, adjusts the initial parameters of the preset training model according to the prediction loss value, and converges the model, thereby realizing the acquisition of the intention recognition model, and further ensuring that the intention recognition model has a high prediction accuracy.

[0078] In one embodiment, such as Figure 4 As shown, in step S50, that is, by fine-tuning the intent recognition model using all the target scenario questions, all the target scenario answers, and the target scenario labels to obtain the target intent recognition model, the following steps are included:

[0079] S501, the target scenario question and the target scenario answer are predicted by the intent recognition model to obtain the target prediction label.

[0080] Understandably, target prediction labels are used to characterize the intent behind the predicted target scene data.

[0081] Specifically, after obtaining the target scene dataset, the target text is encoded through the input layer of the intent recognition model. Specifically, a CLS flag is added between the first characters of the target text, a SEP flag is added between the target scene question and the target scene answer, and a SEP flag is added after the last character of the target scene answer. The target text is then represented as a vector using one-hot encoding to obtain the target input vector. The semantic layer extracts the semantics from the target input vector, specifically through an attention mechanism, to obtain the target semantic vector. Finally, a classification layer performs intent recognition on the target semantic vector. The number of intent categories in this classification layer differs from that in the intent recognition model, thus obtaining the target predicted label.

[0082] S502, determine the scene loss value of the intent recognition model based on the target scene label and target prediction label corresponding to the same target scene data.

[0083] Understandably, the scene loss value is generated during the fine-tuning of the intent recognition model.

[0084] Specifically, after obtaining the target prediction label, the target prediction label corresponding to each historical dialogue data is arranged according to the order of the historical dialogue data in the historical unlabeled dataset. Then, the target scene label corresponding to the historical dialogue data is compared with the target prediction label of the historical dialogue data with the same sequence. That is, according to the historical dialogue data, the target scene label corresponding to the first historical dialogue data is compared with the target prediction label corresponding to the first historical dialogue data, and the loss value between the target scene label and the target prediction label is calculated by the loss function. Then, the target scene label corresponding to the second historical dialogue data is compared with the target prediction label corresponding to the second historical dialogue data, until all target prediction labels and target scene labels are compared, and the scene loss value of the preset training model can be determined.

[0085] S503, when the scene loss value reaches a preset convergence condition, determining the intention recognition model after convergence as a target intention recognition model.

[0086] Specifically, after obtaining the scene loss value, it is judged whether the model loss value reaches a convergence condition. When the model loss value does not reach the preset convergence condition, the initial parameters of the intention recognition model are adjusted according to the model loss value, and all target scene data are re-input into the intention recognition model after adjusting the initial parameters. The intention recognition model after adjusting the initial parameters is re-trained by the target scene data, a model loss value corresponding to the intention recognition model after adjusting the initial parameters is obtained, and the model loss value of the preset training model after adjusting the initial parameters reaches the preset convergence condition. In this way, the result output by the intention recognition model can be continuously close to the accurate result, and the prediction accuracy is higher and higher until the convergent intention recognition model is recorded as the target intention recognition model.

[0087] The embodiment of the present application fine-tunes the intention recognition scene through the target scene data, so that the intention recognition of the intention recognition model is more in line with the target scene. By making the target scene data include the target scene question, the target scene answer and the target scene label, the gap between the pre-training stage and the fine-tuning stage is reduced, and the intention recognition effect of the target intention recognition model is improved.

[0088] In an embodiment, in step S501, that is, the target prediction label is obtained by performing label prediction on the target scene question and the target scene answer through the intention recognition model, comprising:

[0089] S5011, the number of classification categories of the classification layer of the intention recognition model is set to a preset category number of the target intention scene, to obtain an updated intention recognition model.

[0090] S5012, the target scene question and the target scene answer are encoded and processed through the input layer of the updated intention recognition model, to obtain a target input vector corresponding to the target scene data.

[0091] Understandably, the updated intention recognition model is obtained by updating the number of classification categories of the classification layer of the intention recognition model to the preset category number of the target scene. The preset category number is associated with the target scene and is set according to each specific scene.

[0092] Specifically, after obtaining the target scene data set, the preset category number corresponding to the target scene is called from the database according to the target scene data set, and the classification category number of the classification layer of the intent recognition model is updated to the preset category number, so as to obtain the updated intent recognition model. The target scene question and the target scene answer of the same target scene data are spliced in the manner of question first and answer later, to obtain a group of target texts. In this way, the target texts corresponding to the target scene data are obtained. All the target texts are input into the updated intent recognition model, and the target scene question and the target scene answer are encoded by the input layer of the updated intent recognition model, that is, a CLS flag is added between the first characters of the target text, a SEP flag is added between the target scene question and the target scene answer, and a SEP flag is added after the last character of the target scene answer, and the target text is represented in the form of a vector, and is encoded in a one-hot encoding manner, so as to obtain the target input vectors corresponding to the target scene data.

[0093] S5013, performing semantic extraction on all the target input vectors through the semantic layer of the updated intent recognition model, to obtain target semantic vectors corresponding to the target scene data.

[0094] S5014, performing intent recognition processing on all the target semantic vectors through the classification layer of the updated intent recognition model, to obtain target prediction labels corresponding to the target scene data.

[0095] Understandably, the target semantic vector is obtained by performing semantic extraction on the target input vector according to the updated intent recognition model, that is, the important content in the target text.

[0096] Specifically, after obtaining the target input vector, the semantic layer of the updated intent recognition model is used to perform semantic extraction on all the target input vectors, that is, the Q vector, the K vector and the V vector in the input vector are calculated through multiple groups of attention mechanisms respectively, to obtain a score vector. Then the score vector corresponding to the input vector is converted into a probability distribution between 0 and 1, and then multiplied by the corresponding Values value, to obtain an attention result. Different groups of attention results are spliced to obtain an attention vector, and the attention vector is averaged and pooled, to obtain the target semantic vectors corresponding to the target scene data. Further, the classification layer of the updated intent recognition model is used to perform intent recognition processing on all the target semantic vectors, that is, the hidden units in the classification layer are used to perform intent recognition on all the semantic vectors with different weights, to output an intent vector, that is, the intent vector is calculated through the relationship (P = softmax(SW + b), W and b are both learnable model parameters), and the intent vector with the maximum probability is determined as the target prediction label corresponding to the target scene data.

[0097] The embodiment of the present application updates the intention recognition model, so that the intention recognition of the intention recognition model is more in line with the target scene, thereby realizing the determination of the updated intention recognition model. The intention recognition model is updated to recognize the intention of the target scene data, thereby realizing the determination of the target prediction label, and thereby ensuring the accuracy of all target prediction labels.

[0098] It should be understood that the size of the serial number of each step in the above embodiment does not mean the order of execution, and the execution order of each process should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiment of the present application.

[0099] In an embodiment, an intention recognition device is provided, which corresponds to the intention recognition method described above. As shown in the figure, the intention recognition device includes a data acquisition module 11, a label prediction module 12, a model training module 13, a target data acquisition module 14, a model fine-tuning module 15 and an identification result module 16. The functions of each module are described in detail as follows: Figure 5

[0100] The data acquisition module 11 is configured to acquire a historical unlabeled data set, wherein the historical unlabeled data set includes at least one historical dialogue data, and the historical dialogue data includes a historical question and a historical answer corresponding to the historical question.

[0101] The label prediction module 12 is configured to acquire a preset training model, and predict the relevance of all the historical questions and all the historical answers by using the preset training model to obtain a question and answer relevance label. The question and answer relevance label is used to represent whether the historical question and the historical answer are relevant.

[0102] The model training module 13 is configured to iteratively train the preset training model by using the historical question, the historical answer and the question and answer relevance label corresponding to the same historical dialogue data to obtain an intention recognition model.

[0103] The target data acquisition module 14 is configured to acquire a target scene data set, wherein the target scene data set includes at least one target scene data and a target scene label corresponding to the target scene data, and the target scene data includes a target scene question and a target scene answer.

[0104] The model fine-tuning module 15 is configured to fine-tune the intention recognition model by using all the target scene questions, all the target scene answers and the target scene label to obtain a target intention recognition model.

[0105] ​The recognition result module 16 is configured to obtain at least one to-be-recognized question, perform intent recognition on all the to-be-recognized questions based on the target intent recognition model, and obtain an intent recognition result corresponding to each to-be-recognized question.

[0106] In an embodiment, the label prediction module 12 comprises:

[0107] The input vector unit is configured to encode the historical question and the historical answer by using an input layer of the preset training model, and obtain an input vector corresponding to the historical dialogue data.

[0108] The semantic vector unit is configured to perform semantic extraction on all the input vectors by using a semantic layer of the preset training model, and obtain a semantic vector corresponding to the historical dialogue data.

[0109] The related label unit is configured to perform prediction processing on all the semantic vectors by using a classification layer of the preset training model, and obtain a question-and-answer related label corresponding to the historical dialogue data.

[0110] In an embodiment, the question-and-answer related label comprises:

[0111] The comparison unit is configured to perform similarity processing on the semantic vector and an input vector corresponding to the historical answer, and obtain a similarity value.

[0112] The related unit is configured to determine that the question-and-answer related label represents that the historical question and the historical answer are related when the similarity value is greater than or equal to a preset similarity threshold.

[0113] The unrelated unit is configured to determine that the question-and-answer related label represents that the historical question and the historical answer are unrelated when the similarity value is less than the preset similarity threshold.

[0114] In an embodiment, the model training module 13 comprises:

[0115] The label prediction unit is configured to perform prediction on all the historical dialogue data by using the preset training model, and obtain a predicted label.

[0116] The loss prediction unit is configured to determine a prediction loss value of the preset training model according to the historical question, the historical answer, the question-and-answer related label, and the predicted label corresponding to the same historical dialogue data.

[0117] The model convergence unit is configured to iteratively update initial parameters in the preset training model when the prediction loss value does not reach a preset convergence condition, and record the preset training model after convergence as an intent recognition model when the prediction loss value reaches the convergence condition.

[0118] In an embodiment, the model fine-tuning module 15 comprises:

[0119] a target label prediction unit configured to perform label prediction on the target scene question and the target scene answer by the intention recognition model, to obtain target prediction labels corresponding to the target scene data;

[0120] a scene loss value unit configured to determine a scene loss value of the intention recognition model according to the target scene labels and the target prediction labels corresponding to the same target scene data;

[0121] a target model unit configured to determine the intention recognition model after convergence as a target intention recognition model when the scene loss value reaches a preset convergence condition.

[0122] In an embodiment, the target label prediction unit comprises:

[0123] an updating unit configured to set the number of classification categories of the classification layer of the intention recognition model to a preset number of categories of the target intention scene, to obtain an updated intention recognition model;

[0124] a target input vector unit configured to perform encoding processing on the target scene question and the target scene answer by the input layer of the updated intention recognition model, to obtain target input vectors corresponding to the target scene data;

[0125] a target semantic vector unit configured to perform semantic extraction on all the target input vectors by the semantic layer of the updated intention recognition model, to obtain target semantic vectors corresponding to the target scene data;

[0126] a target prediction label unit configured to perform intention recognition processing on all the target semantic vectors by the classification layer of the updated intention recognition model, to obtain target prediction labels corresponding to the target scene data.

[0127] The specific limitations of the intention recognition device can refer to the limitations of the intention recognition method described above, which will not be repeated here. Each module in the above intention recognition device can be realized by software, hardware and their combinations in whole or in part. The above modules can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory in the computer device in software form, so that the processor calls and executes the operations corresponding to each module.

[0128] In one embodiment, a computer device is provided, which can be a server, and its internal structure diagram can be as shown in Figure 6The computer device includes a processor, a memory, a network interface and a database connected through a system bus. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for running the operating system and the computer program in the non-volatile storage medium. The database of the computer device is configured to store data used by the intent recognition method. The network interface of the computer device is configured to communicate with an external terminal through a network connection. The computer program, when executed by the processor, implements the intent recognition method.

[0129] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the intent recognition method when executing the computer program.

[0130] In one embodiment, a computer readable storage medium is provided, which stores a computer program, wherein the computer program, when executed by a processor, implements the intent recognition method.

[0131] Those skilled in the art can understand that all or part of the processes of the above-mentioned embodiments can be completed by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer readable storage medium. When the computer program is executed, it can include the processes of the above-mentioned embodiments. Any reference to memory, storage, database or other medium used in the embodiments provided by the present application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0132] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above-mentioned division of each functional unit and module is exemplified, and in actual application, the above-mentioned functions can be completed by different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above.

[0133] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can be modified, or some technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. An intention recognition method characterized by, The method comprises the following steps: obtaining a historical unlabeled data set, the historical unlabeled data set comprising at least one historical dialogue data, the historical dialogue data comprising a historical question and a historical answer corresponding to the historical question; obtaining a preset training model, predicting the relevance of all the historical questions and all the historical answers by using the preset training model, and obtaining a question and answer relevance label; the question and answer relevance label is used to represent whether the historical question and the historical answer are relevant; iteratively training the preset training model by using the historical question, the historical answer and the question and answer relevance label corresponding to the same historical dialogue data, and obtaining an intent recognition model; obtaining a target scene data set, the target scene data set comprising at least one target scene data and a target scene label corresponding to the target scene data; the target scene data comprises a target scene question and a target scene answer; fine-tuning the intent recognition model by using all the target scene questions, all the target scene answers and the target scene label, and obtaining a target intent recognition model; obtaining at least one question to be recognized, and performing intent recognition on all the questions to be recognized based on the target intent recognition model, and obtaining an intent recognition result corresponding to each question to be recognized.

2. The intention recognition method of claim 1, wherein, The method comprises the following steps: encoding the historical question and the historical answer by using an input layer of the preset training model, and obtaining an input vector corresponding to the historical dialogue data; extracting semantics of all the input vectors by using a semantic layer of the preset training model, and obtaining a semantic vector corresponding to the historical dialogue data; performing prediction processing on all the semantic vectors by using a classification layer of the preset training model, and obtaining a question and answer relevance label corresponding to the historical dialogue data.

3. The intention recognition method according to claim 2, wherein The method comprises the following steps: performing similarity processing on the semantic vector and an input vector corresponding to the historical answer, and obtaining a similarity value; when the similarity value is greater than or equal to a preset similarity threshold, the question and answer relevance label represents that the historical question and the historical answer are relevant; when the similarity value is less than the preset similarity threshold, the question and answer relevance label represents that the historical question and the historical answer are not relevant.

4. The intention recognition method of claim 1, wherein, The method comprises the following steps: predicting all the historical dialogue data by using the preset training model, and obtaining a prediction label; determining a prediction loss value of the preset training model according to the historical question, the historical answer, the question and answer relevance label and the prediction label corresponding to the same historical dialogue data; When the predicted loss value does not reach the preset convergence condition, the initial parameters in the preset training model are iteratively updated until the predicted loss value reaches the convergence condition, and the preset training model after convergence is recorded as an intent recognition model.

5. The intention recognition method of claim 1, wherein, The fine-tuning of the intent recognition model by all the target scene questions, all the target scene answers and the target scene labels comprises: Label prediction of the target scene questions and the target scene answers by the intent recognition model to obtain target predicted labels; Determination of a scene loss value of the intent recognition model according to the target scene labels and the target predicted labels corresponding to the same target scene data; When the scene loss value reaches a preset convergence condition, the intent recognition model after convergence is determined as a target intent recognition model.

6. The intention recognition method according to claim 5, wherein The label prediction of the target scene questions and the target scene answers by the intent recognition model to obtain target predicted labels comprises: Setting the number of classification categories of the classification layer of the intent recognition model as a preset number of categories of target intent scenes to obtain an updated intent recognition model; Encoding processing of the target scene questions and the target scene answers by the input layer of the updated intent recognition model to obtain target input vectors corresponding to the target scene data; Semantic extraction of all the target input vectors by the semantic layer of the updated intent recognition model to obtain target semantic vectors corresponding to the target scene data; Intent recognition processing of all the target semantic vectors by the classification layer of the updated intent recognition model to obtain target predicted labels corresponding to the target scene data.

7. An intention recognition apparatus characterized by comprising: Comprise: A data acquisition module is configured to acquire a historical unlabeled data set, wherein the historical unlabeled data set comprises at least one historical dialogue data, and the historical dialogue data comprises a historical question and a historical answer corresponding to the historical question; A label prediction module is configured to acquire a preset training model, and predict the relevance of all the historical questions and all the historical answers by the preset training model to obtain a question and answer relevance label, wherein the question and answer relevance label is used to represent whether the historical question and the historical answer are relevant; A model training module is configured to iteratively train the preset training model by the historical question, the historical answer and the question and answer relevance label corresponding to the same historical dialogue data to obtain an intent recognition model; A target data acquisition module is configured to acquire a target scene data set, wherein the target scene data set comprises at least one target scene data and a target scene label corresponding to the target scene data, and the target scene data comprises a target scene question and a target scene answer; A model fine-tuning module is configured to fine-tune the intent recognition model by all the target scene questions, all the target scene answers and the target scene labels to obtain a target intent recognition model; The identification result module is configured to acquire at least one to-be-identified question, perform intent identification on all the to-be-identified questions based on the target intent identification model, and obtain an intent identification result corresponding to each to-be-identified question.

8. The intention recognition apparatus according to claim 7, characterized by, The label prediction module comprises: An input vector unit configured to perform encoding processing on the historical question and the historical answer through an input layer of the preset training model to obtain an input vector corresponding to the historical dialogue data; A semantic vector unit configured to perform semantic extraction on all the input vectors through a semantic layer of the preset training model to obtain a semantic vector corresponding to the historical dialogue data; A related label unit configured to perform prediction processing on all the semantic vectors through a classification layer of the preset training model to obtain a question-and-answer related label corresponding to the historical dialogue data.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the intent identification method in any one of claims 1 to 6.

10. A computer-readable storage medium storing a computer program, the computer program comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1-9. The computer program is executed by the processor to implement the intent identification method in any one of claims 1 to 6.

Images