Artificial intelligence-based intent recognition model training method and related device

Abstract

The application relates to the field of artificial intelligence and digital medicine, and proposes an intention recognition model training method and device based on artificial intelligence, an electronic device and a storage medium. The intention recognition model training method based on artificial intelligence comprises the following steps: performing word embedding on a plurality of natural sentences collected in advance to obtain a sentence vector of each natural sentence; identifying an intention vector and a slot vector of the sentence vector by using a preset semantic coding model; labeling the intention vector and the slot vector to construct a training data set; constructing an initial intention recognition model, training the initial intention recognition model by using the training data set, and obtaining an intention recognition model. The method can jointly train the intention recognition model by using the intention vector and the slot vector of the natural sentence, so that the accuracy of the intention recognition model can be improved.

Description

Artificial Intelligence-Based Intent Recognition Model Training Method and Related Equipment Technical Field

[0001] This application relates to the fields of artificial intelligence and digital healthcare technology, and in particular to an artificial intelligence-based intention recognition model training method, apparatus, electronic device and storage medium. Background Technology

[0002] With the development of artificial intelligence technology, Natural Language Understanding (NLU) has received widespread attention in human-computer interaction scenarios. The purpose of NLU is to identify the intent and fill in the slots for user-generated questions. Intent identification involves classifying the natural language statements input by the user and determining their semantic category. For example, when a patient consults online through a digital healthcare system, the system needs to identify the intent and fill in the slots for the patient's input of speech or text related to their symptoms to understand their needs and provide better medical services.

[0003] Currently, various task frameworks are typically used to model the intent recognition and slot filling tasks in digital healthcare systems separately to train intent recognition models applicable to digital healthcare systems. However, this approach does not consider the correlation between intent recognition and slot filling, thus resulting in low accuracy of the intent recognition model. Summary of the Invention

[0004] In view of the above, it is necessary to provide an artificial intelligence-based intention recognition model training method and related equipment to solve the technical problem of how to improve the accuracy of the intention recognition model. The related equipment includes an artificial intelligence-based intention recognition model training device, electronic equipment and storage medium.

[0005] This application provides a method for training an intent recognition model based on artificial intelligence, the method comprising:

[0006] Word embedding is performed on multiple pre-collected natural sentences to obtain the sentence vector of each natural sentence;

[0007] The intent vector and slot vector of the sentence vector are identified using a preset semantic encoding model;

[0008] The intent vector and the slot vector are labeled to construct a training dataset;

[0009] Construct an initial intent recognition model, and train the initial intent recognition model using the training dataset to obtain the intent recognition model.

[0010] In some embodiments, each natural language sentence includes multiple words, and the step of embedding words into the pre-collected multiple natural language sentences to obtain a sentence vector for each natural language sentence specifically includes:

[0011] For each of the natural sentences, each word in the natural sentence is encoded using a preset encoding dictionary to obtain a word vector for each word;

[0012] For each natural sentence, each word vector corresponding to the natural sentence is input into a preset context embedding model to obtain the context embedding vector of each word;

[0013] The context embedding vectors are combined according to the order of the words to obtain the sentence vector of the natural sentence.

[0014] In some embodiments, the preset semantic encoding model includes a first semantic encoding model and a second semantic encoding model, and the step of using the preset semantic encoding model to identify the intent vector and slot vector of the sentence vector specifically includes:

[0015] The sentence vector is input into the first semantic encoding model to obtain the intent vector corresponding to the sentence vector;

[0016] Each context embedding vector in the sentence vector is input into the second semantic encoding model to obtain the annotation vector corresponding to each word;

[0017] The tag vectors corresponding to each word are combined according to the order of the words in the natural sentence to obtain the slot vectors corresponding to the sentence vector.

[0018] In some embodiments, the step of labeling the intent vector and the slot vector to construct a training dataset specifically includes:

[0019] Query the first semantic category of the natural language statement corresponding to the intent vector, and construct a first probability list corresponding to the intent vector based on the first semantic category;

[0020] For each labeled vector in the slot vector, query the second semantic category of the word corresponding to the labeled vector, and construct a second probability list for each labeled vector based on the second semantic category;

[0021] The intent vectors are mapped one-to-one with the first probability list, and the slot vectors are mapped one-to-one with the second probability list to form a training dataset.

[0022] In some embodiments, the initial intent recognition model includes a first sub-model and a second sub-model, and training the initial intent recognition model using the training dataset to obtain the intent recognition model specifically includes:

[0023] Calculate the target intent vector and target slot vector based on the intent vector and slot vector in the training dataset;

[0024] The target intent vector is input into the first sub-model to obtain the predicted intent list, and the target slot vector is input into the second sub-model to obtain the predicted slot list;

[0025] Based on the predicted intent list, predicted slot list, first probability list and second probability list, the loss value of the initial intent recognition model is calculated. The initial intent recognition model is iteratively updated using the gradient descent method until the loss value is less than a preset termination threshold, at which point the iteration stops and the intent recognition model is obtained.

[0026] In some embodiments, calculating the target intent vector and the target slot vector based on the intent vector and slot vector in the training dataset specifically includes:

[0027] A. For each of the natural statements, the slot vector corresponding to the natural statement is used as the first vector, and the intent vector corresponding to the natural statement is used as the second vector. The relevance benchmark value is initialized to 0.

[0028] B, Calculate the correlation between the first vector and the second vector;

[0029] C, the product of the correlation and the slot vector is used as the enhanced slot vector;

[0030] D, Calculate the enhanced intent vector based on the enhanced slot vector and the intent vector;

[0031] E. If the difference between the correlation and the correlation benchmark value is greater than the preset correlation threshold, then the enhanced slot vector is used as the first vector, the enhanced intent vector is used as the second vector, the correlation is used as the correlation benchmark value, and steps B to E are repeated. If the difference between the correlation and the correlation benchmark value is not greater than the preset correlation threshold, then the enhanced slot vector is used as the target slot vector, and the enhanced intent vector is used as the target intent vector.

[0032] In some embodiments, calculating the loss value of the initial intent recognition model based on the predicted intent list, the predicted slot list, the first probability list, and the second probability list specifically includes:

[0033] The predicted intent list and the first probability list are input into a preset first loss function to calculate the first loss value corresponding to the natural statement, and the predicted slot list and the second probability list are input into the preset second loss function to calculate the second loss value corresponding to the natural statement;

[0034] The average of the first loss value and the second loss value is used as the loss value of the initial intent recognition model.

[0035] This application embodiment also provides an artificial intelligence-based intent recognition model training device, the device comprising:

[0036] An embedding unit is used to perform word embedding on multiple pre-collected natural sentences to obtain a sentence vector for each natural sentence;

[0037] The encoding unit is used to identify the intent vector and slot vector of the sentence vector using a preset semantic encoding model;

[0038] A construction unit is used to label the intent vector and the slot vector to construct a training dataset;

[0039] A joint training unit is used to construct an initial intent recognition model, and to train the initial intent recognition model using the training dataset to obtain an intent recognition model.

[0040] This application embodiment also provides an electronic device, the electronic device comprising:

[0041] Memory, which stores computer-readable instructions; and

[0042] The processor executes computer-readable instructions stored in the memory to implement the artificial intelligence-based intent recognition model training method.

[0043] This application also provides a computer-readable storage medium storing computer-readable instructions, which are executed by a processor in an electronic device to implement the artificial intelligence-based intent recognition model training method.

[0044] The aforementioned AI-based intent recognition model training method involves embedding words into multiple pre-collected natural sentences to obtain sentence vectors for each sentence; using a pre-defined semantic encoding model to identify the intent vector and slot vector of the sentence vectors; labeling the intent vector and slot vector to construct a training dataset; building an initial intent recognition model; and training the initial intent recognition model using the training dataset to obtain the final intent recognition model. This method can jointly train the intent recognition model using the intent vector and slot vector of natural sentences, thereby improving the accuracy of the intent recognition model. Attached Figure Description

[0045] Figure 1 is a flowchart of a preferred embodiment of an artificial intelligence-based intent recognition model training method involved in this application.

[0046] Figure 2 is a functional block diagram of a preferred embodiment of the artificial intelligence-based intent recognition model training device involved in this application.

[0047] Figure 3 is a schematic diagram of the structure of an electronic device according to a preferred embodiment of the artificial intelligence-based intent recognition model training method involved in this application. Detailed Implementation

[0048] To better understand the purpose, features, and advantages of this application, a detailed description of the application is provided below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in the embodiments of this application can be combined with each other. Numerous specific details are set forth in the following description to provide a thorough understanding of this application; the described embodiments are only a part of the embodiments of this application, and not all of them.

[0049] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the stated features. In the description of this application, "a plurality of" means two or more, unless otherwise explicitly specified.

[0050] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0051] This application provides an artificial intelligence-based intent recognition model training method, which can be applied to one or more electronic devices. The electronic device is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions. Its hardware includes, but is not limited to, microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0052] The electronic device can be any electronic product that can interact with the user, such as a personal computer, tablet computer, smartphone, personal digital assistant (PDA), game console, interactive network television (IPTV), smart wearable device, etc.

[0053] The electronic device may also include network devices and / or user devices. The network devices include, but are not limited to, a single network server, a server group consisting of multiple network servers, or a cloud based on cloud computing consisting of a large number of hosts or network servers.

[0054] The networks in which the electronic devices are located include, but are not limited to, the Internet, wide area networks, metropolitan area networks, local area networks, and virtual private networks (VPNs).

[0055] The intent recognition model training method described in this solution can be applied to fields such as the Internet of Things (IoT) and digital healthcare. For example, when applied to the IoT field, the interactive voice between the user and the IoT device can be used as natural language, and the interactive voice can be embedded and encoded to obtain the intent vector and slot vector corresponding to the interactive voice. Furthermore, the intent vector and slot vector are simultaneously used to train the intent recognition model. When applied to the digital healthcare field, the interactive information between the patient and the dialogue model assisting in diagnosis can be used as natural language, and the interactive information can be embedded and encoded to obtain the intent vector and slot vector corresponding to the interactive information. Finally, the intent vector and slot vector corresponding to the interactive information can be simultaneously used to train an intent recognition model suitable for the digital healthcare field.

[0056] Figure 1 shows a flowchart of a preferred embodiment of the artificial intelligence-based intent recognition model training method of this application. The order of the steps in this flowchart can be changed, and some steps can be omitted, depending on different requirements.

[0057] S10, perform word embedding on multiple pre-collected natural sentences to obtain the sentence vector of each natural sentence.

[0058] In an optional embodiment, each natural sentence includes multiple words, and the step of embedding words into the pre-collected multiple natural sentences to obtain a sentence vector for each natural sentence specifically includes:

[0059] For each of the natural sentences, each word in the natural sentence is encoded using a preset encoding dictionary to obtain a word vector for each word;

[0060] For each natural sentence, each word vector corresponding to the natural sentence is input into a preset context embedding model to obtain the context embedding vector of each word;

[0061] The context embedding vectors are combined according to the order of the words to obtain the sentence vector of the natural sentence.

[0062] In this optional embodiment, the natural language statement can be a natural language dialogue collected by the IoT device, or it can be an instruction received by the IoT device; this application does not limit this. For example, when the natural language is a consultation voice collected by the digital medical system, the natural language could be "My stomach has been hurting for three days" or "He felt a headache three hours ago." The natural language can also be medical text that the digital medical system needs to recognize. The medical text can be an electronic healthcare record, an electronic personal health record, including medical records, electrocardiograms, medical images, and other electronic records with archival value. The natural language statement can also be user dialogue received by the IoT device in daily life, such as "What's the weather like today?" or "Watch an action movie."

[0063] In this optional embodiment, the preset encoding dictionary is used to record the word vector corresponding to each word. The word vector can be a vector of any dimension, and this application does not limit it. For example, the word vector corresponding to the word "weather" can be in the form of [1,2,3,4,5,6].

[0064] In this optional embodiment, for each natural sentence, the word vectors corresponding to each word can be sequentially input into a preset context embedding model according to the word order in the natural sentence to obtain the context embedding vector of each word, and the context embedding vectors can be combined according to the word order to obtain the sentence vector of the natural sentence. The preset context embedding model refers to a model with word embedding function, which is used to convert the word vectors corresponding to the sentence into word vectors with context information. For example, the preset context embedding model can be an existing neural network model with word embedding function, such as the ELMO model or the BiLSTM model (bidirectional long short-term memory model), and this application does not limit it to this.

[0065] Thus, adding contextual information to the word vectors of natural sentences can improve the accuracy of subsequent intent recognition.

[0066] S11, using a preset semantic encoding model to identify the intent vector and slot vector of the sentence vector.

[0067] In an optional embodiment, the preset semantic encoding model includes a first semantic encoding model and a second semantic encoding model. The step of using the preset semantic encoding model to identify the intent vector and slot vector of the sentence vector specifically includes:

[0068] The sentence vector is input into the first semantic encoding model to obtain the intent vector corresponding to the sentence vector;

[0069] Each context embedding vector in the sentence vector is input into the second semantic encoding model to obtain the annotation vector corresponding to each word;

[0070] The tag vectors corresponding to each word are combined according to the order of the words in the natural sentence to obtain the slot vectors corresponding to the sentence vector.

[0071] In this optional embodiment, the preset first semantic encoding model refers to a pre-trained model with intent recognition capabilities. The input of the preset first semantic encoding model is the sentence vector, and the output is the intent vector corresponding to the sentence vector. The intent vector is used to represent the intent of the natural language corresponding to the sentence vector. For example, the preset first semantic encoding model can be an existing neural network model such as the BERT model, LSTM model, or RNN model, and this application does not limit it.

[0072] In this optional embodiment, the input of the preset second semantic encoding model is the context embedding vector in the sentence vector, and the output is the annotation vector corresponding to each context embedding vector. The annotation vector is used to represent the category corresponding to each word. For example, the category can be the semantic category of the slot where the word is located, such as subject, predicate, object, etc.

[0073] In this optional embodiment, the annotation vectors corresponding to each word can be combined according to the order of the words in the natural sentence to obtain the slot vector corresponding to the sentence vector. For example, when the order of words in the natural sentence is [word 1, word 2, word 3], and the annotation vector corresponding to word 1 is [1,2,3,4], the annotation vector corresponding to word 2 is [2,3,4,5], and the annotation vector corresponding to word 3 is [3,4,5,6], then the slot vector is {[1,2,3,4],[2,3,4,5],[3,4,5,6]}.

[0074] Thus, by outputting the intent vector and slot vector corresponding to the sentence vector through the semantic encoding model, the intent type and slot type of the natural sentence are represented in a quantitative form, providing data support for the subsequent training of the intent recognition model.

[0075] S12, the intent vector and the slot vector are labeled to construct a training dataset.

[0076] In an optional embodiment, the step of labeling the intent vector and the slot vector to construct a training dataset specifically includes:

[0077] Query the first semantic category of the natural language statement corresponding to the intent vector, and construct a first probability list corresponding to the intent vector based on the first semantic category;

[0078] For each labeled vector in the slot vector, query the second semantic category of the word corresponding to the labeled vector, and construct a second probability list for each labeled vector based on the second semantic category;

[0079] The intent vectors are mapped one-to-one with the first probability list, and the slot vectors are mapped one-to-one with the second probability list to form a training dataset.

[0080] In this optional embodiment, the first semantic category refers to the meaning of the natural statement corresponding to the intent vector. For example, when the semantic categories of all natural statements include [query weather, query temperature, query books, query movies], and the natural statement is "query today's weather", then the first semantic category corresponding to the natural statement is "query weather". The first probability list corresponding to the intent vector can be in the form of [1,0,0,0]. The first probability list is used to represent that the probability of the first semantic category of the natural statement being "query weather" is 1, and the probability of being other categories is 0.

[0081] In this optional embodiment, the second semantic category is used to characterize the semantic type corresponding to each word. For example, when the semantic types corresponding to all words include [subject, verb, modifier, object], and when the words include "query", "today", and "weather", the second semantic category corresponding to the word "query" can be in the form of [0, 1, 0, 0], which is used to characterize that the probability of the semantic category of the word "query" in the natural sentence being "verb" is 1, and the probability of being any of the other semantic categories is 0.

[0082] Thus, a first probability list and a second probability list are constructed based on the semantics of the natural sentence and the semantics of each word therein, respectively, providing data support for the subsequent training of the intent recognition model.

[0083] S13, Construct an initial intent recognition model, and train the initial intent recognition model using the training dataset to obtain the intent recognition model.

[0084] In an optional embodiment, the initial intent recognition model includes a first sub-model and a second sub-model, and the step of training the initial intent recognition model using the training dataset to obtain the intent recognition model specifically includes:

[0085] Calculate the target intent vector and target slot vector based on the intent vector and slot vector in the training dataset;

[0086] The target intent vector is input into the first sub-model to obtain the predicted intent list, and the target slot vector is input into the second sub-model to obtain the predicted slot list;

[0087] Based on the predicted intent list, predicted slot list, first probability list and second probability list, the loss value of the initial intent recognition model is calculated. The initial intent recognition model is iteratively updated using the gradient descent method until the loss value is less than a preset termination threshold, at which point the iteration stops and the intent recognition model is obtained.

[0088] In an optional embodiment, the step of calculating the target intent vector and target slot vector based on the intent vector and slot vector in the training dataset specifically includes:

[0089] A. For each of the natural statements, the slot vector corresponding to the natural statement is used as the first vector, and the intent vector corresponding to the natural statement is used as the second vector. The relevance baseline value is initialized to 0.

[0090] In this optional embodiment, the intent vector corresponding to the natural language statement can be denoted as v. intent The slot vector corresponding to the natural statement can be denoted as v. slot And the slot vector T is the number of words in the natural language statement, for example, This represents the first labeled vector in the slot vector corresponding to the natural statement.

[0091] B. Calculate the correlation between the first vector and the second vector.

[0092] In this optional embodiment, the correlation f between the intent vector and the slot vector can be calculated, and the correlation f is calculated in a manner that satisfies the following relationship:

[0093]

[0094] Where U and W represent preset weight parameters; tanh represents a hyperbolic sine function; This represents the i-th labeled vector in the slot vector corresponding to the natural sentence; the value of i is (1, T), where T represents the number of words in the natural sentence; v intent This represents the intent vector corresponding to the natural statement.

[0095] C, the product of the correlation and the slot vector is used as the enhanced slot vector.

[0096] In this optional embodiment, after obtaining the correlation f, the product of the correlation and each of the labeled vectors can be used as the enhanced slot vector, and the enhanced slot vector is calculated as follows:

[0097]

[0098]

[0099] Where f represents the correlation; This represents the i-th label vector in the slot vector; The i-th augmented label vector is obtained by multiplying the relevance and the i-th label vector; r slot The vector represents the augmented slot vector; T represents the number of words in the natural language sentence, that is, the number of labeled vectors contained in the slot vector. For example, when T is 3, the augmented slot vector is...

[0100] D, Calculate the enhanced intent vector based on the enhanced slot vector and the intent vector.

[0101] In this optional embodiment, an enhanced intent vector can be calculated using the enhanced slot vector and the intent vector to fuse the slot information with the intent vector, so that the enhanced intent vector contains both slot information and intent vector information. The calculation method of the enhanced intent vector satisfies the following relationship:

[0102]

[0103] Where, r intent Represents the enhanced intent vector; v intent The intent vector representing the natural language statement; α i This represents the preset attention weight corresponding to the i-th slot.

[0104] E. If the difference between the correlation and the correlation benchmark value is greater than the preset correlation threshold, then the enhanced slot vector is used as the first vector, the enhanced intent vector is used as the second vector, and steps B to E are repeated. If the difference between the correlation and the correlation benchmark value is not greater than the preset correlation threshold, then the enhanced slot vector is used as the target slot vector, and the enhanced intent vector is used as the target intent vector.

[0105] In this way, by repeatedly calculating the similarity between the intent vector and the slot vector, and using the similarity to iteratively update the intent vector and the slot vector, the information in the intent vector and the slot vector are integrated with each other, ensuring that the target intent vector and the target slot vector can comprehensively encompass global information, thereby improving the performance of the subsequent intent recognition model.

[0106] In this optional embodiment, the target intent vector is input into a first sub-model to obtain a predicted intent list, and the target slot vector is input into a second sub-model to obtain a predicted slot list. Both the first and second sub-models are classification models. For example, both the first and second sub-models can be pre-trained convolutional neural networks, recurrent neural networks, or other existing models with classification capabilities; this application does not limit this. The input to the first sub-model is the target intent vector, and the output of the first sub-model is the predicted intent list corresponding to the target intent vector. The predicted intent list includes multiple intent prediction values, each of which represents the probability that the natural language statement belongs to a certain semantic intent.

[0107] For example, when the predicted intent list is [0.1, 0.2, 0.3, 0.4], the predicted intent list is used to characterize the probability that the semantic intent of the natural statement is "to query the weather" is 0.1, the probability that the semantic intent is "to query the temperature" is 0.2, the probability that the semantic intent is "to query books" is 0.3, and the probability that the semantic intent is "to query movies" is 0.4.

[0108] For example, when the predicted slot list is {[0,0.8,0.1,0.1],[0,0,0.9,0.1],[0,0,0.1,0.9]}, then this predicted slot list is used to characterize the probability that the semantic category of the first word in the natural sentence is a verb is 0.8, the probability that it is an attributive is 0.1, and the probability that it is an object is 0.1; and the probability that the semantic category of the second word in the natural sentence is a subject and a verb is 0, the probability that it is an attributive is 0.9, and the probability that it is an object is 0.1; and the probability that the semantic category of the third word in the natural sentence is a subject and a verb is 0, the probability that it is an attributive is 0.1, and the probability that it is an object is 0.9.

[0109] In an optional embodiment, calculating the loss value of the initial intent recognition model based on the predicted intent list, the predicted slot list, the first probability list, and the second probability list specifically includes:

[0110] The predicted intent list and the first probability list are input into a preset first loss function to calculate the first loss value corresponding to the natural statement, and the predicted slot list and the second probability list are input into the preset second loss function to calculate the second loss value corresponding to the natural statement;

[0111] The average of the first loss value and the second loss value is used as the loss value of the initial intent recognition model.

[0112] In this optional embodiment, the preset loss function can be an existing loss function such as the cross-entropy loss function or the contrastive loss function, and this application does not limit it.

[0113] In this optional embodiment, the initial intent recognition model can be iteratively updated using the gradient descent method until the loss value is less than a preset termination threshold, indicating that the performance of the initial intent recognition model has stabilized, and the update can be stopped to obtain the intent recognition model.

[0114] In this way, the average of the first loss value and the second loss value is used as the loss value of the intent recognition model, ensuring that the first sub-model and the second sub-model have the same degree of influence on the intent recognition model, thereby balancing the importance of the first sub-model and the second sub-model.

[0115] The aforementioned AI-based intent recognition model training method involves embedding words into multiple pre-collected natural sentences to obtain sentence vectors for each sentence; using a pre-defined semantic encoding model to identify the intent vector and slot vector of the sentence vectors; labeling the intent vector and slot vector to construct a training dataset; building an initial intent recognition model; and training the initial intent recognition model using the training dataset to obtain the final intent recognition model. This method can jointly train the intent recognition model using the intent vector and slot vector of natural sentences, thereby improving the accuracy of the intent recognition model.

[0116] The intent recognition model can be applied to any operational scenario requiring natural language understanding. For example, when the intent recognition model is trained using natural language from an e-commerce platform, it can be used to assist the platform's intelligent customer service. The model receives customer inquiries (natural language) from the e-commerce platform, performs intent recognition on the inquiries, and outputs intent vectors and slot vectors corresponding to the inquiries to represent the semantic information of the natural language, providing response suggestions to the intelligent customer service. When the intent recognition model is trained using natural language from the medical field, it can be applied to an assisted consultation system. The model receives consultation information (natural language) from patients, performs intent recognition on the consultation information, obtains the semantics corresponding to the consultation information, and provides semantic suggestions for responding to patients.

[0117] Figure 2 shows a functional block diagram of a preferred embodiment of the AI-based intent recognition model training device provided in this application. The AI-based intent recognition model training device 11 includes an embedding unit 110, an encoding unit 111, a construction unit 112, and a joint training unit 113. The module / unit referred to in this application refers to a series of computer program segments that can be executed by the processor 13 and perform a fixed function, and are stored in the memory 12. In this embodiment, the functions of each module / unit will be described in detail in subsequent embodiments.

[0118] The embedding unit 110 is used to embed words into a plurality of pre-collected natural sentences to obtain a sentence vector for each natural sentence.

[0119] The encoding unit 111 is used to identify the intent vector and slot vector of the sentence vector using a preset semantic encoding model.

[0120] The construction unit 112 is used to label the intent vector and the slot vector to construct a training dataset.

[0121] The joint training unit 113 is used to construct an initial intent recognition model, train the initial intent recognition model using the training dataset, and obtain an intent recognition model.

[0122] In an optional embodiment, the embedding unit 110 is specifically used for:

[0123] For each of the natural sentences, each word in the natural sentence is encoded using a preset encoding dictionary to obtain a word vector for each word;

[0124] For each natural sentence, each word vector corresponding to the natural sentence is input into a preset context embedding model to obtain the context embedding vector of each word;

[0125] The context embedding vectors are combined according to the order of the words to obtain the sentence vector of the natural sentence.

[0126] In an optional embodiment, the encoding unit 111 is specifically used for:

[0127] The sentence vector is input into the first semantic encoding model to obtain the intent vector corresponding to the sentence vector;

[0128] Each context embedding vector in the sentence vector is input into the second semantic encoding model to obtain the annotation vector corresponding to each word;

[0129] The tag vectors corresponding to each word are combined according to the order of the words in the natural sentence to obtain the slot vectors corresponding to the sentence vector.

[0130] In an optional embodiment, the building unit 112 is specifically used for:

[0131] Query the first semantic category of the natural language statement corresponding to the intent vector, and construct a first probability list corresponding to the intent vector based on the first semantic category;

[0132] For each labeled vector in the slot vector, query the second semantic category of the word corresponding to the labeled vector, and construct a second probability list for each labeled vector based on the second semantic category;

[0133] The intent vectors are mapped one-to-one with the first probability list, and the slot vectors are mapped one-to-one with the second probability list to form a training dataset.

[0134] In an optional embodiment, the joint training unit 113 is specifically used for:

[0135] Calculate the target intent vector and target slot vector based on the intent vector and slot vector in the training dataset;

[0136] The target intent vector is input into the first sub-model to obtain the predicted intent list, and the target slot vector is input into the second sub-model to obtain the predicted slot list;

[0137] Based on the predicted intent list, predicted slot list, first probability list and second probability list, the loss value of the initial intent recognition model is calculated. The initial intent recognition model is iteratively updated using the gradient descent method until the loss value is less than a preset termination threshold, at which point the iteration stops and the intent recognition model is obtained.

[0138] In an optional embodiment, the joint training unit 113 is further configured to:

[0139] A. For each of the natural statements, the slot vector corresponding to the natural statement is used as the first vector, and the intent vector corresponding to the natural statement is used as the second vector. The relevance benchmark value is initialized to 0.

[0140] B, Calculate the correlation between the first vector and the second vector;

[0141] C, the product of the correlation and the slot vector is used as the enhanced slot vector;

[0142] D, Calculate the enhanced intent vector based on the enhanced slot vector and the intent vector;

[0143] E. If the difference between the correlation and the correlation benchmark value is greater than the preset correlation threshold, then the enhanced slot vector is used as the first vector, the enhanced intent vector is used as the second vector, the correlation is used as the correlation benchmark value, and steps B to E are repeated. If the difference between the correlation and the correlation benchmark value is not greater than the preset correlation threshold, then the enhanced slot vector is used as the target slot vector, and the enhanced intent vector is used as the target intent vector.

[0144] In an optional embodiment, the joint training unit 113 is further configured to:

[0145] The predicted intent list and the first probability list are input into a preset first loss function to calculate the first loss value corresponding to the natural statement, and the predicted slot list and the second probability list are input into the preset second loss function to calculate the second loss value corresponding to the natural statement;

[0146] The average of the first loss value and the second loss value is used as the loss value of the initial intent recognition model.

[0147] Figure 3 shows a schematic diagram of an electronic device provided in an embodiment of this application. The electronic device 1 includes a memory 12 and a processor 13. The memory 12 is used to store computer-readable instructions, and the processor 13 uses the computer-readable instructions stored in the memory to implement the artificial intelligence-based intent recognition model training method of any of the above embodiments.

[0148] In an alternative embodiment, the electronic device 1 further includes a bus and a computer program stored in memory 12 and executable on processor 13, such as an artificial intelligence-based intent recognition model training program.

[0149] Figure 3 only shows an electronic device 1 with a memory 12 and a processor 13. Those skilled in the art will understand that the structure shown in Figure 3 does not constitute a limitation on the electronic device 1, and may include fewer or more components than shown, or combine certain components, or have different component arrangements.

[0150] Referring to Figure 1, the memory 12 in the electronic device 1 stores multiple computer-readable instructions to implement an artificial intelligence-based intent recognition model training method, and the processor 13 can execute multiple instructions to achieve:

[0151] Word embedding is performed on multiple pre-collected natural sentences to obtain the sentence vector of each natural sentence;

[0152] The intent vector and slot vector of the sentence vector are identified using a preset semantic encoding model;

[0153] The intent vector and the slot vector are labeled to construct a training dataset;

[0154] Construct an initial intent recognition model, and train the initial intent recognition model using the training dataset to obtain the intent recognition model.

[0155] Specifically, the specific implementation method of the processor 13 for the above instructions can be referred to the description of the relevant steps in the embodiment corresponding to Figure 1, which will not be repeated here.

[0156] Those skilled in the art will understand that the schematic diagram is merely an example of electronic device 1 and does not constitute a limitation on electronic device 1. Electronic device 1 can be either a bus topology or a star topology. Electronic device 1 may also include more or fewer other hardware or software than shown in the diagram, or different component arrangements. For example, electronic device 1 may also include input / output devices, network access devices, etc.

[0157] It should be noted that electronic device 1 is only an example. Other existing or future electronic products that are suitable for this application should also be included within the scope of protection of this application and are incorporated herein by reference.

[0158] The memory 12 includes at least one type of readable storage medium, which can be non-volatile or volatile. The readable storage medium includes flash memory, portable hard drives, multimedia cards, card-type memory (e.g., SD or DX memory), magnetic storage, magnetic disks, optical disks, etc. In some embodiments, the memory 12 can be an internal storage unit of the electronic device 1, such as a portable hard drive of the electronic device 1. In other embodiments, the memory 12 can be an external storage device of the electronic device 1, such as a plug-in portable hard drive, Smart Media Card (SMC), Secure Digital (SD) card, Flash Card, etc., equipped on the electronic device 1. Furthermore, the memory 12 can include both internal storage units and external storage devices of the electronic device 1. The memory 12 can be used not only to store application software and various types of data installed on the electronic device 1, such as the code of an artificial intelligence-based intent recognition model training program, but also to temporarily store data that has been output or will be output.

[0159] In some embodiments, the processor 13 may be composed of integrated circuits, such as a single packaged integrated circuit or multiple integrated circuits packaged with the same or different functions, including combinations of one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and various control chips. The processor 13 is the control unit of the electronic device 1, connecting various components of the electronic device 1 through various interfaces and lines. It executes programs or modules stored in the memory 12 (e.g., executing an artificial intelligence-based intent recognition model training program) and calls data stored in the memory 12 to perform various functions and process data in the electronic device 1.

[0160] The processor 13 executes the operating system of the electronic device 1 and various installed applications. The processor 13 executes the applications to implement the steps in the above-described embodiments of the artificial intelligence-based intent recognition model training method, such as the steps shown in FIG1.

[0161] For example, the computer program may be divided into one or more modules / units, which are stored in memory 12 and executed by processor 13 to complete this application. The one or more modules / units may be a series of computer-readable instruction segments capable of performing a specific function, which describe the execution process of the computer program in electronic device 1. For example, the computer program may be divided into an embedding unit 110, an encoding unit 111, a construction unit 112, and a joint training unit 113.

[0162] The integrated unit implemented as a software functional module described above can be stored in a computer-readable storage medium. This software functional module, stored in a storage medium, includes several instructions to cause a computer device (which may be a personal computer, computer equipment, or network device, etc.) or processor to execute portions of the artificial intelligence-based intent recognition model training method described in the various embodiments of this application.

[0163] If the modules / units integrated in electronic device 1 are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware devices. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above.

[0164] The computer program includes computer program code, which may be in the form of source code, object code, executable file, or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory, and other memory.

[0165] Furthermore, the computer-readable storage medium may primarily include a stored program area and a stored data area, wherein the stored program area may store the operating system, an application program required for at least one function, etc.; and the stored data area may store data created based on the use of blockchain nodes, etc.

[0166] The bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one arrow is used in Figure 3, but this does not mean that there is only one bus or one type of bus. The bus is configured to implement communication between memory 12 and at least one processor 13, etc.

[0167] This application also provides a computer-readable storage medium (not shown), which stores computer-readable instructions. These computer-readable instructions are executed by a processor in an electronic device to implement the artificial intelligence-based intent recognition model training method described in any of the above embodiments.

[0168] Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices described in the specification may also be implemented by a single unit or device through software or hardware. Terms such as "first," "second," etc., are used to indicate names and do not indicate any specific order.

[0169] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application and are not intended to limit it. Although this application has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of this application without departing from the spirit and scope of the technical solutions of this application.

Claims

1. A method for training an intent recognition model based on artificial intelligence, characterized in that, The method includes: embedding words into multiple pre-collected natural sentences to obtain sentence vectors for each natural sentence; identifying intent vectors and slot vectors of the sentence vectors using a preset semantic coding model; labeling the intent vectors and slot vectors to construct a training dataset; constructing an initial intent recognition model; training the initial intent recognition model using the training dataset to obtain an intent recognition model, including: calculating target intent vectors and target slot vectors based on the intent vectors and slot vectors in the training dataset; training the intent recognition model based on the target intent vectors and target slot vectors; specifically, calculating the target intent vectors and target slot vectors based on the intent vectors and slot vectors in the training dataset includes: A, for each natural sentence, using the slot vector corresponding to the natural sentence as a first vector and the intent vector corresponding to the natural sentence as a second vector, and initializing the relevance baseline value to 0; B, calculating the relevance between the first vector and the second vector; the calculation of the relevance f satisfies the following relationship: Where U and W represent preset weight parameters; tanh represents the hyperbolic tangent function; This represents the i-th label vector in the slot vector corresponding to the natural sentence; the value of i is (1, T), where T represents the number of words in the natural sentence; C represents the intent vector corresponding to the natural language statement; D represents the enhanced slot vector, which is the product of the relevance and the slot vector; E represents the enhanced intent vector, which is calculated based on the enhanced slot vector and the intent vector. The calculation method of the enhanced intent vector satisfies the following relationship: ;in, Represents the enhanced intent vector; The intent vector representing the natural language statement; E represents the preset attention weight corresponding to the i-th slot; if the difference between the correlation and the correlation benchmark value is greater than the preset correlation threshold, then the enhanced slot vector is used as the first vector, the enhanced intent vector is used as the second vector, the correlation is used as the correlation benchmark value, and steps B to E are repeated; if the difference between the correlation and the correlation benchmark value is not greater than the preset correlation threshold, then the enhanced slot vector is used as the target slot vector, and the enhanced intent vector is used as the target intent vector.

2. The artificial intelligence-based intent recognition model training method as described in claim 1, characterized in that, Each natural language statement includes multiple words. The step of embedding words into the pre-collected multiple natural language statements to obtain a sentence vector for each natural language statement specifically includes: for each natural language statement, encoding each word in the natural language statement using a preset encoding dictionary to obtain a word vector for each word; for each natural language statement, inputting each word vector corresponding to the natural language statement into a preset context embedding model to obtain a context embedding vector for each word; and combining the context embedding vectors according to the order of the words to obtain the sentence vector of the natural language statement.

3. The artificial intelligence-based intent recognition model training method as described in claim 2, characterized in that, The preset semantic encoding model includes a first semantic encoding model and a second semantic encoding model. The step of using the preset semantic encoding model to identify the intent vector and slot vector of the sentence vector specifically includes: inputting the sentence vector into the first semantic encoding model to obtain the intent vector corresponding to the sentence vector; inputting each context embedding vector in the sentence vector into the second semantic encoding model to obtain the annotation vector corresponding to each word; and combining the annotation vectors corresponding to each word according to the order of the words in the natural sentence to obtain the slot vector corresponding to the sentence vector.

4. The artificial intelligence-based intent recognition model training method as described in claim 1, characterized in that, The step of labeling the intent vector and the slot vector to construct a training dataset specifically includes: querying the first semantic category of the natural language corresponding to the intent vector, and constructing a first probability list corresponding to the intent vector based on the first semantic category; for each labeled vector in the slot vector, querying the second semantic category of the word corresponding to the labeled vector, and constructing a second probability list for each labeled vector based on the second semantic category; and mapping the intent vector to the first probability list one by one, and mapping the slot vector to the second probability list one by one to use as the training dataset.

5. The artificial intelligence-based intent recognition model training method as described in claim 1, characterized in that, The initial intent recognition model includes a first sub-model and a second sub-model. Training the initial intent recognition model using the training dataset to obtain the intent recognition model specifically includes: calculating a target intent vector and a target slot vector based on the intent vector and slot vector in the training dataset; inputting the target intent vector into the first sub-model to obtain a predicted intent list, and inputting the target slot vector into the second sub-model to obtain a predicted slot list; calculating the loss value of the initial intent recognition model based on the predicted intent list, the predicted slot list, the first probability list, and the second probability list; iteratively updating the initial intent recognition model using gradient descent until the loss value is less than a preset termination threshold, stopping the iteration and obtaining the intent recognition model.

6. The artificial intelligence-based intent recognition model training method as described in claim 5, characterized in that, The step of calculating the loss value of the initial intent recognition model based on the predicted intent list, the predicted slot list, the first probability list, and the second probability list specifically includes: inputting the predicted intent list and the first probability list into a preset first loss function to calculate the first loss value corresponding to the natural sentence, and inputting the predicted slot list and the second probability list into the preset second loss function to calculate the second loss value corresponding to the natural sentence; and using the average of the first loss value and the second loss value as the loss value of the initial intent recognition model.

7. A training device for an intent recognition model based on artificial intelligence, characterized in that, The device includes: an embedding unit for embedding words into multiple pre-collected natural sentences to obtain a sentence vector for each natural sentence; an encoding unit for recognizing the intent vector and slot vector of the sentence vector using a preset semantic encoding model; a construction unit for labeling the intent vector and the slot vector to construct a training dataset; and a joint training unit for constructing an initial intent recognition model, training the initial intent recognition model using the training dataset, and obtaining an intent recognition model, including: calculating a target intent vector and a target slot vector based on the intent vector and slot vector in the training dataset, and training the intent recognition model based on the target intent vector and the target slot vector; the calculation of the target intent vector and target slot vector based on the intent vector and slot vector in the training dataset specifically includes: A, for each natural sentence, taking the slot vector corresponding to the natural sentence as a first vector and the intent vector corresponding to the natural sentence as a second vector, and initializing the relevance benchmark value to 0; B, calculating the relevance between the first vector and the second vector; the calculation method of the relevance f satisfies the following relationship: Where U and W represent preset weight parameters; tanh represents the hyperbolic tangent function; This represents the i-th annotation vector in the slot vector corresponding to the natural sentence; the value of i is (1, T), where T represents the number of words in the natural sentence; C represents the intent vector corresponding to the natural language statement; D represents the enhanced slot vector, which is the product of the relevance and the slot vector; E represents the enhanced intent vector, which is calculated based on the enhanced slot vector and the intent vector. The calculation method of the enhanced intent vector satisfies the following relationship: ;in, Represents the enhanced intent vector; The intent vector representing the natural language statement; E represents the preset attention weight corresponding to the i-th slot; if the difference between the correlation and the correlation benchmark value is greater than the preset correlation threshold, then the enhanced slot vector is used as the first vector, the enhanced intent vector is used as the second vector, the correlation is used as the correlation benchmark value, and steps B to E are repeated; if the difference between the correlation and the correlation benchmark value is not greater than the preset correlation threshold, then the enhanced slot vector is used as the target slot vector, and the enhanced intent vector is used as the target intent vector.

8. An electronic device, characterized in that, The electronic device includes: a memory for storing computer-readable instructions; and a processor for executing the computer-readable instructions stored in the memory to implement the artificial intelligence-based intent recognition model training method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-readable instructions, which are executed by a processor in an electronic device to implement the artificial intelligence-based intent recognition model training method as described in any one of claims 1 to 6.

Images