Intention recognition method and device, electronic equipment and readable storage medium

Abstract

The application provides an intention recognition method and device, electronic equipment and readable storage medium. The method comprises: obtaining a first text of a multi-round conversation; obtaining a first intention corresponding to the first text recognized by a trained first model; in the case that the first text contains a preset first keyword, performing semantic similarity calculation on the first keyword and the first intention to obtain a first similarity value; and determining a target intention corresponding to the first text according to a comparison result between the first similarity value and a first threshold. The application can make the target intention corresponding to the first text combine the context information of the multi-round conversation, thereby improving the accuracy of intention recognition of the multi-round conversation text.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the field of natural language processing, and particularly relate to an intent recognition method and device, an electronic device, and a readable storage medium. BACKGROUND

[0002] With the development of natural language processing technology, intent recognition technology is increasingly applied to human-computer interaction systems in many different vertical fields such as intelligent customer service and intelligent assistants, and plays a great role in people's life and work.

[0003] Because people often use short sentences and omissions in communication, it is difficult to clearly explain the content of the dialogue in a single round of dialogue, so multi-round conversation technology has great commercial value. In the prior art, common intent recognition is mainly for single-round conversation. However, if the intent recognition method of single-round conversation is used to recognize the intent of the text in multi-round conversation, the following problems are likely to occur: the semantic coherence cannot be guaranteed; the extracted semantic features are single and not strong in generalization; the understanding of the context is not high, resulting in low reliability of intent recognition of the text in multi-round conversation. SUMMARY

[0004] Embodiments of the present application provide an intent recognition method, device, electronic device, and readable storage medium to solve the problem of low reliability of intent recognition of the text in multi-round conversation caused by using the intent recognition method of single-round conversation to recognize the intent of the text in multi-round conversation in the prior art.

[0005] To solve the above problems, the present application is implemented as follows:

[0006] In a first aspect, embodiments of the present application provide an intent recognition method, comprising:

[0007] obtaining a first text of a multi-round conversation;

[0008] obtaining a first intent corresponding to the first text recognized by a trained first model;

[0009] In the case where the first text contains a preset first keyword, performing semantic similarity calculation on the first keyword and the first intent to obtain a first similarity value;

[0010] According to the comparison result between the first similarity value and a first threshold value, determining a target intent corresponding to the first text.

[0011] In a second aspect, embodiments of the present application also provide an intent recognition device, comprising:

[0012] a first obtaining module configured to obtain a first text of a multi-round conversation;

[0013] a second obtaining module, configured to obtain a first intent corresponding to the first text identified by the trained first model;

[0014] a calculating module, configured to, when the first text contains a preset first keyword, perform semantic similarity calculation on the first keyword and the first intent to obtain a first similarity value;

[0015] a first determining module, configured to determine a target intent corresponding to the first text according to a comparison result between the first similarity value and a first threshold.

[0016] In a third aspect, an embodiment of the present application further provides an electronic device, comprising: a transceiver, a memory, a processor, and a program stored in the memory and capable of running on the processor; and the processor is configured to read the program in the memory to implement the steps in the method in the first aspect.

[0017] In a fourth aspect, an embodiment of the present application further provides a readable storage medium for storing a program, and the program is executed by a processor to implement the steps in the method in the first aspect.

[0018] In the embodiment of the present application, after the first intent corresponding to the first text is identified by the first model, semantic similarity calculation is performed on the keyword contained in the first text and the first intent to obtain a first similarity value, so as to determine whether the first intent is directly obtained based on the first text or is obtained by combining the context information of the multi-round dialogue in addition to the first text based on the comparison result between the first similarity value and a first threshold, and then determine the target intent corresponding to the first text according to the determination result. In this way, the target intent corresponding to the first text can be combined with the context information of the multi-round dialogue, so as to improve the accuracy of intent recognition of the multi-round dialogue text, and further improve the reliability of intent recognition of the multi-round dialogue text. BRIEF DESCRIPTION OF DRAWINGS

[0019] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the description of the embodiments of the present application. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0020] Figure 1 is one of the flowcharts of the intent recognition method provided by the embodiments of the present application;

[0021] Figure 2 is a training schematic diagram of the first model provided by the embodiments of the present application;

[0022] Figure 3 This is a second schematic flowchart of the intent recognition method provided in the embodiments of this application;

[0023] Figure 4 This is the third flowchart illustrating the intent recognition method provided in the embodiments of this application;

[0024] Figure 5 This is a schematic diagram of the structure of the first model provided in the embodiments of this application;

[0025] Figure 6 This is the fourth flowchart illustrating the intent recognition method provided in the embodiments of this application;

[0026] Figure 7 This is a schematic diagram of the intent recognition device provided in this application.

[0027] Figure 8 This is a schematic diagram of the structure of the electronic device provided in this application. Detailed Implementation

[0028] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0029] The terms "first," "second," etc., used in the embodiments of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to these processes, methods, products, or devices. Additionally, the use of "and / or" in this application indicates at least one of the connected objects, such as A and / or B and / or C, representing seven possibilities: including A alone, B alone, C alone, and the presence of both A and B, both B and C, both A and C, and the presence of A, B, and C.

[0030] In this embodiment, the terminal can engage in multi-turn dialogues with the user. In practical applications, the user can communicate with the terminal by inputting text or by inputting voice, depending on the actual situation. This embodiment does not limit this.

[0031] The terminal can communicate with electronic devices. After acquiring the user's current dialogue data (which can be text or voice), the terminal can send the dialogue data or the corresponding text data to the electronic device. The electronic device can then identify the target intent corresponding to the dialogue data and determine the response content based on that target intent. After determining the response content, the electronic device sends the response content to the terminal, allowing the terminal to output the response content in the form of voice or text.

[0032] In practical applications, terminals can be mobile phones, computers, robots, etc.; electronic devices can be cloud servers, etc.

[0033] The intent recognition method provided in the embodiments of this application will be described below. The intent recognition method in the embodiments of this application can be executed by the aforementioned electronic device.

[0034] See Figure 1 The intent recognition method in this application embodiment may include the following steps:

[0035] Step 101: Obtain the first text in the multi-turn conversation.

[0036] In a specific implementation, the first text can be the text corresponding to the user's dialogue data in the i-th round of a multi-turn conversation, where i is a positive integer. Specifically, if the user's dialogue data is speech, the text corresponding to the dialogue data is the text converted from the user's input speech; if the user's dialogue data is text, the text corresponding to the dialogue data is the text input by the user.

[0037] Step 102: Obtain the first intent corresponding to the first text, which is identified by the trained first model.

[0038] The first model can be any model that can be used to identify textual intent, such as: Convolutional Neural Network (CNN), Long Short-Term Memory (LSTM), Bidirectional Long Short-Term Memory Network combined with attention mechanism (Bilstm+attention), Bidirectional Attention Neural Network (BERT), etc.

[0039] In real-world scenarios, users exhibit a wide variety of intents, but the number of samples with the same intent is limited, meaning there is a lack of labeled data. To address the small sample problem, the first model can optionally be an inductive network based on few-sample learning. In this way, even with limited labeled data, the first model can accurately and quickly identify user intents, making it suitable for situations where specialized data is scarce in vertical domains.

[0040] Step 103: If the first text contains a preset first keyword, perform semantic similarity calculation on the first keyword and the first intent to obtain a first similarity value.

[0041] In this application, the electronic device may have P preset keywords, which are used to detect whether the text intent identified by the first model incorporates contextual information from a multi-turn conversation. When P is greater than 1, the P keywords may correspond to at least one level. When the P keywords correspond to at least two levels, the at least two levels may have a subordinate relationship, such as: the P keywords correspond to a first level and a second level, and a keyword at the first level may include at least one keyword at the second level. It is understood that the first keyword can be any one of the P keywords.

[0042] After obtaining the first intent predicted by the first model corresponding to the first text, the electronic device can detect whether the first text includes a keyword from the P keywords, and determine the next execution step based on the detection result of this detection action. In implementation, if the first text includes a keyword from the P keywords, step 103 can be executed; if the first text does not include a keyword from the P keywords, the first intent can be directly determined as the target intent corresponding to the first text.

[0043] Optionally, the electronic device can further detect whether the first text is the first text in the multi-turn dialogue, and combine the detection result of whether the first text is the first text in the multi-turn dialogue, and the detection result of whether the first text includes a keyword from the P keywords, to determine the next execution step. In specific implementation, the above two detection actions can be executed simultaneously or sequentially, and this application embodiment does not limit this.

[0044] Detection Result 1: The first text contains a preset first keyword, and the first text is not the first text in the multi-turn dialogue.

[0045] In this case, since the first intent predicted by the first model corresponding to the first text does not incorporate the contextual information of the multi-turn dialogue, the electronic device can perform semantic similarity calculation on the first keyword and the first intent to obtain a first similarity value, so as to determine whether the first intent incorporates the contextual information of the multi-turn dialogue through the first similarity value.

[0046] In specific implementation, electronic devices can use methods such as cosine distance to calculate the semantic similarity between the first keyword and the first intent. The embodiments of this application do not limit the calculation method of semantic similarity.

[0047] Detection result two: The first text does not contain any of the P keywords, or the first text is not the first text in the multi-turn dialogue.

[0048] In this case, the electronic device may optionally determine the first intent as the target intent corresponding to the first text.

[0049] If the first detection result indicates that the first text is the first text in the multi-turn dialogue, the multi-turn dialogue has not yet generated context information. The intent of the first text in the multi-turn dialogue is the current intent of the multi-turn conversation. Therefore, the electronic device can directly determine the first intent as the target intent corresponding to the first text.

[0050] If the first text does not contain any of the P keywords, the user is very likely to start the next topic. That is, the first text and the first i-1 texts in the multi-turn dialogue are not discussing the same topic. At this time, the newly started topic does not generate context information, and the intent of the first text is the current intent of the topic. Therefore, the electronic device can directly determine the first intent as the target intent corresponding to the first text.

[0051] As can be seen, the target intent determined by the above method is the user's true intent in expressing the first text, thereby improving the accuracy of intent recognition of the first text and thus improving the reliability of intent recognition of text in multi-turn conversations.

[0052] Step 104: Determine the target intent corresponding to the first text based on the comparison result between the first similarity value and the first threshold.

[0053] The comparison result between the first similarity value and the first threshold can be used to characterize whether the first intent matches the user's true intent.

[0054] Specifically, when the first similarity value is less than the first threshold, it indicates that the first intent matches the user's true intent. The reason is as follows: In this case, the semantic similarity between the first keyword and the first intent is small, indicating that the first intent, in addition to containing the semantic information of the first keyword, is very likely to contain the context information of the multi-turn conversation. Therefore, the electronic device can determine that the first intent matches the user's true intent.

[0055] If the first similarity value is greater than or equal to the first threshold, it indicates that the first intent does not match the user's true intent, for the following reasons: In this case, the semantic similarity between the first keyword and the first intent is high, indicating that the first intent, in addition to containing the semantic information of the first keyword, is very likely not to contain the context information of the multi-turn conversation. Therefore, the electronic device can determine that the first intent does not match the user's true intent.

[0056] Therefore, the electronic device can determine the target intent of the first text, i.e., the user's true intent after considering contextual information, based on the comparison result between the first similarity value and the first threshold. It is understood that the target intent may or may not be the first intent.

[0057] For ease of understanding, the following explanation is provided with reference to the example in Table 1:

[0058] The multi-turn dialogue in Table 1 includes four texts. The order of these four texts from top to bottom corresponds to the order in which they were acquired. The single-turn user intent can be understood as the aforementioned first intent, while the multi-turn dialogue intent, combined with the context, can be understood as the aforementioned target intent.

[0059] Assume that the default keywords for electronic devices include: general version, professional version, and mobile banking.

[0060] Table 1: Multi-round conversations for purchasing wealth management products

[0061]

[0062] As shown in Table 1, for the user's first question, the user intent in a single round is the target intent.

[0063] Regarding the user's second question, the first model predicted the user's single-round intent as "general version," which is the same as the keyword "general version" included in the question, showing a very high degree of similarity. The single-round user intent predicted by the first model is different from the user's true intent.

[0064] Regarding the user's third question, the first model predicted a single-round user intent of "how to buy wealth management products through mobile banking," which has little similarity to the keyword "mobile banking" in the question. This is because the single-round user intent predicted by the first model includes not only the intent of "mobile banking" but also the intent of "buying wealth management products," making the intent more complete and reflecting the user's true intent in asking the question.

[0065] The intent recognition method in this embodiment, after recognizing a first intent corresponding to a first text through a first model, calculates semantic similarity between the keywords contained in the first text and the first intent to obtain a first similarity value. Based on the comparison result of the first similarity value and a first threshold, it determines whether the first intent is obtained directly from the first text or whether it is obtained by combining contextual information from multi-turn dialogues in addition to the first text. Based on this determination result, the target intent corresponding to the first text is determined. This allows the target intent corresponding to the first text to incorporate contextual information from multi-turn dialogues, thereby improving the accuracy and reliability of intent recognition in multi-turn conversations.

[0066] The specific implementation of step 204 is explained below:

[0067] Implementation Method 1

[0068] Optionally, determining the target intent corresponding to the first text based on the comparison result between the first similarity value and the first threshold includes:

[0069] If the first similarity value is less than the first threshold, the first intent is determined to be the target intent corresponding to the first text.

[0070] In this embodiment, the first similarity value is less than the first threshold, indicating that the first intent matches the user's true intent. Therefore, the electronic device can directly determine the first intent as the target intent.

[0071] For example, for the third question in Table 1, since the single-round user intent predicted by the first model is "how to buy wealth management products in mobile banking", which has little similarity to the keyword "mobile banking" contained in the question, the electronic device can directly determine the single-round user intent as a multi-round dialogue intent after combining the context.

[0072] As can be seen, by implementing the first method, the target intent corresponding to the first text can be combined with the contextual information of the multi-turn dialogue, thereby improving the accuracy of intent recognition of text in multi-turn conversations, and further improving the reliability of intent recognition of text in multi-turn conversations.

[0073] Implementation Method 2

[0074] Optionally, determining the target intent corresponding to the first text based on the comparison result between the first similarity value and the first threshold includes:

[0075] If the first similarity value is greater than or equal to the first threshold, it is detected whether the second text of the multi-turn conversation contains a preset second keyword. The second text is acquired before the first text is acquired, and the second keyword and the first keyword are preset keywords of the same level.

[0076] If the second text contains the second keyword, control the first keyword to replace the second keyword in the second text to obtain the first target text;

[0077] Obtain the second intent corresponding to the first target text identified by the first model;

[0078] The second intent is determined to be the target intent corresponding to the first text.

[0079] In the second implementation, if the first similarity value is greater than or equal to the first threshold, it indicates that the first intent does not match the user's true intent. Therefore, the electronic device can abandon identifying the first intent as the target intent and re-determine the target intent.

[0080] In this embodiment, the electronic device can combine the first text and the second text of the multi-turn conversation to determine the target intent corresponding to the first text. The second text can be regarded as historical text in the multi-turn conversation relative to the first text. Optionally, the second text can be the text corresponding to the user's dialogue data in the (i-1)th round of the multi-turn conversation, but it is not limited to this.

[0081] In practice, the electronic device can detect whether the second text contains a preset second keyword. It can be understood that the second keyword can be any keyword of the same level as the first keyword included in the P keywords.

[0082] When the second text contains the second keyword, the electronic device can replace the second keyword in the second text with the first keyword to obtain the first target text. It is understood that the first target text includes the first keyword, as well as other information from the second text besides the second keyword. Then, the intent of the first target text is identified using the first model, and this intent is determined as the target intent of the first text. This allows the target intent corresponding to the first text to incorporate contextual information from multi-turn dialogues, thereby improving the accuracy of intent recognition in multi-turn conversations and consequently enhancing the reliability of intent recognition in multi-turn conversations.

[0083] For ease of understanding, the fourth question in Table 1 is used as an example for illustration below:

[0084] The similarity value between the intent "professional version" of the fourth question identified by the first model and the first keyword "professional version" included in the question is greater than the first threshold. The electronic device can detect that the third question includes the second keyword "mobile banking". Therefore, the second keyword "mobile banking" in the third question can be replaced with the first keyword "professional version" to obtain the first target text "how to purchase wealth management products using the professional version". The intent of the first target text is identified by the first model as "how to purchase wealth management products using the professional version", which is consistent with the user's true intent.

[0085] Furthermore, after detecting whether the first text input by the electronic device contains the second keyword, the method further includes:

[0086] If the second text does not contain the second keyword, the first text and the second text are concatenated to obtain the second target text;

[0087] Obtain the third intent corresponding to the second target text identified by the first model;

[0088] The third intent is determined to be the target intent corresponding to the first text.

[0089] It is understood that the second target text includes both the first text and the second text.

[0090] To facilitate understanding, the second question in Table 1 is used as an example for illustration below:

[0091] The similarity value between the intent of the second question identified by the first model, "general version," and the first keyword "general version" included in the question, is greater than the first threshold. The electronic device can detect that the first question does not contain the keyword. Therefore, the first question and the second question can be concatenated to obtain the second target text, "How to purchase financial products using the general version." The intent of the first text is identified by the first model as "How to purchase financial products using the general version," which matches the user's true intent.

[0092] As can be seen, by implementing the second method, the target intent corresponding to the first text can be combined with the contextual information of the multi-turn dialogue, thereby improving the accuracy of intent recognition of text in multi-turn conversations, and further improving the reliability of intent recognition of text in multi-turn conversations.

[0093] The principle of the first model in this application for recognizing text intent is explained below:

[0094] Optionally, the first model includes an encoding layer, an induction module, a relation module, and an intent classification result output module;

[0095] The encoding layer is used to: obtain the feature vector corresponding to the first text;

[0096] The induction module is connected to the encoding layer and is used to: obtain the feature vector from the encoding layer, determine at least two class vectors based on the feature vector, and obtain the target class vector based on the at least two class vectors;

[0097] The relation module is connected to the encoding layer and the induction module respectively, and is used to: determine the first intent corresponding to the first text based on the feature vector obtained from the encoding layer and the target class vector obtained from the encoding layer;

[0098] The intent classification result output module is connected to the relationship module and is used to obtain the first intent from the relationship module and output the first intent.

[0099] In this optional embodiment, the first model is an inductive model. It should be noted that the training method of the inductive model can be the same as in related technologies. Optionally, the idea of ​​meta-learning can be adopted to construct multiple episodes, each episode being considered a task. During each training process, for each episode, C categories are randomly selected from the training data, K samples are randomly selected from the C categories as the support set, and Q samples are randomly collected from the remaining samples as the query set. Each time the model trains on the support set, the loss is calculated on the query set. For each episode in the training phase, the parameters are updated according to the magnitude of the loss using the backpropagation algorithm and the Adamw optimization algorithm, allowing the model to learn and thus complete the training of the entire network. The network with the best performance is then saved.

[0100] The main difference between the inductive model in this application and existing inductive models lies in the fact that the inductive module of this application innovatively proposes a category feature fusion processing method. Specifically, the inductive module can obtain W class vectors through different methods, and by fusing these W class vectors, a target class vector for determining the text intent is obtained, where W is an integer greater than 1. This avoids noise interference from different expressions of the same intent while considering the semantic information of different expressions, thereby obtaining a class vector representation containing more useful semantic information, improving the accuracy of intent recognition, and obtaining more accurate single-round user intent.

[0101] Optionally, the at least two class vectors may include, but are not limited to, a first class vector and a second class vector;

[0102] The first type of vector is obtained by performing a nonlinear transformation on the feature vector and then performing a weighted summation on the nonlinearly transformed feature vector;

[0103] The second type of vector is obtained by averaging the feature vectors.

[0104] The specific explanation is as follows:

[0105] For the first type of quantity: First, using formula (1), the sample representation matrix of the j-th support set of the i-th category, i.e., the feature vector, is obtained. Using the squash activation function for non-linear transformation:

[0106]

[0107] Where s represents the support set; w s b represents the weight values ​​of the eigenvector; s Indicates bias.

[0108] Then, the transformed feature vectors are weighted and summed using formula (2) to obtain the first type of vector.

[0109]

[0110] Where, d i =softmax(b i ), b i Let be the logarithm of the coupling coefficient of the i-th category.

[0111] For the second type vector: the second type vector is obtained by averaging the k samples in each class using formula (3).

[0112]

[0113] Where, N i Let be the total number of samples in category i.

[0114] After obtaining the first class vector and the second class vector, the induction module can fuse the first class vector and the second class vector to obtain the target class vector. Optionally, the category-level features can be calculated using formula (4).

[0115]

[0116] Where ω represents the category feature, i.e., the weight value of the class vector, and ω1 + ω2 = 1. In practical applications, a larger weight can be assigned to class vectors that can better represent rich semantic information, and a smaller weight can be assigned to those that cannot. The specific weight can be determined according to the actual situation, and this application does not limit this.

[0117] The fused class vector can be compressed using formula (5) and then further compressed using the squash activation function to nonlinearly map it to the interval [0,1], thus obtaining the target class vector c. i :

[0118]

[0119] Understandably, in practical applications, the dynamic routing value b can be determined using formula (6). ij The weights w are updated using formulas (6) and (1) to obtain the optimal class vector through multiple iterations:

[0120]

[0121] The implementation principles of the other modules of the first model can be the same as those in related technologies, or they can be implemented in the following ways:

[0122] In practice, before the electronic device uses the first model to identify the first intent corresponding to the first text, it can process the first text first, such as preprocessing operations like Chinese word segmentation, removal of special symbols, and removal of stop words. Then, the processed text is input into the encoding layer of the first model.

[0123] For the encoding layer, it can include a bidirectional gate recurrent unit (Bigru) + attention network. The encoding layer first quantifies the processed text using embedding to obtain a numerical vector; then, the bidirectional GRU (Bigru) + attention network extracts high-level features from the numerical vector, i.e., the feature vector. Specifically, in the Bigru + Attention network, the Bigru network is used to obtain the state at time t, the states from both directions are concatenated to obtain the hidden state of the Bigru network, and the outputs of the hidden states at each time step are passed through the attention layer to obtain the deep semantic features of the text, i.e., the feature vector.

[0124] In the relational module, a neural tensor layer can be used to simulate the class vector c. i The semantic feature vector of a sample in the query set, i.e., the feature ring e q The relationship between the query set (where q represents the query set) is given by the relation vector output by the tensor layer: M k ∈R 2u×2u ,k∈[1,...,h], and then the final relation score is obtained through the sigmoid function to determine the first intention.

[0125] It should be noted that the various optional implementation methods described in the embodiments of this application can be combined with each other or implemented individually without conflict, and the embodiments of this application do not limit this.

[0126] In this application embodiment, for multi-turn dialogues, an inductive network based on few-shot learning is proposed to identify the intent of the current statement. The inductive network's induction module innovatively introduces a category feature fusion method to fuse the class vectors obtained by the dynamic routing algorithm with those obtained using the averaging method, improving the accuracy of intent recognition for the current statement. Simultaneously, combining preset scene keywords of different levels, a semantic matching-based intent inheritance method is proposed to obtain user intent incorporating dialogue history information. This proposal effectively improves the accuracy and speed of intent recognition in multi-turn dialogues, addressing the lack of specialized data in vertical domains, thus facilitating smooth human-computer dialogue and providing a better user experience.

[0127] The following is combined with Figure 2 as well as Figure 3 The embodiments of this application will be described in detail below.

[0128] Figure 2 The process of training an inductive network is demonstrated, which mainly involves corpus acquisition, data preprocessing, model building, model training, and storage.

[0129] Figure 3 This demonstrates the intent recognition process for multi-turn dialogues using a trained network model in a real-world application. Its main components include pre-set keywords, collection of dialogue data, data preprocessing, AI network recognition, semantic matching of keywords and intents, and user intent recognition results.

[0130] The specific explanation is as follows:

[0131] 1. Data acquisition.

[0132] The corpus used to train the model can be publicly available data within the vertical domain, while in the model testing phase, the human-computer dialogue content in real-world scenarios is mainly obtained through terminals such as computers, mobile phones, and robot display screens.

[0133] 2. Keyword construction.

[0134] Based on different scenarios within this vertical field, different levels of keywords are preset, such as primary keywords and secondary keywords. Among them, primary keywords may have zero or more secondary keywords preset.

[0135] 3. Data preprocessing.

[0136] For publicly available data, text processing requires Chinese word segmentation and stop word removal. For multi-turn dialogues acquired from real-world scenarios, when some dialogue content is missing, the entire multi-turn dialogue sample with missing content needs to be deleted to avoid affecting the user intent recognition results. Furthermore, it is necessary to extract the user's dialogue text from each turn, label their intent, and then perform Chinese word segmentation, special symbol removal, and stop word removal on the dialogue text.

[0137] 4. Algorithm modeling.

[0138] Because real-world user intents are diverse, but samples of the same intent are few (i.e., labeled data is limited), we employ an inductive network based on few-shot learning to address the small-sample problem. We embed the preprocessed public text data to quantify the text, then extract high-level features of the samples using a Bigru+Attention network. These features are then input into the inductive module of the network. Within this module, we innovatively propose a class feature fusion method. This avoids noise interference from different expressions of the same intent while considering the semantic information of different expressions, resulting in class vector representations containing more useful semantic information. This improves the accuracy of intent recognition and yields more accurate single-turn user intents.

[0139] 5. Multi-turn dialogue intent recognition in real-world scenarios.

[0140] A support set is constructed based on user data from real-world scenarios to facilitate subsequent intent recognition. Addressing the issue of user input in real-world scenarios, we use a pre-trained model from the algorithm modeling section to obtain the current user's intent. Since the current user's intent is not combined with contextual information, it is necessary to combine it with preset scenario keywords. Cosine distance is used to semantically match the keywords contained in the current sentence with the current user's intent. By setting a threshold, we determine whether it is necessary to replace or inherit keywords in the current sentence based on the preceding text. Finally, the intent combined with the context is the target intent.

[0141] like Figure 4 As shown, the intent recognition method in this application embodiment may include the following steps:

[0142] Step S1: Obtain text corpus.

[0143] Step S2: Data preprocessing.

[0144] Specifically, the corpus undergoes preprocessing such as Chinese word segmentation, removal of special symbols, and removal of stop words.

[0145] Step S3: Network model construction.

[0146] Step S4: Training and saving the model.

[0147] Step S5: Set scene keywords.

[0148] Step S6: User text data collection and processing in real-world scenarios.

[0149] Step S7: Obtain the user intent of a single-turn dialogue that is not combined with the dialogue history.

[0150] Step S8: Combine scenario keywords to obtain the user's target intent.

[0151] Specifically, based on whether the current user input contains scene keywords, semantic matching is used to obtain the target intent combined with dialogue history information.

[0152] The specific explanation is as follows:

[0153] In step S1, a public dataset of intent recognition related to the vertical domain can be obtained for subsequent data preprocessing.

[0154] In step S2, to improve the final recognition results, preprocessing operations such as Chinese word segmentation, removal of special symbols, and removal of stop words can be performed on the public dataset.

[0155] In step S3, considering that in real-world scenarios, users have many different intentions in each round of dialogue, but few samples with the same intention (i.e., few labeled data), we use an inductive network to model the preprocessed text from step S2 to address the small sample size problem. The model structure is as follows: Figure 5 As shown. In the encoding layer, we quantify the processed text using embedding, then extract high-level features of the samples using the Bigru+Attention network, and use these features as input to the induction module. The induction module obtains class vectors, and the relation module calculates the relationship score between the feature vector and class vector of each sample in the support set, thereby achieving the classification effect.

[0156] In the encoding layer, for any numerical vector of text, we use a bidirectional GRU (Bigru) network to obtain the state at time t. We concatenate the states in the two directions to obtain the hidden state of the Bigru network. The outputs of the hidden states at each time step are passed through the attention layer to obtain the deep semantic features of the text.

[0157] In the induction module, this application innovatively designs a weighted fusion method, which avoids noise interference from different expressions of the same intent, while taking into account the semantic information of different expressions, so as to obtain a class vector representation containing more useful semantic information and improve the accuracy of intent recognition. Specific details are as follows:

[0158] Firstly, according to The representation matrix of the j-th support set samples of the i-th category A non-linear transformation is performed using the squash activation function (where 's' represents the support set); then, the transformed sample representations are weighted and summed to obtain the class vectors. Where d i =softmax(b i ), here b i Let be the logarithm of the coupling coefficient for the i-th class. To improve model performance, we take the average of k samples from each class to represent the class vector.

[0159]

[0160] In the formula N i Let be the total number of samples in category i. Label the fused category-level features as... Then we have: Where ω represents the weight value of the category feature, and ω1 + ω2 = 1. A larger weight is assigned to class vectors that can better represent rich semantic information, and a smaller weight is assigned to those that cannot.

[0161] The merged class vector is then compressed using the squash activation function, which non-linearly maps it to the interval [0,1], resulting in a new class vector c. i :

[0162]

[0163] Finally, the formula is used. Update dynamic route value b ij And the weight w, through multiple iterations, to obtain the optimal class vector.

[0164] In the relation module, a neural tensor layer was used to simulate the class vector c. i The semantic feature vector e of a sample in the query set q The relationship between the query set (where q represents the query set) is given by the relation vector output by the tensor layer: M k ∈R 2u×2u k∈[1,...,h], and then the final relation score is obtained through the sigmoid function.

[0165] In step S4, the constructed network model is trained and saved. Employing the idea of ​​meta-learning, multiple episodes are constructed, each of which can be considered a task. During each training iteration, for each episode, C classes are randomly selected from the training data, K samples from each of the C classes are randomly selected as the support set, and Q samples are randomly selected from the remaining samples as the query set. Each time the model trains on the support set, the loss is calculated on the query set. For each episode during the training phase, the parameters are updated based on the magnitude of the loss using the backpropagation algorithm and the Adamw optimization algorithm, allowing the model to learn and thus complete the training of the entire network. Finally, the network with the best performance is saved.

[0166] In step S5, in order to make the final identified intent more consistent with the user's true intent in a specific scenario by combining historical information, we manually construct keywords at different levels based on the business processes of different scenarios. For example, in a certain scenario, we set first-level keywords and second-level keywords, with zero or more second-level keywords under the first-level keywords.

[0167] In step S6, we will obtain the user's dialogue content in the actual application scenario. We need to delete the entire multi-turn dialogue sample with missing dialogue content. For the multiple dialogue samples obtained, we need to extract the user's dialogue text in each round of dialogue and label its intent.

[0168] In step S7, the acquired text is grouped by category to construct a support set, and the text entered by the user online is used as the query set. The query set and support set are cleaned and features are extracted according to steps S3 and S4. Then, through the trained model, the text intent of the user's current online input without combining the dialogue history is obtained.

[0169] In step S8, since the current user's intent is not combined with contextual information, it is necessary to combine preset scene keywords to determine the target intent in the multi-turn dialogue. Here, the target intent refers to the intent combined with the preceding historical information. For the first sentence input by the user, the user's intent is relatively clear and complete; the current intent is the target intent. When the user performs another dialogue operation, the recognition process is as follows: Figure 6 As shown, the specific intent recognition method is as follows:

[0170] Step 601: Obtain the first text input by the user.

[0171] Step 602: Determine the intent of the first text.

[0172] Step 603: Determine whether the first text contains keywords.

[0173] If not included, proceed to step 604; if included, proceed to step 605.

[0174] Step 604: Determine the intent of the first text as the target intent;

[0175] Step 605: Calculate the semantic similarity between the keywords contained in the first text and the intent of the first text to obtain the first similarity value.

[0176] Step 606: Determine whether the first similarity value is greater than or equal to the first threshold.

[0177] If not, it means that the current user's intent contains other information besides the semantic information of the keywords. For example, user: "How to buy wealth management products in the public version?" The intent of this sentence is "buy wealth management products in the public version", and the keyword is "public version". The similarity between the keyword "public version" and the intent "buy wealth management products in the public version" is less than the set threshold, that is, they are not similar. This is because in addition to the intent of the public version, there is also the intent of buying wealth management products. The intent is more complete. The current intent is the intent combined with the context, which is the target intent. Then, proceed to step 604.

[0178] If so, proceed to step 607.

[0179] Step 607: Determine whether the second text entered by the user contains keywords at the same level as the first text.

[0180] If yes, proceed to step 608; otherwise, proceed to step 609.

[0181] Step 608: Replace the keywords in the second text with the keywords in the first text to obtain the first target text, and determine the intent of the first target text as the target intent.

[0182] Step 609: Combine the first text and the second text to obtain the second target text, and determine the intent of the second target text as the target intent.

[0183] In the embodiments of this application, 1) the network's induction module innovatively proposes a class feature fusion method to fuse the class vectors obtained by the dynamic routing algorithm with those obtained by the averaging method. This avoids noise interference from different expressions of the same intent and takes into account the semantic information of different expressions, so as to obtain class features containing more useful semantic information and improve the accuracy of intent recognition. 2) In the intent recognition of multi-turn dialogues in real-world scenarios, keywords at different levels in a specific scenario are semantically matched with the dialogue intent. Based on the matching results, keywords in the previous sentence are replaced or concatenated to achieve semantic inheritance, thereby obtaining the target intent combined with historical information.

[0184] It is evident that, compared to existing intent recognition solutions, this application effectively improves the accuracy of intent recognition in multi-turn dialogues, especially in situations where specialized data in vertical fields is scarce. This facilitates smooth human-computer dialogue and provides a better user experience.

[0185] See Figure 7 , Figure 7 This is a structural diagram of the intent recognition device provided in the embodiments of this application. Figure 7 As shown, the intent recognition device 700 includes:

[0186] The first acquisition module 701 is used to acquire the first text of a multi-turn conversation;

[0187] The second acquisition module 702 is used to acquire the first intent corresponding to the first text, which is identified by the trained first model.

[0188] The calculation module 703 is used to calculate the semantic similarity between the first keyword and the first intent when the first text contains a preset first keyword, and obtain a first similarity value.

[0189] The first determining module 704 is used to determine the target intent corresponding to the first text based on the comparison result between the first similarity value and the first threshold.

[0190] Optionally, the first determining module 704 is configured to:

[0191] If the first similarity value is less than the first threshold, the first intent is determined to be the target intent corresponding to the first text.

[0192] Optionally, the first determining module 704 includes:

[0193] The detection unit is used to detect whether the second text of the multi-turn conversation contains a preset second keyword when the first similarity value is greater than or equal to the first threshold. The second text is acquired before the first text is acquired, and the second keyword and the first keyword are preset keywords of the same level.

[0194] The replacement unit is configured to control the first keyword to replace the second keyword in the second text when the second text contains the second keyword, thereby obtaining the first target text;

[0195] The first acquisition unit is used to acquire the second intent corresponding to the first target text identified by the first model;

[0196] The first determining unit is used to determine that the second intent is the target intent corresponding to the first text.

[0197] Optionally, the first determining module 704 further includes:

[0198] The splicing unit is used to splice the first text and the second text together to obtain the second target text when the second text does not contain the second keyword;

[0199] The second acquisition unit is used to acquire the third intent corresponding to the second target text identified by the first model;

[0200] The second determining unit is used to determine that the third intent is the target intent corresponding to the first text.

[0201] Optionally, the intent recognition device further includes:

[0202] The second determining module is used to determine the first intent as the target intent corresponding to the first text when the first text does not contain any of the P keywords, or when i equals 1.

[0203] Optionally, the first model includes an encoding layer, an induction module, a relation module, and an intent classification result output module;

[0204] The encoding layer is used to: obtain the feature vector corresponding to the first text;

[0205] The induction module is connected to the encoding layer and is used to: obtain the feature vector from the encoding layer, determine at least two class vectors based on the feature vector, and obtain the target class vector based on the at least two class vectors;

[0206] The relation module is connected to the encoding layer and the induction module respectively, and is used to: determine the first intent corresponding to the first text based on the feature vector obtained from the encoding layer and the target class vector obtained from the encoding layer;

[0207] The intent classification result output module is connected to the relationship module and is used to obtain the first intent from the relationship module and output the first intent.

[0208] Optionally, the at least two class vectors include a first class vector and a second class vector;

[0209] The first type of vector is obtained by performing a nonlinear transformation on the feature vector and then performing a weighted summation on the nonlinearly transformed feature vector;

[0210] The second type of vector is obtained by averaging the feature vectors.

[0211] The intent recognition device 700 can realize the embodiments of this application.Figure 2 The various processes in the method embodiments, and the ways to achieve the same beneficial effects, will not be repeated here to avoid repetition.

[0212] This application also provides an electronic device. Please refer to [link to relevant documentation]. Figure 8 The electronic device may include a processor 801, a memory 802, and a program 8021 stored in the memory 802 and executable on the processor 801. When the program 8021 is executed by the processor 801, it can achieve... Figure 2 Any steps in the corresponding method embodiments and the achievement of the same beneficial effects will not be repeated here.

[0213] Those skilled in the art will understand that all or part of the steps of the methods described in the above embodiments can be implemented by hardware related to program instructions, and the program can be stored in a readable medium. This application also provides a readable storage medium storing a computer program, which, when executed by a processor, can implement the above-described methods. Figure 3 Any step in the corresponding method embodiment can achieve the same technical effect, and will not be repeated here to avoid repetition.

[0214] The storage medium may be a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0215] The above description represents the preferred embodiments of this application. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles described in this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. An intent recognition method, characterized in that, The method includes: Retrieve the first text of a multi-turn conversation; Obtain the first intent corresponding to the first text, which is identified by the trained first model; If the first text contains a preset first keyword, the semantic similarity between the first keyword and the first intent is calculated to obtain a first similarity value; Based on the comparison result between the first similarity value and the first threshold, the target intent corresponding to the first text is determined; The step of determining the target intent corresponding to the first text based on the comparison result between the first similarity value and the first threshold includes: If the first similarity value is less than the first threshold, the first intent is determined to be the target intent corresponding to the first text; The step of determining the target intent corresponding to the first text based on the comparison result between the first similarity value and the first threshold includes: If the first similarity value is greater than or equal to the first threshold, it is detected whether the second text of the multi-turn conversation contains a preset second keyword. The second text is acquired before the first text is acquired, and the second keyword and the first keyword are preset keywords of the same level. If the second text contains the second keyword, control the first keyword to replace the second keyword in the second text to obtain the first target text; Obtain the second intent corresponding to the first target text identified by the first model; The second intent is determined to be the target intent corresponding to the first text.

2. The method according to claim 1, characterized in that, After detecting whether the second text of the multi-turn conversation contains a preset second keyword, the method further includes: If the second text does not contain the second keyword, the first text and the second text are concatenated to obtain the second target text; Obtain the third intent corresponding to the second target text identified by the first model; The third intent is determined to be the target intent corresponding to the first text.

3. The method according to claim 1, characterized in that, After obtaining the first intent corresponding to the first text identified by the first model, the method further includes: If the first text does not contain preset keywords, the first intent is determined as the target intent corresponding to the first text.

4. The method according to claim 1, characterized in that, The first model includes an encoding layer, an induction module, a relation module, and an intent classification result output module; The encoding layer is used to: obtain the feature vector corresponding to the first text; The induction module is connected to the encoding layer and is used to: obtain the feature vector from the encoding layer, determine at least two class vectors based on the feature vector, and obtain the target class vector based on the at least two class vectors; The relation module is connected to the encoding layer and the induction module respectively, and is used to: determine the first intent corresponding to the first text based on the feature vector obtained from the encoding layer and the target class vector obtained from the encoding layer; The intent classification result output module is connected to the relationship module and is used to obtain the first intent from the relationship module and output the first intent.

5. The method according to claim 4, characterized in that, The at least two class vectors include a first class vector and a second class vector; The first type of vector is obtained by performing a nonlinear transformation on the feature vector and then performing a weighted summation on the nonlinearly transformed feature vector; The second type of vector is obtained by averaging the feature vectors.

6. An intent recognition device, characterized in that, include: The first acquisition module is used to acquire the first text of a multi-turn conversation. The second acquisition module is used to acquire the first intent corresponding to the first text, which is identified by the trained first model. The calculation module is used to calculate the semantic similarity between the first keyword and the first intent when the first text contains a preset first keyword, and obtain a first similarity value. The first determining module is used to determine the target intent corresponding to the first text based on the comparison result between the first similarity value and the first threshold. The first determination is used for: If the first similarity value is less than the first threshold, the first intent is determined to be the target intent corresponding to the first text; The first determining module includes: The detection unit is used to detect whether the second text of the multi-turn conversation contains a preset second keyword when the first similarity value is greater than or equal to the first threshold. The second text is acquired before the first text is acquired, and the second keyword and the first keyword are preset keywords of the same level. The replacement unit is configured to control the first keyword to replace the second keyword in the second text when the second text contains the second keyword, thereby obtaining the first target text; The first acquisition unit is used to acquire the second intent corresponding to the first target text identified by the first model; The first determining unit is used to determine that the second intent is the target intent corresponding to the first text.

7. The intent recognition device according to claim 6, characterized in that, The first determining module further includes: The splicing unit is used to splice the first text and the second text together to obtain the second target text when the second text does not contain the second keyword; The second acquisition unit is used to acquire the third intent corresponding to the second target text identified by the first model; The second determining unit is used to determine that the third intent is the target intent corresponding to the first text.

8. The intent recognition device according to claim 6, characterized in that, The first model includes an encoding layer, an induction module, a relation module, and an intent classification result output module; The encoding layer is used to: obtain the feature vector corresponding to the first text; The induction module is connected to the encoding layer and is used to: obtain the feature vector from the encoding layer, determine at least two class vectors based on the feature vector, and obtain the target class vector based on the at least two class vectors; The relation module is connected to the encoding layer and the induction module respectively, and is used to: determine the first intent corresponding to the first text based on the feature vector obtained from the encoding layer and the target class vector obtained from the encoding layer; The intent classification result output module is connected to the relationship module and is used to obtain the first intent from the relationship module and output the first intent.

9. An electronic device, comprising: A transceiver, a memory, a processor, and a program stored in the memory and executable on the processor; characterized in that the processor is configured to read the program in the memory to implement the steps of the intent recognition method as described in any one of claims 1 to 5.

10. A readable storage medium for storing a program, characterized in that, When the program is executed by the processor, it implements the steps in the intent recognition method as described in any one of claims 1 to 5.

Images