Intention recognition method, electronic device, storage medium

Abstract

The application relates to the technical field of artificial intelligence, intelligent customer service and financial transaction, in particular to an intention recognition method, an electronic device and a storage medium. The intention recognition method of the application needs to acquire target information to be recognized and a pre-trained intention recognition model, the intention recognition model comprises an attention encoding layer, an evidence neural network and an output layer, the target information is analyzed and processed based on the attention encoding layer to obtain global semantic representation and mean semantic representation, the global semantic representation and the mean semantic representation are respectively predicted based on the evidence neural network to obtain a global classification probability corresponding to the global semantic representation and a mean classification probability corresponding to the mean semantic representation, the global classification probability and the mean classification probability are integrated based on the output layer to obtain an intention recognition result, the intention recognition result is used for representing a dialogue intention category corresponding to the target information, the calibration effect of uncertainty calibration can be improved, and the accuracy of intention recognition is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, intelligent customer service and financial transaction, in particular to an intent recognition method, an electronic device and a storage medium. BACKGROUND

[0002] Intent recognition refers to analyzing core needs of a user according to input information of the user and outputting semantic content most relevant to the input information. The accuracy of intent recognition greatly affects the accuracy of search and the intelligence of a dialogue system. Ideally, the output of a model should not only be accurate but also have good calibration effect. In business scenarios such as intelligent customer service and financial transaction, a dialogue robot or an intelligent customer service assistant is often used to achieve intent recognition based on a dialogue between a user and a machine, so as to make a response action matching the user's intent.

[0003] Uncertainty calibration of an artificial intelligence model refers to correcting the accuracy of prediction uncertainty of the artificial intelligence model. It should be pointed out that when intent recognition is performed on input information of a user, a response action should not be made if the model obtains a low-confidence prediction result, for example, the low-confidence prediction result is fed back to an expert for decision-making. Therefore, in order to avoid that the artificial intelligence model gives an unreasonable response action for a possible incorrect recognition result, uncertainty calibration needs to be performed on the artificial intelligence model in related technologies. However, how to improve the calibration effect of uncertainty calibration to improve the accuracy of intent recognition has become a major problem to be solved in the industry. SUMMARY

[0004] The present application aims to at least solve one of the technical problems existing in the prior art. To this end, the present application provides an intent recognition method, an electronic device and a storage medium, which can improve the calibration effect of uncertainty calibration and thus improve the accuracy of intent recognition.

[0005] The intent recognition method according to the first aspect of the present application comprises:

[0006] obtaining target information to be recognized and a pre-trained intent recognition model, the intent recognition model comprising an attention encoding layer, an evidence neural network and an output layer;

[0007] performing analysis processing on the target information based on the attention encoding layer to obtain a global semantic representation and a mean semantic representation;

[0008] performing prediction processing on the global semantic representation and the mean semantic representation based on the evidence neural network to obtain a global classification probability corresponding to the global semantic representation and a mean classification probability corresponding to the mean semantic representation;

[0009] Integrate the global classification probability and the mean classification probability based on the output layer to obtain an intention recognition result, where the intention recognition result is used to represent a dialog intention category corresponding to the target information.

[0010] According to some embodiments of the present application, before the target information to be recognized and the pre-trained intention recognition model are obtained, the intention recognition model is pre-trained, specifically including:

[0011] Select a training text from a preset training data set;

[0012] Extract semantic features of the training text based on the attention encoding layer to obtain a sentence semantic feature vector and a word semantic mean vector;

[0013] Respectively predict and train the sentence semantic feature vector and the word semantic mean vector based on the evidence neural network to obtain a sentence semantic classification probability corresponding to the sentence semantic feature vector and a word semantic classification probability corresponding to the word semantic mean vector;

[0014] Analyze classification bias based on the sentence semantic classification probability and the word semantic classification probability to obtain training bias data;

[0015] Iteratively update the intention recognition model based on the training bias data until the training bias data meets a preset error condition to obtain the pre-trained intention recognition model.

[0016] According to some embodiments of the present application, the semantic feature extraction of the training text based on the attention encoding layer to obtain a sentence semantic feature vector and a word semantic mean vector includes:

[0017] Perform word segmentation processing on the training text to obtain a training word sequence, where the training word sequence includes a plurality of word segmentation elements;

[0018] Extract semantic features of each word segmentation element based on the attention mechanism of the attention encoding layer to obtain a sentence semantic feature vector and a plurality of word semantic feature vectors;

[0019] Perform mean analysis based on a plurality of word semantic feature vectors to obtain the word semantic mean vector.

[0020] According to some embodiments of the present application, the training word sequence includes a start sequence bit and an element sequence bit, the start sequence bit is configured with a classification identifier, and the element sequence bit is configured with each word segmentation element. The semantic feature extraction of each word segmentation element based on the attention mechanism of the attention encoding layer to obtain a sentence semantic feature vector and a plurality of word semantic feature vectors includes:

[0021] perform global semantic extraction on all the word segmentation elements based on a self-attention mechanism of the attention encoding layer, and obtain the sentence meaning feature vector based on the classification identifier;

[0022] perform local semantic extraction on each of the word segmentation elements based on the attention encoding layer, and obtain a plurality of word meaning feature vectors based on each of the word segmentation elements.

[0023] According to some embodiments of the present application, the classification bias analysis based on the sentence meaning classification probability and the word meaning classification probability obtains training bias data, including:

[0024] Based on the sentence meaning classification probability and the word meaning classification probability, a classification loss function is constructed.

[0025] The output value of the classification loss function is determined as the training bias data.

[0026] According to some embodiments of the present application, the classification loss function is constructed based on the sentence meaning classification probability and the word meaning classification probability, including:

[0027] An evidence mapping analytical expression corresponding to the evidence neural network is obtained.

[0028] Based on the word meaning classification probability, the sentence meaning classification probability, and the evidence mapping analytical expression, the classification loss function is constructed.

[0029] According to some embodiments of the present application, the classification loss function is constructed based on the word meaning classification probability, the sentence meaning classification probability, and the evidence mapping analytical expression, including:

[0030] Based on the sentence meaning classification probability and the evidence mapping analytical expression, a first cross-entropy sub-function is constructed.

[0031] Based on the word meaning classification probability and the evidence mapping analytical expression, a second cross-entropy sub-function is constructed.

[0032] The first cross-entropy sub-function and the second cross-entropy sub-function are integrated to obtain the classification loss function.

[0033] According to some embodiments of the present application, the first cross-entropy sub-function and the second cross-entropy sub-function are integrated to obtain the classification loss function, including:

[0034] Based on the word meaning classification probability and the sentence meaning classification probability, a relative entropy analysis is performed to obtain semantic information divergence.

[0035] The first cross-entropy sub-function, the second cross-entropy sub-function, and the semantic information divergence are integrated to obtain the classification loss function.

[0036] In a second aspect, an electronic device is provided, including a memory and a processor. The memory stores a computer program. The processor executes the computer program to implement the intent recognition method according to any one of the embodiments of the first aspect of the present application.

[0037] In a third aspect, a computer readable storage medium is provided, which stores a computer program. The computer program is executed by a processor to implement the intent recognition method according to any one of the embodiments of the first aspect of the present application.

[0038] According to the intent recognition method, the electronic device and the storage medium provided by the embodiments of the present application, at least the following advantages are achieved

[0039] Advantages:

[0040] The intent recognition method provided by the present application needs to obtain target information to be recognized and a pre-trained intent recognition model. The intent recognition model includes an attention encoding layer, an evidence neural network and an output layer. The target information is analyzed and processed based on the attention encoding layer to obtain global semantic representation and mean semantic representation. The global semantic representation and the mean semantic representation are respectively predicted based on the evidence neural network to obtain a global classification probability corresponding to the global semantic representation and a mean classification probability corresponding to the mean semantic representation. The global classification probability and the mean classification probability are integrated based on the output layer to obtain an intent recognition result. The intent recognition result is used to represent a dialog intent category corresponding to the target information. After the intent recognition model obtains the target information to be recognized, the three layers of the attention encoding layer, the evidence neural network and the output layer are processed, which can improve the calibration effect of uncertainty calibration and further improve the accuracy of intent recognition. In business scenarios such as intelligent customer service and financial transactions, dialog robots and intelligent customer service assistants are often used to implement intent recognition based on the dialog between users and machines, so as to make a response action matching the user's intent. The application of the intent recognition method provided by the present application is helpful to achieve better human-computer interaction effect.

[0041] Additional aspects and advantages of the present application will be in part apparent and in part pointed out hereinafter. BRIEF DESCRIPTION OF DRAWINGS

[0042] The above and / or additional aspects and advantages of the present application will become apparent and be readily appreciated from the following description, including the appended drawings, wherein:

[0043] Figure 1 An optional flowchart of the intent recognition method according to the embodiments of the present application;

[0044] Figure 2An optional flowchart of step S202 in FIG. 2;

[0045] Figure 3 An optional flowchart of step S302 in FIG. 3; Figure 2

[0046] Figure 4 An optional flowchart of step S302 in FIG. 3; Figure 3

[0047] Figure 5 An optional flowchart of step S204 in FIG. 2; Figure 2

[0048] Figure 6 An optional flowchart of step S501 in FIG. 5; Figure 5

[0049] Figure 7 An optional flowchart of step S602 in FIG. 6; Figure 6

[0050] Figure 8 An optional flowchart of step S703 in FIG. 7; Figure 7

[0051] FIG. 8 is a hardware structure schematic diagram of an electronic device provided by an embodiment of the present application. Figure 9 DETAILED DESCRIPTION

[0052] The embodiments of the present application are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar notations represent the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the drawings are exemplary only, and are used only for explaining the present application, and cannot be understood as limiting the present application.

[0053] In the description of the present application, the meaning of several is one or more, the meaning of multiple is two or more, greater than, less than, more than, etc. are understood as not including the number, above, below, etc. are understood as including the number. If it is described that the first, the second is only used for distinguishing the purpose of the technical features, and cannot be understood as indicating or implying the relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features.

[0054] ​​​​​​In the description of the specification, the description of the terms "one embodiment", "some embodiments", "exemplary embodiment", "example", "specific example", or "some examples" and the like means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In the specification, the exemplary description of the above terms does not necessarily mean the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0055] In the description of the present application, it should be noted that, unless otherwise explicitly limited, the words such as setting, installing, connecting and the like should be understood in a broad sense, and those skilled in the art can reasonably determine the specific meaning of the above words in the present application in combination with the specific content of the technical solution. In addition, the identification of the specific steps in the following does not represent the limitation of the order and execution logic of the steps, and the execution order and execution logic between the steps should be understood and inferred with reference to the content expressed in the embodiments.

[0056] Intention recognition refers to analyzing the core needs of the user according to the input of the user and outputting information most relevant to the query input. The accuracy of intention recognition greatly affects the accuracy of the search and the intelligence of the dialogue system. For example, in a recommendation task, it is difficult to find relevant movies, music or office needs with incorrect recognition, resulting in very poor user experience. In related technologies, the difficulty of intention recognition lies in that incorrect intention recognition makes it difficult for the machine to accurately understand the meaning expressed by the user, so the answer is often not asked. Therefore, accurate dialogue intention recognition is a challenging task.

[0057] Uncertainty calibration of artificial intelligence model refers to correcting the accuracy of the prediction uncertainty of the artificial intelligence model. It should be noted that when the information input by the user is subjected to intention recognition, the model should not make a response action that wastes resources if it obtains a low confidence prediction result, such as feeding back the low confidence prediction result to an expert for decision making. Therefore, in order to avoid the artificial intelligence model giving an unreasonable response action to the incorrect recognition result, the related art needs to calibrate the uncertainty of the artificial intelligence model. However, how to improve the calibration effect of the uncertainty calibration to improve the accuracy of the intention recognition has become a major problem to be solved in the industry.

[0058] The present application aims to at least solve one of the technical problems existing in the prior art. To this end, the present application proposes an intention recognition method, an electronic device and a storage medium, which can improve the calibration effect of the uncertainty calibration and thus improve the accuracy of the intention recognition.

[0059] Further illustration is made below with reference to the drawings.

[0060] Referring to Figure 1 An optional flowchart of the intent recognition method of the present application can include, but is not limited to, the following steps S101 to S104.

[0061] In step S101, a target information to be recognized and a pre-trained intent recognition model are obtained, the intent recognition model including an attention encoding layer, an evidence neural network and an output layer;

[0062] In step S102, the target information is parsed based on the attention encoding layer to obtain global semantic representation and mean semantic representation;

[0063] In step S103, the global semantic representation and the mean semantic representation are respectively predicted based on the evidence neural network to obtain global classification probability corresponding to the global semantic representation and mean classification probability corresponding to the mean semantic representation;

[0064] In step S104, the global classification probability and the mean classification probability are integrated based on the output layer to obtain an intent recognition result, the intent recognition result being used to represent a dialog intent category corresponding to the target information.

[0065] The intent recognition method of the present application needs to first obtain a target information to be recognized and a pre-trained intent recognition model, the intent recognition model including an attention encoding layer, an evidence neural network and an output layer, the target information is parsed based on the attention encoding layer to obtain global semantic representation and mean semantic representation, the global semantic representation and the mean semantic representation are respectively predicted based on the evidence neural network to obtain global classification probability corresponding to the global semantic representation and mean classification probability corresponding to the mean semantic representation, the global classification probability and the mean classification probability are integrated based on the output layer to obtain an intent recognition result, the intent recognition result being used to represent a dialog intent category corresponding to the target information. After the intent recognition model obtains the target information to be recognized, the three-layer processing of the attention encoding layer, the evidence neural network and the output layer can improve the calibration effect of uncertainty calibration, and thus improve the accuracy of intent recognition.

[0066] In business scenarios such as intelligent customer service and financial transactions, a dialogue robot or an intelligent customer service assistant is often used to recognize an intent based on a dialogue between a user and a machine, so as to make a response action matching the user's intent. It should be noted that the accuracy of intent recognition in business scenarios such as intelligent customer service and financial transactions determines the effect of human-computer interaction. The more accurate the intent recognition is, the more the machine can make corresponding actions according to the recognized intent, such as providing services. Therefore, applying the above-mentioned intent recognition method to business scenarios such as intelligent customer service and financial transactions can greatly improve the effect of human-computer interaction and facilitate the dialogue robot or the intelligent customer service assistant to provide services to the user.

[0067] In step S101 of some embodiments of the present application, the target information to be recognized and the pre-trained intent recognition model need to be obtained first. The intent recognition model includes an attention encoding layer, an evidence neural network, and an output layer. It should be noted that the target information refers to the text information input by the user. The target information to be recognized contains semantic content to be recognized. The purpose of intent recognition is to analyze the core needs of the user and output the most relevant information according to the target information input by the user. It should be noted that the intent recognition model is an artificial intelligence model for intent recognition of the target information. It should be understood that the type of intent recognition model is various, such as a convolutional neural network (CNN) based intent recognition model, a long short-term memory (LSTM) based intent recognition model, an intent recognition model based on LSTM combined with an attention mechanism, and the like, which are not limited to the specific embodiments mentioned above.

[0068] In some exemplary embodiments of the present application, the intent recognition model includes an attention encoding layer, an evidence neural network, and an output layer. The attention encoding layer processes the target information input by the user through an attention mechanism to generate global semantic representation and mean semantic representation. The evidence neural network respectively processes the global semantic representation and the mean semantic representation to obtain global classification probability corresponding to the global semantic representation and mean classification probability corresponding to the mean semantic representation. The output layer is used to integrate the global classification probability and the mean classification probability to obtain an intent recognition result.

[0069] Referring to Figure 2 According to the intent recognition method of some embodiments of the present application, before step S101, the intent recognition method of the present application further includes pre-training an intent recognition model, specifically including but not limited to the following steps S201 to S205.

[0070] Step S201: selecting a training text from a pre-set training data set;

[0071] Step S202, based on the attention encoding layer, the semantic feature of the training text sentence is extracted, and the sentence meaning feature vector and the word meaning average vector are obtained.

[0072] Step S203, based on the evidence neural network, the sentence meaning feature vector and the word meaning average vector are respectively predicted and trained, and the sentence meaning classification probability corresponding to the sentence meaning feature vector and the word meaning classification probability corresponding to the word meaning average vector are obtained.

[0073] Step S204, based on the sentence meaning classification probability and the word meaning classification probability, the classification deviation is analyzed, and the training deviation data is obtained.

[0074] Step S205, based on the training deviation data, the intention recognition model is iteratively updated until the training deviation data meets the preset error condition, and the pre-trained intention recognition model is obtained.

[0075] In step S201 of some embodiments of the present application, a training text sentence is selected from a preset training data set. It should be noted that the training data set refers to a data set used for pre-training the intention recognition model, which contains a plurality of alternative training text sentences. In the process of pre-training the intention recognition model, a training text sentence needs to be selected from the training data set as training material. It should be pointed out that the training data set is a pre-set text sentence data set, and the purpose of pre-training the intention recognition model is to improve the calibration effect of the intention recognition model uncertainty calibration. Therefore, among the plurality of alternative training text sentences, some contain obvious intentions and some contain ambiguous intentions.

[0076] In step S202 of some embodiments of the present application, based on the attention encoding layer, the semantic feature of the training text sentence is extracted, and the sentence meaning feature vector and the word meaning average vector are obtained. In some exemplary embodiments of the present application, the attention encoding layer can be composed of an artificial intelligence model for semantic recognition, such as a bidirectional encoder based on Transformer (Bidirectional Encoder Representations from Transformer, BERT). It should be pointed out that the attention encoding layer can extract semantic features of each word element in the training text sentence to obtain a word meaning feature vector corresponding to each word element and a sentence meaning feature vector of the whole training text sentence. It should be understood that the artificial intelligence model for semantic recognition can use a conventional semantic recognition model. In some embodiments, the ability of the attention encoding layer to extract semantic features of each word element in the training text sentence can also be obtained by performing semantic training on the artificial intelligence model in the attention encoding layer before step S202.

[0077] In some more specific embodiments of the present application, before the training text sentence is input into the attention encoding layer, a classification marker [CLS] is added as a start marker at the start sequence position of the entire training word sequence x i Therefore, the input form of the training text sentence can be represented as x i ={ [CLS], w1, w2,..., w t}. It should be noted that the training text sentence x i ={ [CLS], w1, w2,..., w t} input at the input end of the attention encoding layer can obtain the semantic feature sequence h i ={ h CLS , h1, h2,..., h t} at the output end of the attention encoding layer through the semantic feature extraction of the attention encoding layer, wherein h cLS is obtained from the start sequence position corresponding to the classification marker [CLS], that is, the sentence semantic feature vector of the entire training text sentence x i , which can be used for downstream classification tasks, and h1, h2,..., h t corresponding to the vector are the word semantic feature vectors corresponding to the word elements w1, w2,..., w i in the training text sentence x t . It should be noted that there are various embodiments for extracting semantic features from the training text sentence based on the attention encoding layer to obtain the sentence semantic feature vector and the word semantic feature vector, which can include, but are not limited to, the specific embodiments mentioned above.

[0078] In step S203 of some embodiments of the present application, the sentence semantic feature vector and the word semantic feature vector are respectively predicted and trained based on the evidence neural network to obtain the sentence semantic classification probability corresponding to the sentence semantic feature vector and the word semantic classification probability corresponding to the word semantic feature vector. It should be noted that the evidence neural network (ENN) is developed on the basis of the evidence framework of the evidence theory (Dempster-Shafer Theory, DST) and the subjective logic (Subjective Logic, SL). Its function is to output the probability distribution of each intent category according to the sentence semantic feature vector or the word semantic feature vector.

[0079] In some more specific embodiments of the present application, a certain semantic feature vector h i is input into the evidence neural network ENN, f(.) represents the output of an ENN classification layer, g(.) represents a function that makes the output value of f non-negative, and the ENN classification layer can be represented as: f (h i ) = W e *hi + b, g(f(h i )) = σ(f(h i )) = e i where W e denotes the weights of the ENN classification layer, b denotes the residual, and σ denotes the sigmoid function. If e ik denotes the evidence of the sample h i on the kth class, the evidence output by the ENN classification layer is e i = g(f(h i )). It should be noted that the evidence e i is defined as a measure of the amount of support collected from the data to support the classification of a sample into a certain class, and thus e ik denotes the evidence of the training text sentence x i on the kth class, the probability distribution of the sample x i on the kth class can be reflected, and the predicted probability of the kth intent class is the mean of the corresponding Dirichlet distribution, which is calculated as: α ik = e ik + 1.

[0080] In some example embodiments of the present application, the sentence meaning feature vector represents the overall sentence meaning of each segmented word element in the training text sentence, and thus the sentence meaning feature vector is input into the ENN classification layer to obtain the corresponding sentence meaning classification probability, reflecting the intent classification probability of the training text sentence as a whole. The word meaning mean vector represents the semantic information embodied by the sum and mean of the components of the training text sentence. In some embodiments, the word meaning mean vector is obtained by summing and averaging the word meaning feature vectors corresponding to each segmented word element, and then the word meaning mean vector is input into the ENN classification layer to obtain the corresponding word meaning classification probability, reflecting the intent classification probability of the training text sentence in the mean dimension.

[0081] In steps S204 to S205 of some embodiments of the present application, classification deviation analysis is performed based on the sentence meaning classification probability and the word meaning classification probability to obtain training deviation data, and the intention recognition model is iteratively updated based on the training deviation data until the training deviation data meets the preset error condition, and a pre-trained intention recognition model is obtained. It should be emphasized that the sentence meaning feature vector reflects the semantic information embodied by the training text sentence as a whole, and the word meaning average vector reflects the semantic information embodied by the sum of the components of the training text sentence. Both of them can represent the sentence information of the training text sentence. The closer the sentence meaning feature vector and the word meaning average vector are, the more certain the semantic recognition result of the attention encoding layer for the training text sentence is. If the sentence meaning feature vector and the word meaning average vector are far apart, it means that the semantic recognition result of the attention encoding layer for the training text sentence is difficult to determine. Therefore, in order to improve the calibration effect of the intention recognition model based on uncertainty calibration, classification deviation analysis is performed based on the sentence meaning classification probability and the word meaning classification probability to obtain training deviation data, and the intention recognition model is iteratively updated based on the training deviation data until the training deviation data meets the preset error condition, and a pre-trained intention recognition model is obtained.

[0082] Through steps S201 to S205 of the embodiments of the present application, with the continuous iteration of the update, the recognition accuracy of the sentence meaning feature vector and the average classification probability is improved. Therefore, the calibration effect of the attention encoding layer based on uncertainty calibration can be improved, and the accuracy of intention recognition can be improved.

[0083] Reference Figure 3 According to the intention recognition method of some embodiments of the present application, step S202 can include but is not limited to steps S301 to S303.

[0084] Step S301, performing word segmentation processing on the training text sentence to obtain a training word sequence, the training word sequence including a plurality of word segmentation elements;

[0085] Step S302, performing semantic feature extraction on each word segmentation element based on the attention mechanism of the attention encoding layer to obtain a sentence meaning feature vector and a plurality of word meaning feature vectors;

[0086] Step S303, performing average analysis based on the plurality of word meaning feature vectors to obtain a word meaning average vector.

[0087] In step S301 of some embodiments of the present application, the training text sentence is subjected to word segmentation processing to obtain a training word sequence, and the training word sequence includes a plurality of word segmentation elements. In some embodiments, the training text sentence can be a word sequence format that has been pre-set, such as "x i = {w i , w2, …, w t}", and in other embodiments, the training text sentence is extracted from various types of text materials, such as "xxxx, xxx, xxxx". The advantage of extracting the text sentence from various types of text materials is that the materials are extensive and the cost of making the training data set is low. However, the disadvantage is that the text sentence extracted from various types of text materials cannot be directly input into the attention encoding layer. Therefore, when the training text sentence is extracted from various types of text materials, the training text sentence needs to be subjected to word segmentation processing to obtain a training word sequence in some embodiments of the present application, and the training word sequence includes a plurality of word segmentation elements. Word segmentation is a process of recombining continuous character sequences into word sequences according to certain specifications. Since the expression habits of English, German and other foreign languages naturally divide each word by spaces and punctuation marks, word segmentation for English, German and other foreign languages is often based on spaces or other punctuation marks. Chinese word segmentation refers to the process of dividing a sequence of Chinese characters into individual words. Since Chinese characters are square characters, continuous sub-sequences in Chinese expression habits can contain multiple words, so Chinese word segmentation is more difficult than English word segmentation. It should be understood that word segmentation processing of the training text sentence can be implemented in various ways, not limited to the specific embodiments described above.

[0088] In step S302 of some embodiments of the present application, the attention mechanism of the attention encoding layer is used to extract semantic features of each word segmentation element to obtain a sentence semantic feature vector and a plurality of word semantic feature vectors. It should be emphasized that the attention encoding layer can extract semantic features of each word segmentation element in the training text sentence to obtain a word semantic feature vector corresponding to each word segmentation element and a sentence semantic feature vector of the entire training text sentence. In some embodiments, the attention encoding layer is used to identify the intent of the training text sentence, and the obtained training identification result is a word semantic feature vector extracted for each word segmentation element and a sentence semantic feature vector extracted for the entire training text sentence. Since the training word sequence includes a plurality of word segmentation elements, the number of word semantic feature vectors extracted for each word segmentation element is also multiple.

[0089] In some more specific embodiments of the present application, before the training text sentence is input into the attention encoding layer, a start flag character [CLS] is added to the entire training word sequence x i , so that the input form of the entire model can be represented as x i = {[CLS], w1, w2, …, w t}. It is to be noted that, for the entire training word sequence x i The character [CLS] is added as a start flag. With the semantic feature extraction of the attention encoding layer, the h i = {h CLS , h1, h2,..., h t} obtained at the output end of the attention encoding layer, h CLS is used to extract w1, w2,..., w t The overall sentence meaning, so h CLS The corresponding vector can be used as the x i sentence meaning feature vector of the entire sentence, which can be used for downstream classification tasks, etc., and h1, h2,..., h t The corresponding vectors are the respective word meaning feature vectors w1, w2,..., w t The corresponding word meaning feature vectors (the correspondence between the segmented elements and the word meaning feature vectors is reflected in the subscript sequence number). It is to be noted that the attention mechanism based on the attention encoding layer performs semantic feature extraction on each segmented element, and the implementation of the sentence meaning feature vector and the plurality of word meaning feature vectors is various, which can include, but is not limited to, the specific embodiments mentioned above.

[0090] In step S303 of some embodiments of the present application, the mean value vector is obtained by performing mean value analysis based on the plurality of word meaning feature vectors. It is to be noted that the self-attention mechanism based on the attention encoding layer performs semantic feature extraction on each segmented element to obtain the sentence meaning feature vector and the plurality of word meaning feature vectors. For each segmented element in the training word sequence, the semantic recognition is performed to obtain the word meaning feature vector reflecting the semantics of each segmented element, and then the mean value of each word meaning feature vector is obtained by adding and averaging, so as to obtain the word meaning mean value vector. In some more specific embodiments, if the input form of the training text sentence is represented as x i = {[CLS], w1, w2,..., w t}, wherein the number of segmented elements is t, and the semantic feature of each segmented element is extracted by the self-attention mechanism based on the attention encoding layer to obtain the word meaning feature vector h1, h2,..., h t , then the analysis expression of the word meaning mean value vector can be represented as:

[0091] The sentence semantic feature vector and the word semantic mean vector are obtained through the method shown in steps S301 to S303 of the embodiments of the present application. In the process of continuous iteration and update, the reliability of the sentence semantic feature vector and the word semantic mean vector extracted by the attention encoding layer is improved. It should be noted that the reliability of the sentence semantic feature vector and the word semantic mean vector is the prerequisite for uncertainty calibration. In the case that the sentence semantic feature vector and the word semantic mean vector are both reliable, the closer the sentence semantic feature vector and the word semantic mean vector, the more certain the semantic recognition of the training text sentence by the attention encoding layer is. If the sentence semantic feature vector and the word semantic mean vector are far apart, it is difficult to determine the semantic recognition of the training text sentence by the attention encoding layer.

[0092] With reference to Figure 4 According to some embodiments of the present application, the training word sequence includes a start sequence bit and an element sequence bit. The start sequence bit is configured with a classification identifier, and the element sequence bit is configured with each word element. Step S302 can include, but is not limited to, steps S401 to S402.

[0093] Step S401: performing global semantic extraction on all word elements based on the self-attention mechanism of the attention encoding layer, and obtaining a sentence semantic feature vector based on the classification identifier;

[0094] Step S402: performing local semantic extraction on each word element based on the attention encoding layer, and obtaining a plurality of word semantic feature vectors based on each word element.

[0095] In steps S401 to S402 of some embodiments of the present application, global semantic extraction is first performed on all word elements based on the self-attention mechanism of the attention encoding layer, and a sentence semantic feature vector is obtained based on the classification identifier. Then, local semantic extraction is performed on each word element based on the attention encoding layer, and a plurality of word semantic feature vectors are obtained based on each word element. It should be noted that in some more specific embodiments of the present application, before the training text sentence is input into the attention encoding layer, a classification identifier [CLS] is added as a start mark at the start sequence bit of the entire training word sequence x i , and each word element w1, w2,..., w i is added at the element sequence bit of the training word sequence x t . Therefore, the input form of the training text sentence can be represented as x i ={ [CLS], w1, w2,..., w t}. It should be noted that the training text sentence x i ={ [CLS], w1, w2,..., w t} input from the input end of the attention encoding layer can obtain a semantic feature sequence hi = {h CLS , h1, h2,..., h t}, wherein h CLS is obtained from the start sequence position corresponding to the classification symbol [CLS], that is, the training text sentence x i The sentence meaning feature vector of the whole sentence can be used for downstream classification tasks and the like, and the vectors corresponding to h1, h2,..., h t are the word meaning feature vectors corresponding to the word elements w1, w2,..., w i in the training text sentence x t . It should be pointed out that there are various implementation manners for extracting semantic features of the training text sentence based on the attention encoding layer to obtain the sentence meaning feature vector and the word meaning average vector, which can include, but are not limited to, the specific examples mentioned above.

[0096] Through steps S401 to S402 of the embodiments of the present application, reliable sentence meaning feature vectors and multiple word meaning feature vectors are obtained, which helps to improve the calibration effect of the intent recognition model uncertainty calibration, and further improves the accuracy of intent recognition.

[0097] Referring to Figure 5 , according to some embodiments of the present application, step S204 can include, but is not limited to, steps S501 to S502.

[0098] Step S501, constructing a classification loss function based on the sentence meaning classification probability and the word meaning classification probability;

[0099] Step S502, determining the output value of the classification loss function as the training bias data.

[0100] In steps S501 to S502 of some embodiments of the present application, a classification loss function is first constructed based on the sentence meaning classification probability and the word meaning classification probability, and then the output value of the classification loss function is determined as the training bias data. In some exemplary embodiments of the present application, the sentence meaning feature vector represents the overall sentence meaning of each word element in the training text sentence, so that the corresponding sentence meaning classification probability can be obtained by inputting the sentence meaning feature vector into the ENN classification layer, reflecting the intent classification probability of the training text sentence as a whole. The word meaning average vector represents the semantic information embodied by the sum and average of the components of the training text sentence. In some embodiments, the word meaning average vector is obtained by summing and averaging the word meaning feature vectors corresponding to each word element, and then inputting the word meaning average vector into the ENN classification layer to obtain the corresponding word meaning classification probability, reflecting the intent classification probability of the training text sentence in the average dimension.

[0101] It should be emphasized that the sentence meaning feature vector reflects the semantic information embodied by the training text sentence as a whole, and the word meaning average vector reflects the semantic information embodied by the sum of the components of the training text sentence and the average value, both of which can represent the sentence information of the training text sentence. The closer the sentence meaning feature vector and the word meaning average vector are, the more certain the semantic recognition of the attention encoding layer for the training text sentence is. If the sentence meaning feature vector and the word meaning average vector are far apart, it is difficult to determine the semantic recognition of the attention encoding layer for the training text sentence. It should be clear that the classification loss function is used to calculate the difference between the sentence classification probability and the word classification probability. In the process of continuous iteration training, the output value of the classification loss function is determined as the training deviation data. Based on the training deviation data, the intention recognition model can be iteratively updated in each iteration until the training deviation data meets the preset error condition, and the pre-trained intention recognition model is obtained.

[0102] Therefore, by constructing the classification loss function through steps S501 to S502 of the embodiments of the present application, and determining the output value of the classification loss function as the training deviation data, the intention recognition model can be iteratively trained, which can improve the calibration effect of the intention recognition model uncertainty calibration, and further improve the accuracy of the intention recognition.

[0103] Reference Figure 6 According to some embodiments of the present application, step S501 can include but is not limited to steps S601 to S602.

[0104] Step S601, obtaining an evidence mapping analytical expression corresponding to the evidence neural network;

[0105] Step S602, constructing a classification loss function based on the word classification probability, the sentence classification probability and the evidence mapping analytical expression.

[0106] In steps S601 to S602 of some embodiments of the present application, the evidence mapping analytical expression corresponding to the evidence neural network is first obtained, and then the classification loss function is constructed based on the word classification probability, the sentence classification probability and the evidence mapping analytical expression. It should be noted that for a K-class classification problem, ENN will provide a belief quality and an overall uncertainty u for any one class k∈{1,2,…,K}. These two will satisfy the condition and b k and u are non-negative. The evidence is defined as a measure of the amount of support collected from the data to support the classification of the sample into a certain class. Let e ik represent the evidence of the sample x i in the kth class, then the belief quality b k and the uncertainty u i are calculated as:

[0107]

[0108] Assume that the allocation of this belief quality (i.e. subjective opinion) obeys a Dirichlet distribution with parameters ik = e ik + 1, where S i is called the Dirichlet strength.

[0109] Therefore, the density function of the Dirichlet distribution with K parameters K = [a1, a2,..., aK] is represented as:

[0110]

[0111] where S K is a K-dimensional simplex: B(a) is a K-dimensional beta function.

[0112] In some more specific embodiments of the present application, a semantic feature vector h i is input into an evidence neural network ENN, and f(.) represents the output of an ENN classification layer, and g(.) represents a function that makes the output value of f non-negative, then the ENN classification layer can be represented as: f(h i ) = W e *h i + b, g(f(h i )) = σ(f(h i )) = e i , where W e represents the weights of the ENN classification layer, b represents the residual, and σ represents the sigmoid function. If e ik represents the evidence of the sample h i in the kth class, then the evidence output by the ENN classification layer is: e i = g(f(h i )). It should be pointed out that the evidence e i is defined as a measure of the amount of support collected from the data to support the classification of the sample into a certain class, so e ik represents the evidence of the training text sentence x i in the kth class, then it can reflect the probability distribution of the sample x i in the kth class, and the predicted probability in the kth intent class is the mean of the corresponding Dirichlet distribution, which is calculated as: a ik = e ik + 1, Therefore, based on the integration of the above formulas, the evidence mapping analytical expression corresponding to the evidence neural network can be obtained:

[0113]

[0114] By constructing the classification loss function through the evidence mapping analysis formula of steps S601 to S602 of the embodiments of the present application, and determining the output value of the classification loss function as the training bias data, the intent recognition model can be iteratively trained, the calibration effect of the uncertainty calibration of the intent recognition model can be improved, and the accuracy of the intent recognition can be further improved.

[0115] With reference to Figure 7 According to some embodiments of the present application, step S602 comprises:

[0116] Step S701, constructing a first cross-entropy sub-function based on the sentence meaning classification probability and the evidence mapping analysis formula;

[0117] Step S702, constructing a second cross-entropy sub-function based on the word meaning classification probability and the evidence mapping analysis formula;

[0118] Step S703, integrating the first cross-entropy sub-function and the second cross-entropy sub-function to obtain the classification loss function.

[0119] Steps S701 to S703 of some embodiments of the present application, based on the sentence meaning classification probability and the evidence mapping analysis formula, construct a first cross-entropy sub-function, and then based on the word meaning classification probability and the evidence mapping analysis formula, construct a second cross-entropy sub-function, and further integrate the first cross-entropy sub-function and the second cross-entropy sub-function to obtain the classification loss function. It should be noted that in some more specific embodiments, the evidence mapping analysis formula corresponding to the evidence neural network is:

[0120]

[0121] On this basis, the sentence meaning classification probability e ik CLS is substituted into the evidence mapping analysis formula L ENN , and the corresponding first cross-entropy sub-function

[0122] Similarly, the word meaning classification probability is substituted into the evidence mapping analysis formula L ENN , and the corresponding second cross-entropy sub-function

[0123] Therefore, after constructing the first cross-entropy sub-function and the second cross-entropy sub-function, further integrating the first cross-entropy sub-function and the second cross-entropy sub-function can obtain the classification loss function.

[0124] By constructing the classification loss function through steps S701 to S703 of the embodiments of the present application, and determining the output value of the classification loss function as the training deviation data, the intent recognition model can be iteratively trained, the calibration effect of the uncertainty calibration of the intent recognition model can be improved, and the accuracy of the intent recognition can be improved.

[0125] With reference to Figure 8 According to some embodiments of the present application, step S703 can include but is not limited to steps S801 to S802.

[0126] In step S801, relative entropy analysis is performed based on the word sense classification probability and the sentence sense classification probability to obtain semantic information divergence.

[0127] In step S802, the first cross-entropy sub-function and the second cross-entropy sub-function are integrated with the semantic information divergence to obtain the classification loss function.

[0128] In steps S801 to S802 of the embodiments of the present application, first, relative entropy analysis is performed based on the word sense classification probability and the sentence sense classification probability to obtain semantic information divergence, and then the first cross-entropy sub-function and the second cross-entropy sub-function are integrated with the semantic information divergence to obtain the classification loss function. It should be emphasized that the sentence sense feature vector reflects the semantic information embodied by the training text sentence as a whole, and the word sense mean vector reflects the semantic information embodied by the sum of the components of the training text sentence. Both of them can represent the sentence information of the training text sentence. The closer the sentence sense feature vector and the word sense mean vector are, the more certain the semantic recognition result of the attention encoding layer for the training text sentence is. If the sentence sense feature vector and the word sense mean vector are far apart, it means that the semantic recognition result of the attention encoding layer for the training text sentence is difficult to determine. It should be clear that the classification loss function is used to calculate the difference between the sentence sense classification probability and the word sense classification probability. In the process of continuous iteration training, the output value of the classification loss function is determined as the training deviation data, so that the intent recognition model can be iteratively updated based on the training deviation data in each iteration until the training deviation data meets the preset error condition, and the pre-trained intent recognition model is obtained. In some preferred embodiments of the present application, calculating the KL divergence of the first cross-entropy sub-function and the second cross-entropy sub-function can make the intent recognition model tend to have a high accuracy and good calibration effect, and can effectively alleviate overfitting:

[0129]

[0130] Therefore, in some more specific embodiments of the present application, the first cross-entropy sub-function and the second cross-entropy sub-function are integrated with the semantic information divergence to obtain the classification loss function, which is specifically:

[0131] Totalloss = aloss1 + b loss2 + g loss3

[0132] wherein a, b, g are hyperparameters. It should be noted that there are two types of parameters in a machine learning model: one type is learned and estimated from data, called model parameters, which are parameters of the model itself. For example, the weighting coefficient (slope) and the bias term (intercept) of a linear regression line are model parameters. The other type is tuning parameters in machine learning algorithms, which need to be flexibly set according to existing or existing experience, called hyperparameters. For example, the regularization coefficient λ and the depth of the tree in the decision tree model. Hyperparameters are also a kind of parameter, which has the characteristics of a parameter, such as unknown, that is, it is not a known constant. A manually configurable setting needs to specify the "correct" value for it according to existing or existing experience, that is, a flexibly set value, which is not learned by the system.

[0133] By constructing the classification loss function through steps S801 to S802 of the embodiments of the present application, and determining the output value of the classification loss function as the training bias data, the intent recognition model can be iteratively trained, which can improve the calibration effect of the uncertainty calibration of the intent recognition model, and further improve the accuracy of intent recognition.

[0134] In step S102 of some embodiments of the present application, the target information is analyzed and processed based on the attention encoding layer to obtain the global semantic representation and the mean semantic representation. It should be noted that the attention encoding layer processes the target information input by the user through the attention mechanism to analyze and process the target information, thereby generating the global semantic representation and the mean semantic representation. The target information refers to the text information input by the user. In the attention encoding layer, the target information in the form of text can be first processed by word segmentation to obtain a corresponding target word sequence, the target word sequence is composed of a plurality of target word segmentation elements, then the global semantic representation reflecting the overall sentence meaning of the target word sequence is obtained by performing overall semantic recognition on the target word sequence, and the element semantic vector reflecting the semantic of each target word segmentation element is obtained by performing local semantic recognition on each target word segmentation element in the target word sequence, and then the mean semantic representation is obtained by summing and averaging each element semantic vector. It should be noted that the global semantic representation reflects the semantic information embodied by the target information as a whole, and the mean semantic representation reflects the semantic information embodied by the sum and average of the components of the target information. Both of them can represent the sentence information of the target information. The closer the global semantic representation and the mean semantic representation, the more certain the semantic recognition result of the target information by the attention encoding layer is. If the global semantic representation and the mean semantic representation are quite different, it is difficult for the attention encoding layer to determine the semantic recognition result of the target information.

[0135] It should be noted that the bidirectional encoder based on the transformer (Bidirectional Encoder Representations from Transformer, BERT) is an artificial intelligence model based on the attention mechanism of the transformer. The BERT model uses large-scale unlabeled pre-training to obtain representations containing the inherent semantic information of the text. It should be noted that the core of the BERT model is the transformer, and the core of the transformer is the attention mechanism. The role of the attention mechanism is to allow the neural network to focus on a part of the input. The attention mechanism of the BERT model mainly involves three concepts: Query, Value, and Key. Query refers to the target word or the word to be generated, Value refers to the original Value representation of each word in the context of the input information, and Key refers to the Key vector representation of each word in the context of the input information. The semantic representation of the target word can be generated by calculating the similarity between Query and Key and adding Value, i.e., Attention(Query, Source) = ∑ i Similarity(Query, Key i )*Value, where Source = <Key, Value>. Through the above formula, the semantic representation (Attention value) of the target word can be obtained.

[0136] There are generally two types of parameters in machine learning models: one type needs to be learned and estimated from data, called model parameters (Parameter), which are the parameters of the model itself. For example, the weighted coefficients (slope) and bias terms (intercept) of the linear regression straight line are model parameters. The other type is the tuning parameter (Tuning Parameters) in the machine learning algorithm, which needs to be flexibly set according to existing or existing experience, called hyperparameter (Hyperparameter). For example, the regularization coefficient λ and the depth of the tree in the decision tree model. Hyperparameters are also a type of parameter, which has the characteristics of a parameter, such as being unknown, that is, it is not a known constant. A manually configurable setting needs to specify the "correct" value for it according to existing or existing experience, that is, a flexibly set value that is not learned by the system.

[0137] According to the self-attention mechanism of the BERT model, each word in the input information needs to pass through three weight matrices W Q , WK , W V A linear change is made to generate the Query, Value, Key three vectors of each word itself. When self-attention processing is performed with a word as the center, the Key vector of the word is used to do dot product with the Query vector of each word, and then the weight is normalized through Softmax. Then, the semantic information of all words is calculated by weighting and fusion, and the semantic representation of each word can be obtained. It should be noted that due to the length limitation of the text, the effect of the BERT model in processing long text is not very good. Therefore, in some preferred embodiments of the present application, a hyperparameter μ is used to improve the prediction effect of the BERT model when inputting long text. Let n represent the length of the input text, d represent the dimension of the vector, Q=xW Q , K=xW K , V=xW V wherein x represents the input matrix, that is, the output of the upper model, and the specific analytical expression is: In order to avoid the attention mechanism of the BERT model from weakening when processing long text, the role of the hyperparameter μ is to weight the Attention value, thereby improving the ability of the BERT model to process long text.

[0138] It should be pointed out that the advantages of the BERT model are as follows: firstly, the expression ability of the model can be fully trained, and the feature extraction ability of the Transformer is stronger than that of the bidirectional LSTM; secondly, the BERT model can obtain higher-level semantic representation of the sentence level than the word; thirdly, the BERT model can combine the pre-training model and the downstream task model together, that is, the BERT model can still be used when doing the downstream task, and it is not necessary to modify the model; fourthly, the BERT model has a small fine-tuning cost. Therefore, in some preferred embodiments of the present application, the BERT model is selected as the attention encoding layer of the intent recognition model.

[0139] In step S103 of some embodiments of the present application, the global semantic representation and the mean semantic representation are respectively processed by the evidence neural network to obtain a global classification probability corresponding to the global semantic representation and a mean classification probability corresponding to the mean semantic representation. It should be noted that the evidence neural network (ENN) is developed on the basis of the evidence framework of the Dempster-Shafer Theory (DST) and the Subjective Logic (SL). In some exemplary embodiments of the present application, the evidence neural network obtained through pre-training is used to respectively process the global semantic representation and the mean semantic representation, and then the global classification probability corresponding to the global semantic representation and the mean classification probability corresponding to the mean semantic representation are obtained. It should be emphasized that the global semantic representation reflects the semantic information embodied by the target information as a whole, and the mean semantic representation reflects the semantic information embodied by the sum of the components of the target information. Both of them can represent the sentence information of the target information. After the global semantic representation is processed by the evidence neural network, the corresponding global classification probability can be obtained. After the mean semantic representation is processed by the evidence neural network, the corresponding mean classification probability can be obtained.

[0140] In step S104, the global classification probability and the mean classification probability are integrated by the output layer to obtain an intent recognition result, which is used to represent the dialog intent category corresponding to the target information. It should be noted that the integration of the global classification probability and the mean classification probability can be achieved in various ways, such as summing the global classification probability and the mean classification probability, averaging the global classification probability and the mean classification probability, or weighted summing the global classification probability and the mean classification probability. Then, the comprehensive classification probability is classified by the Softmax Regression. It should be noted that the closer the global classification probability and the mean classification probability, the more accurate the prediction result of the intent recognition model. If the global classification probability and the mean classification probability are far apart, it means that the intent recognition model has obtained a prediction result with low confidence when identifying the intent of the user input. For the prediction result with low confidence, a response action that wastes resources should not be made. Therefore, through the processing of the attention encoding layer and the evidence neural network, the artificial intelligence model can avoid giving an unreasonable response action for a possible incorrect recognition result, thereby improving the calibration effect of uncertainty calibration and the quality of intent recognition. It should be understood that the integration of the global classification probability and the mean classification probability by the output layer to obtain the intent recognition result can be achieved in various ways, which are not limited to the specific embodiments mentioned above.

[0141] Figure 9 An electronic device 900 provided by an embodiment of the present application is shown. The electronic device 900 includes a processor 901, a memory 902, and a computer program stored in the memory 902 and executable on the processor 901, and the computer program, when executed, is configured to perform the intent recognition method described above.

[0142] The processor 901 and the memory 902 can be connected by a bus or other means.

[0143] The memory 902, as a non-transitory computer readable storage medium, can be configured to store non-transitory software programs and non-transitory computer executable programs, such as the intent recognition method described in the embodiments of the present application. The processor 901 executes the non-transitory software programs and instructions stored in the memory 902, thereby implementing the intent recognition method described above.

[0144] The memory 902 can include a program storage area and a data storage area, wherein the program storage area can store an operating system and at least one application required by a function. The data storage area can store the intent recognition method described above. In addition, the memory 902 can include a high-speed random access memory 902, and can also include a non-transitory memory 902, such as at least one storage device, a flash memory device, or other non-transitory solid-state memory device. In some embodiments, the memory 902 can optionally include a memory 902 remotely arranged with respect to the processor 901, and these remote memories 902 can be connected to the electronic device 900 through a network. Examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0145] The non-transitory software programs and instructions required to implement the intent recognition method described above are stored in the memory 902, and when executed by one or more processors 901, the intent recognition method described above is performed, for example, the method steps S101 to S104 in Figure 1 , the method steps S201 to S205 in Figure 2 , the method steps S301 to S303 in Figure 3 , the method steps S401 to S402 in Figure 4 , the method steps S501 to S502 in Figure 5 , the method steps S601 to S602 in Figure 6 , the method steps S701 to S703 in Figure 7 , and the method steps S801 to S802 in Figure 8 .

[0146] The embodiment of the present application further provides a computer readable storage medium, which stores computer executable instructions, and the computer executable instructions are used for executing the above-mentioned intent recognition method.

[0147] In an embodiment, the computer readable storage medium stores computer executable instructions, which are executed by one or more control processors, for example, to execute the method steps S101 to S104 in the method in the embodiment, Figure 1 the method steps S201 to S205 in the method in the embodiment, Figure 2 the method steps S301 to S303 in the method in the embodiment, Figure 3 the method steps S401 to S402 in the method in the embodiment, Figure 4 the method steps S501 to S502 in the method in the embodiment, Figure 5 the method steps S601 to S602 in the method in the embodiment, Figure 6 the method steps S701 to S703 in the method in the embodiment, Figure 7 the method steps S801 to S802 in the method in the embodiment. Figure 8 The apparatus embodiment described above is only illustrative, wherein the units described as separate components can or can not be physically separated, that is, can be located in one place, or can be distributed to multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0148]

[0149] ​As will be appreciated by one of ordinary skill in the art, all or some of the steps, systems, etc. in the methods disclosed above can be embodied in software, firmware, hardware, and / or suitable combinations thereof. Some or all of the physical components can be implemented with software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or can be implemented with hardware, or can be implemented with an integrated circuit, such as an application- specific integrated circuit. Such software can be distributed on computer readable media, which can comprise computer storage media (or non-transitory media), and communication media (or transitory media). As is well known to a person of ordinary skill in the art, the term computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, storage devices, storage apparatus or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by a computer. Further, as is well known to a person of ordinary skill in the art, communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media. It is also to be understood that the various embodiments provided in this application can be combined in any manner, to achieve different technical effects.

[0150] The above is a specific description of the preferred embodiments of the present application, but the present application is not limited to the above-described embodiments. Those skilled in the art can make various equivalent modifications or replacements without departing from the spirit of the present application, and these equivalent modifications or replacements are included in the scope defined by the claims of the present application.

Claims

1. An intent recognition method, characterized in that, include: The target information to be identified and a pre-trained intent recognition model are obtained, wherein the intent recognition model includes an attention encoding layer, an evidence neural network, and an output layer; The target information is parsed and processed based on the attention coding layer to obtain a global semantic representation and a mean semantic representation. Based on the evidence neural network, the global semantic representation and the mean semantic representation are predicted respectively to obtain the global classification probability corresponding to the global semantic representation and the mean classification probability corresponding to the mean semantic representation. Based on the output layer, the global classification probability and the mean classification probability are integrated to obtain the intent recognition result, which is used to characterize the dialogue intent category corresponding to the target information; Prior to acquiring the target information to be identified and the pre-trained intent recognition model, the process further includes pre-training the intent recognition model, specifically including: Select training text sentences from a pre-set training dataset; Based on the attention encoding layer, semantic features are extracted from the training text sentence to obtain a sentence meaning feature vector and a word meaning mean vector; wherein, the sentence meaning feature vector represents the overall sentence meaning of each word segment element in the training text sentence, and the word meaning mean vector represents the semantic information reflected by the summation and averaging of the components of the training text sentence; Based on the evidence neural network, the sentence meaning feature vector and the word meaning mean vector are predicted and trained respectively to obtain the sentence meaning classification probability corresponding to the sentence meaning feature vector and the word meaning classification probability corresponding to the word meaning mean vector; Based on the sentence meaning classification probability and the word meaning classification probability, classification bias analysis is performed to obtain training bias data; The intent recognition model is iteratively updated based on the training bias data until the training bias data meets the preset error conditions, thus obtaining the pre-trained intent recognition model.

2. The method according to claim 1, characterized in that, The step of extracting semantic features from the training text sentences based on the attention encoding layer to obtain sentence meaning feature vectors and word meaning mean vectors includes: The training text sentence is segmented to obtain a training word sequence, which includes multiple segmentation elements. Based on the attention mechanism of the attention encoding layer, semantic features are extracted from each word segmentation element to obtain sentence semantic feature vector and multiple word semantic feature vectors; The mean vector of word meaning is obtained by performing mean analysis on multiple word meaning feature vectors.

3. The method according to claim 2, characterized in that, The training word sequence includes a start sequence position and element sequence positions. The start sequence position is configured with a classification identifier, and the element sequence positions are configured with each word segmentation element. The semantic features of each word segmentation element are extracted based on the attention mechanism of the attention encoding layer to obtain a sentence meaning feature vector and multiple word meaning feature vectors, including: Global semantic extraction is performed on all the word segmentation elements based on the self-attention mechanism of the attention encoding layer, and the sentence semantic feature vector is obtained based on the classification identifier. Based on the attention encoding layer, local semantic extraction is performed on each of the word segmentation elements, and multiple word meaning feature vectors are obtained based on each of the word segmentation elements.

4. The method according to claim 1, characterized in that, The classification bias analysis based on the sentence meaning classification probability and the word meaning classification probability yields training bias data, including: Based on the sentence meaning classification probability and the word meaning classification probability, a classification loss function is constructed; The output value of the classification loss function is determined as the training bias data.

5. The method according to claim 4, characterized in that, The step of constructing a classification loss function based on the sentence meaning classification probability and the word meaning classification probability includes: Obtain the evidence mapping analytical expression corresponding to the evidence neural network; The classification loss function is constructed based on the word meaning classification probability, the sentence meaning classification probability, and the evidence mapping expression.

6. The method according to claim 5, characterized in that, The process of constructing the classification loss function based on the word meaning classification probability, the sentence meaning classification probability, and the evidence mapping expression includes: Based on the semantic classification probability and the evidence mapping expression, a first cross-entropy sub-function is constructed; Based on the word meaning classification probability and the evidence mapping expression, a second cross-entropy sub-function is constructed; The first cross-entropy subfunction and the second cross-entropy subfunction are integrated to obtain the classification loss function.

7. The method according to claim 6, characterized in that, The step of integrating the first cross-entropy sub-function and the second cross-entropy sub-function to obtain the classification loss function includes: Based on the word meaning classification probability and the sentence meaning classification probability, relative entropy analysis is performed to obtain semantic information divergence; The classification loss function is obtained by integrating the first cross-entropy sub-function, the second cross-entropy sub-function, and the semantic information divergence.

8. An electronic device, characterized in that, include: A memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the intent recognition method as described in any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that is executed by a processor to implement the intent recognition method as described in any one of claims 1 to 7.

Images