Intention recognition method and device based on loss function, equipment and storage medium

Abstract

The present application relates to the field of artificial intelligence, and discloses an intention recognition method and device based on a loss function, equipment and a storage medium, which are used for directly calculating the influence of each label in a text on a loss function and improving the recognition ability of a model for multiple intentions. The method comprises the following steps: obtaining a to-be-recognized text, which is used for indicating at least one intention of a target user; calling an input layer of an improved machine learning model FastText to perform word segmentation on the to-be-recognized text and obtain a plurality of segmented vectors; calling a hidden layer of the improved FastText to perform superposition averaging on the plurality of segmented vectors and obtain a document vector; calling a binary classification cross-entropy loss function BCELoss to calculate each potential intention in the document vector and obtain loss values of the plurality of potential intentions; screening the loss values of the plurality of potential intentions according to the loss values corresponding to each potential intention and a pre-set threshold value, and determining a plurality of candidate intention labels of the to-be-recognized text based on the screened loss values.

Description

Technical Field

[0001] The present invention relates to the technical field of classification algorithms, and in particular to a loss function-based intent recognition method, apparatus, device, and storage medium. Background Art

[0002] In multi-turn dialogue systems, natural language understanding (NLU) is a crucial step in obtaining information from users. Recognizing intent in text is one of the most commonly used methods for natural language understanding. However, this approach is often limited to one intent per sentence. A classifier categorizes the user's words into one of multiple pre-designed categories, a process known as multi-classification. However, a user's sentence often expresses more than one intent. In such cases, simply identifying intent alone is insufficient to meet business needs. Therefore, dialogue systems require a natural language understanding module that can simultaneously identify multiple user intents. This extends traditional single-intent recognition to multiple intents.

[0003] Multi-intent recognition has always been a difficult problem in the industry. The usual method is to use rule matching. The principle is to manually design keywords for all intents in advance. If a sentence matches multiple keywords, it is considered to have hit multiple intents. However, this method has disadvantages such as poor scalability, high manual workload, and incomplete coverage. Summary of the Invention

[0004] The present invention provides an intent recognition method, apparatus, device and storage medium based on a loss function, which are used to directly calculate the impact of each label in a text on the loss function, adjust the probability distribution of the text among all intents, and take the losses at all label positions into account, thereby increasing the confidence of the label position and reducing the confidence of the remaining positions, and improving the model's recognition ability for multiple intents.

[0005] A first aspect of an embodiment of the present invention provides an intent recognition method based on a loss function, comprising: obtaining a text to be recognized, wherein the text to be recognized is used to indicate at least one intent of a target user; calling the input layer of an improved machine learning model FastText to segment the text to be recognized to obtain multiple segmentation vectors; calling the hidden layer of the improved FastText to superimpose and average the multiple segmentation vectors to obtain a document vector; calling the cross-entropy loss function BCELoss for binary classification to calculate each potential intent in the document vector to obtain loss values ​​of multiple potential intents; screening the loss values ​​of the multiple potential intents according to the loss value corresponding to each potential intent and a pre-set threshold, and determining multiple candidate intent labels for the text to be recognized based on the screened loss values.

[0006] Optionally, in a first implementation method of the first aspect of the embodiment of the present invention, the cross entropy loss function BCELoss for binary classification is called to calculate each potential intent in the document vector to obtain loss values ​​of multiple potential intents, including: inputting the document vector into the improved FastText for negative sampling to obtain multiple sampling vectors; calling a hierarchical classifier to import the multiple sampling vectors into a tree structure to obtain a tree classification structure, which includes multiple potential intent labels; calculating the multiple potential intent labels through the S-type threshold function Sigmoid and the cross entropy loss function BCELoss for binary classification to obtain loss values ​​of multiple potential intents.

[0007] Optionally, in the second implementation of the first aspect of the embodiment of the present invention, the multiple potential intent labels are calculated using the S-type threshold function Sigmoid and the cross entropy loss function BCELoss for binary classification to obtain the loss values ​​of multiple potential intents, including: obtaining the calculated probability values ​​of multiple potential intent labels through the threshold function Sigmoid ; Calculate the loss values ​​of multiple potential intent labels based on the obtained true probability value y and the preset BCELoss formula, where the preset BCELoss formula is .

[0008] Optionally, in a third implementation method of the first aspect of the embodiment of the present invention, the loss values ​​of the multiple potential intentions are screened according to the loss value corresponding to each potential intention and a pre-set threshold, and multiple candidate intention labels of the text to be recognized are determined based on the screened loss value, including: obtaining a pre-set threshold; judging whether the loss value corresponding to each potential intention is greater than the threshold; if the target loss value is greater than the threshold, determining the potential intention corresponding to the target loss value as the candidate intention label of the text to be recognized, and obtaining multiple candidate intention labels.

[0009] Optionally, in a fourth implementation method of the first aspect of the embodiment of the present invention, the input layer of the improved machine learning model FastText is called to segment the text to be recognized to obtain multiple segmentation vectors, including: segmenting the text to be recognized through a preset word bag to obtain multiple candidate words; calling a preset n-gram model to represent the multiple candidate words to obtain model features of each candidate word; inputting the model features of each candidate word into the input layer of the improved machine learning model FastText to generate multiple segmentation vectors, each segmentation vector corresponding to a candidate word.

[0010] Optionally, in a fifth implementation of the first aspect of the embodiment of the present invention, the calling of the hidden layer of the improved FastText to superimpose and average the multiple word segmentation vectors to obtain a document vector includes: arranging the multiple word segmentation vectors in the word segmentation order to obtain a word segmentation sequence; inputting the word segmentation sequence into the hidden layer of the improved FastText in sequence according to the word segmentation order for averaging processing; obtaining the output result of the hidden layer of the improved FastText to obtain a document vector.

[0011] Optionally, in a sixth implementation of the first aspect of the embodiment of the present invention, after filtering the loss values ​​of the multiple potential intentions according to the loss value corresponding to each potential intention and a pre-set threshold, and determining the multiple candidate intent labels of the text to be recognized based on the filtered loss values, the method further includes: labeling the text to be recognized according to the multiple candidate intent labels.

[0012] The second aspect of an embodiment of the present invention provides an intent recognition device based on a loss function, including: an acquisition unit, used to acquire a text to be recognized, wherein the text to be recognized is used to indicate at least one intention of a target user; a word segmentation unit, used to call the input layer of an improved machine learning model FastText to perform word segmentation on the text to be recognized, and obtain multiple word segmentation vectors; an averaging unit, used to call the hidden layer of the improved FastText to superimpose and average the multiple word segmentation vectors to obtain a document vector; a calculation unit, used to call the cross-entropy loss function BCELoss for binary classification to calculate each potential intent in the document vector, and obtain loss values ​​of multiple potential intents; a screening unit, used to screen the loss values ​​of the multiple potential intents according to the loss value corresponding to each potential intent and a pre-set threshold, and determine multiple candidate intent labels for the text to be recognized based on the screened loss values.

[0013] Optionally, in a first implementation method of the second aspect of the embodiment of the present invention, the computing unit includes: a negative sampling module, used to input the document vector into the improved FastText for negative sampling to obtain multiple sampling vectors; an import module, used to call a hierarchical classifier to import the multiple sampling vectors into a tree structure to obtain a tree classification structure, wherein the tree classification structure includes multiple potential intent labels; a computing module, used to calculate the multiple potential intent labels using the S-type threshold function Sigmoid and the cross entropy loss function BCELoss for binary classification to obtain loss values ​​of multiple potential intents.

[0014] Optionally, in the second implementation of the second aspect of the embodiment of the present invention, the calculation module is specifically used to: obtain the calculated probability values ​​of multiple potential intent labels through the threshold function Sigmoid ; Calculate the loss values ​​of multiple potential intent labels based on the obtained true probability value y and the preset BCELoss formula, where the preset BCELoss formula is .

[0015] Optionally, in a third implementation method of the second aspect of the embodiment of the present invention, the screening unit is specifically used to: obtain a pre-set threshold; determine whether the loss value corresponding to each potential intention is greater than the threshold; if the target loss value is greater than the threshold, then determine the potential intention corresponding to the target loss value as the candidate intention label of the text to be identified, and obtain multiple candidate intention labels.

[0016] Optionally, in a fourth implementation method of the second aspect of the embodiment of the present invention, the word segmentation unit is specifically used to: segment the text to be recognized through a preset word bag to obtain multiple candidate words; call a preset n-gram model to characterize the multiple candidate words to obtain model features of each candidate word; input the model features of each candidate word into the input layer of the improved machine learning model FastText to generate multiple word segmentation vectors, each word segmentation vector corresponds to a candidate word.

[0017] Optionally, in a fifth implementation of the second aspect of the embodiment of the present invention, the averaging unit is specifically used to: arrange multiple word segmentation vectors in a word segmentation order to obtain a word segmentation sequence; input the word segmentation sequence into the hidden layer of the improved FastText in sequence according to the word segmentation order for averaging processing; obtain the output result of the hidden layer of the improved FastText to obtain a document vector.

[0018] Optionally, in a sixth implementation of the second aspect of the embodiment of the present invention, the loss function-based intent recognition device further includes: a labeling unit, configured to label the text to be recognized according to a plurality of candidate intent labels.

[0019] The third aspect of an embodiment of the present invention provides an intention recognition device based on a loss function, comprising a memory, a processor, and a computer program stored on the memory and runnable on the processor, wherein the processor implements the above-mentioned intention recognition method based on the loss function when executing the computer program.

[0020] A fourth aspect of an embodiment of the present invention provides a computer-readable storage medium, which stores computer-readable instructions. When the computer-readable instructions are executed by a processor, the steps of the above-mentioned loss function-based intention recognition method are implemented.

[0021] In the technical solution provided by the embodiment of the present invention, a text to be recognized is obtained, and the text to be recognized is used to indicate at least one intention of the target user; the input layer of the improved machine learning model FastText is called to segment the text to be recognized to obtain multiple segmentation vectors; the hidden layer of the improved FastText is called to superimpose and average the multiple segmentation vectors to obtain a document vector; the cross entropy loss function BCELoss for binary classification is called to calculate each potential intent in the document vector to obtain loss values ​​of multiple potential intents; the loss values ​​of multiple potential intents are screened according to the loss value corresponding to each potential intent and a pre-set threshold, and multiple candidate intent labels of the text to be recognized are determined based on the screened loss values. The embodiment of the present invention directly calculates the influence of each label in the text on the loss function, adjusts the probability distribution of the text in all intentions, and takes into account the losses at all label positions, thereby increasing the confidence at the label position and reducing the confidence at other positions, and improving the model's recognition ability for multiple intents. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 Schematic diagram of an embodiment of the intention recognition method based on the loss function in an embodiment of the present invention;

[0023] Figure 2 2 is a schematic diagram of another embodiment of the method for intention recognition based on a loss function in an embodiment of the present invention;

[0024] Figure 3 Schematic diagram of an embodiment of an intention recognition device based on a loss function in an embodiment of the present invention;

[0025] Figure 4 2 is a schematic diagram of another embodiment of an intention recognition device based on a loss function according to an embodiment of the present invention;

[0026] Figure 5 Schematic diagram of an embodiment of an intention recognition device based on a loss function in an embodiment of the present invention. DETAILED DESCRIPTION

[0027] The present invention provides an intent recognition method, apparatus, device and storage medium based on a loss function, which are used to directly calculate the impact of each label in a text on the loss function, adjust the probability distribution of the text among all intents, and take the losses at all label positions into account, thereby increasing the confidence of the label position and reducing the confidence of the remaining positions, and improving the model's recognition ability for multiple intents.

[0028] In order to enable those skilled in the art to better understand the solutions of the present invention, the embodiments of the present invention will be described below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0029] The terms "first," "second," "third," "fourth," and so on (if any) in the description and claims of the present invention and in the accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a particular order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate, so that the embodiments described herein can be implemented in an order other than that shown or described herein. In addition, the terms "including" or "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus that includes a series of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, product, or apparatus.

[0030] See also Figure 1 , a flowchart of the loss function-based intent recognition method provided by an embodiment of the present invention specifically includes:

[0031] 101. Obtain text to be recognized, where the text to be recognized is used to indicate at least one intention of a target user.

[0032] The server obtains the text to be recognized, where the text to be recognized is used to indicate at least one intention of the target user.

[0033] It should be noted that different texts to be recognized may or may not correspond to the same intentions. Different texts to be recognized may express the same one or more intentions. For example, "Can I buy Ping An Fu?" and "Can I buy Ping An Fu products?" express the same intention; the same text to be recognized may express different intentions. For example, "I want to buy Ping An Fu" and "I want to buy Ping An Fu?" express different intentions.

[0034] It is understandable that the execution subject of the present invention can be an intention recognition device based on a loss function, or a terminal or a server, which is not limited here. The embodiment of the present invention is described by taking a server as the execution subject as an example.

[0035] 102. Call the input layer of the improved machine learning model FastText to segment the text to be recognized and obtain multiple segmentation vectors.

[0036] The server uses the input layer of the improved machine learning model FastText to segment the text to be recognized, generating multiple segmentation vectors. Specifically, the server uses a preset bag-of-words model to segment the text to be recognized, generating multiple candidate words. The server then uses a preset n-gram model to represent these candidate words, generating model features for each candidate word. The server then inputs each candidate word's model features into the input layer of the improved machine learning model FastText to generate multiple segmentation vectors, each corresponding to a candidate word.

[0037] The bag-of-words method segments all data in a dataset into words and then judges sentences based on the frequency of each word. The n-gram model is a language model (LM), a probability-based discriminant model whose input is a sentence (a sequence of words) and whose output is the probability of the sentence, that is, the joint probability of the words.

[0038] For example, FastText uses character-level n-grams to represent a word. For "apple", assuming the value of n is 3, its trigrams are:<ap","app","ppl","ple","le> ", where < represents a prefix and > represents a suffix. We can use the superposition of these five trigram vectors to represent the word vector for "apple." For example, the bag-of-words model features for the Chinese sentence "I love you" are "I," "love," and "you." These features are the same as those for the sentence "You love me." If 2-grams are added, the features of the first sentence also include "I-love" and "love-you," making the two sentences "I love you" and "You love me" distinct.

[0039] It should be noted that in order to improve recognition efficiency, low-frequency n-grams need to be filtered.

[0040] 103. Call the hidden layer of the improved FastText to superimpose and average multiple word segmentation vectors to obtain the document vector.

[0041] The server uses the improved FastText hidden layer to stack and average multiple word segmentation vectors to obtain the document vector. Specifically, the server arranges the multiple word segmentation vectors in word segmentation order to obtain a word segmentation sequence. The server then inputs the word segmentation sequence into the improved FastText hidden layer in word segmentation order for averaging. The server then obtains the output of the improved FastText hidden layer to obtain the document vector.

[0042] 104. Call the cross entropy loss function BCELoss for binary classification to calculate each potential intent in the document vector and obtain the loss values ​​of multiple potential intents.

[0043] The server uses the cross-entropy loss function (BCELoss) for binary classification to calculate each potential intent in the document vector, obtaining multiple loss values ​​for the potential intent. Specifically, the server inputs the document vector into the improved FastText for negative sampling, obtaining multiple sampling vectors. The server then uses a hierarchical classifier to import the multiple sampling vectors into a tree structure, obtaining a tree classification structure that includes multiple potential intent labels. The server then uses the S-type threshold function (Sigmoid) and the cross-entropy loss function (BCELoss) for binary classification to calculate the multiple potential intent labels, obtaining multiple loss values ​​for the potential intent.

[0044] 105. Filter the loss values ​​of multiple potential intents according to the loss value corresponding to each potential intent and a preset threshold, and determine multiple candidate intent labels for the text to be identified based on the filtered loss values.

[0045] The server filters the loss values ​​of multiple potential intents based on the corresponding loss value and a pre-set threshold. Based on these loss values, it determines multiple candidate intent labels for the text to be recognized. The server uses the binary cross entropy loss (BCELOSS) function to individually calculate the impact of each intent on the prediction. This results in a probability distribution with larger values ​​for some intents and smaller values ​​for others, improving the model's ability to recognize multiple intents.

[0046] An embodiment of the present invention directly calculates the impact of each label in the text on the loss function, adjusts the probability distribution of the text in all intents, and takes the losses at all label positions into account, thereby increasing the confidence of the label position and reducing the confidence of the remaining positions, and improving the model's ability to recognize multiple intents.

[0047] See also Figure 2 , another flowchart of the intent recognition method based on the loss function provided by an embodiment of the present invention specifically includes:

[0048] 201. Obtain text to be recognized, where the text to be recognized is used to indicate at least one intention of a target user.

[0049] The server obtains the text to be recognized, where the text to be recognized is used to indicate at least one intention of the target user.

[0050] It should be noted that different texts to be recognized may or may not correspond to the same intentions. Different texts to be recognized may express the same one or more intentions. For example, "Can I buy Ping An Fu?" and "Can I buy Ping An Fu products?" express the same intention; the same text to be recognized may express different intentions. For example, "I want to buy Ping An Fu" and "I want to buy Ping An Fu?" express different intentions.

[0051] It is understandable that the execution subject of the present invention can be an intention recognition device based on a loss function, or a terminal or a server, which is not limited here. The embodiment of the present invention is described by taking a server as the execution subject as an example.

[0052] 202. Call the input layer of the improved machine learning model FastText to segment the text to be recognized and obtain multiple segmentation vectors.

[0053] The server uses the input layer of the improved machine learning model FastText to segment the text to be recognized, generating multiple segmentation vectors. Specifically, the server uses a preset bag-of-words model to segment the text to be recognized, generating multiple candidate words. The server then uses a preset n-gram model to represent these candidate words, generating model features for each candidate word. The server then inputs each candidate word's model features into the input layer of the improved machine learning model FastText to generate multiple segmentation vectors, each corresponding to a candidate word.

[0054] The bag-of-words method segments all data in a dataset into words and then judges sentences based on the frequency of each word. The n-gram model is a language model (LM), a probability-based discriminant model whose input is a sentence (a sequence of words) and whose output is the probability of the sentence, that is, the joint probability of the words.

[0055] For example, FastText uses character-level n-grams to represent a word. For "apple", assuming the value of n is 3, its trigrams are:<ap","app","ppl","ple","le> ", where < represents a prefix and > represents a suffix. We can use the superposition of these five trigram vectors to represent the word vector for "apple." For example, the bag-of-words model features for the Chinese sentence "I love you" are "I," "love," and "you." These features are the same as those for the sentence "You love me." If 2-grams are added, the features of the first sentence also include "I-love" and "love-you," making the two sentences "I love you" and "You love me" distinct.

[0056] It should be noted that in order to improve recognition efficiency, low-frequency n-grams need to be filtered.

[0057] 203. Call the hidden layer of the improved FastText to superimpose and average multiple word segmentation vectors to obtain the document vector.

[0058] The server uses the improved FastText hidden layer to stack and average multiple word segmentation vectors to obtain the document vector. Specifically, the server arranges the multiple word segmentation vectors in word segmentation order to obtain a word segmentation sequence. The server then inputs the word segmentation sequence into the improved FastText hidden layer in word segmentation order for averaging. The server then obtains the output of the improved FastText hidden layer to obtain the document vector.

[0059] 204. Input the document vector into the improved FastText for negative sampling to obtain multiple sampling vectors.

[0060] The server inputs the document vector into the improved FastText for negative sampling, generating multiple sample vectors. Negative sampling is designed to accelerate the text classification process. Its primary solution is to predict whether the context of a given word pair is the target word. For example, orange-juice is the target word, labeled 1, and is called a positive sample; orange-king is not the target word and is called a negative sample. The number of words selected for negative samples is reduced. For large sample sets, only 2-5 negative samples are used for each positive sample trained; for small sample sets, only 5-20 negative samples are used, improving sampling efficiency.

[0061] 205. Call a hierarchical classifier to import multiple sampling vectors into a tree structure to obtain a tree classification structure, which includes multiple potential intent labels.

[0062] The server calls a hierarchical classifier to import multiple sampling vectors into a tree structure to obtain a tree classification structure, which includes multiple potential intent labels.

[0063] Among them, in the standard Softmax classifier, when calculating the Softmax probability of a category, it is necessary to normalize the probabilities of all categories, which is very time-consuming when the category is large. Therefore, a hierarchical classifier HierarchicalSoftmax is proposed. A Huffman tree is constructed according to the frequency of the category to replace the standard Softmax. Hierarchical Softmax can reduce the complexity from N to logN.

[0064] 206. Multiple potential intent labels are calculated using the S-type threshold function Sigmoid and the cross entropy loss function BCELoss for binary classification to obtain loss values ​​for multiple potential intents.

[0065] The server calculates multiple potential intent labels using the S-type threshold function Sigmoid and the cross entropy loss function BCELoss for binary classification to obtain the loss values ​​of multiple potential intents. Specifically, the server obtains the calculated probability values ​​of multiple potential intent labels using the threshold function Sigmoid. The server calculates the loss values ​​of multiple potential intent labels based on the acquired true probability value y and the preset BCELoss formula, where the preset BCELoss formula is: .

[0066] For example, the original single-class loss value of the FastText model only calculates the loss caused by the class where the label is located, and does not calculate the loss value of other positions. Softmax normalizes the confidence of all classes and adds them up to 1, making the model unable to calculate the loss value of multi-label data. The improved loss calculation method of the present invention is: ;

[0067] = [0,0,1,0,1,0]

[0068] = [0.01,0.19,0.72, 0.69, 0.15, 0.03]

[0069] Here, both the loss value of label 1 and the loss value of label 0 are considered, and the total loss value is the sum and average of them. In this way, after training, the confidence of all positions marked with 1 is as large as possible, and the confidence of all positions marked with 0 is as small as possible.

[0070] It should be noted that the sum of the loss function values ​​generated for each potential intent is not 1. For multiple intents with high confidence, the scores of other intents will be very small. Generally, a threshold of 0.1 is sufficient to distinguish the recognized intents.

[0071] 207. Filter the loss values ​​of multiple potential intents according to the loss value corresponding to each potential intent and a preset threshold, and determine multiple candidate intent labels for the text to be recognized based on the filtered loss values.

[0072] The server filters the loss values ​​of multiple potential intents based on the loss value corresponding to each potential intent and a pre-set threshold, and determines multiple candidate intent labels for the text to be recognized based on the filtered loss values. Specifically, the server obtains the pre-set threshold; the server determines whether the loss value corresponding to each potential intent is greater than the threshold; if the target loss value is greater than the threshold, the server determines the potential intent corresponding to the target loss value as a candidate intent label for the text to be recognized, resulting in multiple candidate intent labels.

[0073] 208. Label the text to be recognized according to multiple candidate intent labels.

[0074] The server labels the text to be recognized based on multiple candidate intent labels.

[0075] An embodiment of the present invention directly calculates the impact of each label in the text on the loss function, adjusts the probability distribution of the text in all intents, and takes the losses at all label positions into account, thereby increasing the confidence of the label position and reducing the confidence of the remaining positions, and improving the model's ability to recognize multiple intents.

[0076] The above describes the intention recognition method based on the loss function in the embodiment of the present invention. The following describes the intention recognition device based on the loss function in the embodiment of the present invention. Figure 3 , an embodiment of the intention recognition device based on the loss function in the embodiment of the present invention includes:

[0077] An acquisition unit 301 is configured to acquire text to be recognized, where the text to be recognized is used to indicate at least one intention of a target user;

[0078] A word segmentation unit 302 is configured to call the input layer of the improved machine learning model FastText to segment the text to be recognized and obtain multiple word segmentation vectors;

[0079] An averaging unit 303 is configured to call the hidden layer of the improved FastText to perform superposition and averaging on the multiple word segmentation vectors to obtain a document vector;

[0080] A calculation unit 304 is configured to call a cross entropy loss function BCELoss for binary classification to calculate each potential intent in the document vector to obtain loss values ​​for multiple potential intents;

[0081] The screening unit 305 is used to screen the loss values ​​of the multiple potential intents according to the loss value corresponding to each potential intent and a preset threshold, and determine multiple candidate intent labels of the text to be recognized based on the screened loss values.

[0082] An embodiment of the present invention directly calculates the impact of each label in the text on the loss function, adjusts the probability distribution of the text in all intents, and takes the losses at all label positions into account, thereby increasing the confidence of the label position and reducing the confidence of the remaining positions, and improving the model's ability to recognize multiple intents.

[0083] See also Figure 4 Another embodiment of the intention recognition device based on the loss function in the embodiment of the present invention includes:

[0084] An acquisition unit 301 is configured to acquire text to be recognized, where the text to be recognized is used to indicate at least one intention of a target user;

[0085] A word segmentation unit 302 is configured to call the input layer of the improved machine learning model FastText to segment the text to be recognized and obtain multiple word segmentation vectors;

[0086] An averaging unit 303 is configured to call the hidden layer of the improved FastText to perform superposition and averaging on the multiple word segmentation vectors to obtain a document vector;

[0087] A calculation unit 304 is configured to call a cross entropy loss function BCELoss for binary classification to calculate each potential intent in the document vector to obtain loss values ​​for multiple potential intents;

[0088] The screening unit 305 is used to screen the loss values ​​of the multiple potential intents according to the loss value corresponding to each potential intent and a preset threshold, and determine multiple candidate intent labels of the text to be recognized based on the screened loss values.

[0089] Optionally, the calculation unit 304 includes:

[0090] A negative sampling module 3041 is used to input the document vector into the improved FastText for negative sampling to obtain multiple sampling vectors;

[0091] An import module 3042 is configured to call a hierarchical classifier to import the plurality of sampling vectors into a tree structure to obtain a tree classification structure, wherein the tree classification structure includes a plurality of potential intent labels;

[0092] The calculation module 3043 is used to calculate the multiple potential intent labels using the S-type threshold function Sigmoid and the cross entropy loss function BCELoss for binary classification to obtain loss values ​​of the multiple potential intents.

[0093] Optionally, the calculation module 3043 is specifically configured to:

[0094] The calculated probability values ​​of multiple potential intent labels are obtained through the threshold function Sigmoid ; Calculate the loss values ​​of multiple potential intent labels based on the obtained true probability value y and the preset BCELoss formula, where the preset BCELoss formula is

[0095] Optionally, the screening unit 305 is specifically configured to:

[0096] Obtain a preset threshold; determine whether the loss value corresponding to each potential intent is greater than the threshold; if the target loss value is greater than the threshold, determine the potential intent corresponding to the target loss value as the candidate intent label of the text to be identified, and obtain multiple candidate intent labels.

[0097] Optionally, the word segmentation unit 302 is specifically used to:

[0098] The text to be recognized is segmented using a preset word bag to obtain multiple candidate words; a preset n-gram model is called to characterize the multiple candidate words to obtain model features of each candidate word; the model features of each candidate word are input into the input layer of the improved machine learning model FastText to generate multiple segmentation vectors, each segmentation vector corresponding to a candidate word.

[0099] Optionally, the averaging unit 303 is specifically configured to:

[0100] Arrange multiple word segmentation vectors in word segmentation order to obtain a word segmentation sequence; input the word segmentation sequence into the hidden layer of the improved FastText in sequence according to the word segmentation order for averaging processing; obtain the output result of the hidden layer of the improved FastText to obtain a document vector.

[0101] Optionally, the loss function-based intention recognition device further includes:

[0102] The tagging unit 306 is configured to tag the text to be recognized according to a plurality of candidate intent labels.

[0103] An embodiment of the present invention directly calculates the impact of each label in the text on the loss function, adjusts the probability distribution of the text in all intents, and takes the losses at all label positions into account, thereby increasing the confidence of the label position and reducing the confidence of the remaining positions, and improving the model's ability to recognize multiple intents.

[0104] above Figures 3 and 4 The intention recognition device based on loss function in the embodiment of the present invention is described in detail from the perspective of modular functional entities, and the intention recognition device based on loss function in the embodiment of the present invention is described in detail from the perspective of hardware processing.

[0105] Figure 5This is a schematic diagram of the structure of a loss function-based intent recognition device provided by an embodiment of the present invention. This loss function-based intent recognition device 500 may vary significantly depending on configuration or performance. It may include one or more processors (central processing units, CPUs) 501 (e.g., one or more processors), memory 509, and one or more storage media 508 (e.g., one or more mass storage devices) for storing application programs 507 or data 506. The memory 509 and storage medium 508 may be either transient or persistent storage. The program stored in the storage medium 508 may include one or more modules (not shown), each of which may include a series of instruction operations within the loss function-based intent recognition device. Furthermore, the processor 501 may be configured to communicate with the storage medium 508 to execute the series of instruction operations stored in the storage medium 508 on the loss function-based intent recognition device 500.

[0106] The loss function-based intention recognition device 500 may further include one or more power supplies 502, one or more wired or wireless network interfaces 503, one or more input and output interfaces 504, and / or one or more operating systems 505, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, etc. It will be understood by those skilled in the art that Figure 5 The structure of the loss function-based intention recognition device shown in the figure does not constitute a limitation of the loss function-based intention recognition device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0107] The following combination Figure 5 This section describes in detail the components of the loss function-based intent recognition device:

[0108] The processor 501 is the control center of the loss function-based intention recognition device, and can process according to the set loss function-based intention recognition method. The processor 501 uses various interfaces and lines to connect the various parts of the entire loss function-based intention recognition device, and executes various functions and processes data of the loss function-based intention recognition device by running or executing software programs and / or modules stored in the memory 509, and calling data stored in the memory 509, thereby improving the confidence of the location of the label and reducing the confidence of other locations, and improving the model's recognition ability for multiple intentions. The storage medium 508 and the memory 509 are both carriers for storing data. In an embodiment of the present invention, the storage medium 508 can refer to an internal memory with a small storage capacity but a fast speed, and the memory 509 can be an external memory with a large storage capacity but a slow storage speed.

[0109] Memory 509 can be used to store software programs and modules. Processor 501 executes the various functional applications and data processing of loss function-based intent recognition device 500 by running the software programs and modules stored in memory 509. Memory 509 may primarily include a program storage area and a data storage area. The program storage area may store an operating system and at least one application required for a function (such as calling the hidden layer of the improved FastText to superimpose and average multiple word segmentation vectors to obtain a document vector). The data storage area may store data generated by the use of the loss function-based intent recognition device (such as loss values ​​for multiple potential intents). Memory 509 may also include high-speed random access memory and non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. The loss function-based intent recognition method program and received data stream provided in this embodiment of the present invention are stored in memory. When needed, processor 501 calls them from memory 509.

[0110] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that the technical solutions described in the above embodiments can still be modified, or some of the technical features thereof can be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A loss function-based intent recognition method, characterized in that: include: Acquire a text to be recognized, where the text to be recognized is used to indicate at least one intention of a target user; Segmenting the text to be recognized using a preset bag of words to obtain a plurality of candidate words; calling a preset n-gram model to characterize the plurality of candidate words to obtain a model feature for each candidate word; Inputting the model features of each candidate word into the input layer of the improved machine learning model FastText to segment the text to be recognized, thereby obtaining multiple segmentation vectors; each segmentation vector corresponds to a candidate word; Calling the hidden layer of the improved machine learning model FastText to perform superposition and averaging on the multiple word segmentation vectors to obtain a document vector; Call the cross entropy loss function BCELoss for binary classification to calculate each potential intent in the document vector to obtain loss values ​​of multiple potential intents; Filtering the loss values ​​of the multiple potential intents according to the loss value corresponding to each potential intent and a preset threshold, and determining multiple candidate intent labels for the to-be-recognized text based on the filtered loss values; The cross entropy loss function BCELoss for binary classification is called to calculate each potential intent in the document vector to obtain loss values ​​of multiple potential intents, including: Input the document vector into the improved machine learning model FastText for negative sampling to obtain multiple sampling vectors; Calling a hierarchical classifier to import the plurality of sampling vectors into a tree structure to obtain a tree classification structure, wherein the tree classification structure includes a plurality of potential intent labels; The multiple potential intent labels are calculated using the S-type threshold function Sigmoid and the cross entropy loss function BCELoss for binary classification to obtain loss values ​​of the multiple potential intents.

2. The intention recognition method based on loss function according to claim 1, characterized in that The multiple potential intent labels are calculated using the S-type threshold function Sigmoid and the cross entropy loss function BCELoss for binary classification to obtain loss values ​​of multiple potential intents, including: The calculated probability values ​​of multiple potential intent labels are obtained through the threshold function Sigmoid ; Calculate the loss value of multiple potential intent labels based on the obtained true probability value y and the preset BCELoss formula l , where the preset BCELoss formula is .

3. The intention recognition method based on loss function according to claim 1, characterized in that The step of screening the loss values ​​of the multiple potential intents according to the loss value corresponding to each potential intent and a preset threshold, and determining multiple candidate intent labels of the to-be-recognized text based on the screened loss values, includes: Get the preset threshold; Determine whether the loss value corresponding to each potential intention is greater than the threshold; If the target loss value is greater than the threshold, the potential intent corresponding to the target loss value is determined as the candidate intent label of the text to be recognized, and multiple candidate intent labels are obtained.

4. The intention recognition method based on loss function according to claim 1, characterized in that The model features of each candidate word are input into the input layer of the improved machine learning model FastText to segment the text to be recognized, and multiple segmentation vectors are obtained, including: Segmenting the text to be recognized using a preset bag of words to obtain multiple candidate words; Calling a preset n-gram model to characterize the multiple candidate words to obtain a model feature of each candidate word; The model features of each candidate word are input into the input layer of the improved machine learning model FastText to generate multiple word segmentation vectors, each of which corresponds to a candidate word.

5. The intention recognition method based on loss function according to claim 1, characterized in that The calling of the hidden layer of the improved machine learning model FastText to perform superposition and averaging on the multiple word segmentation vectors to obtain a document vector includes: Arrange multiple word segmentation vectors in the order of word segmentation to obtain a word segmentation sequence; Inputting the word segmentation sequence into the hidden layer of the improved machine learning model FastText in sequence according to the word segmentation order for averaging processing; The output result of the hidden layer of the improved machine learning model FastText is obtained to obtain a document vector.

6. The intention recognition method based on loss function according to any one of claims 1 to 5, characterized in that: After screening the loss values ​​of the multiple potential intents according to the loss value corresponding to each potential intent and a preset threshold, and determining multiple candidate intent labels for the to-be-recognized text based on the screened loss values, the method further includes: The text to be recognized is labeled according to multiple candidate intent labels.

7. An intention recognition device based on a loss function, characterized in that: include: an acquisition unit, configured to acquire text to be recognized, where the text to be recognized is used to indicate at least one intention of a target user; A word segmentation unit is used to segment the text to be recognized using a preset word bag to obtain multiple candidate words; a preset n-gram model is used to characterize the multiple candidate words to obtain a model feature of each candidate word; Inputting the model features of each candidate word into the input layer of the improved machine learning model FastText to segment the text to be recognized, thereby obtaining multiple segmentation vectors; each segmentation vector corresponds to a candidate word; An averaging unit, configured to call a hidden layer of the improved machine learning model FastText to perform superposition and averaging on the multiple word segmentation vectors to obtain a document vector; A calculation unit, configured to call a cross entropy loss function BCELoss for binary classification to calculate each potential intent in the document vector to obtain loss values ​​of multiple potential intents; a screening unit, configured to screen the loss values ​​of the plurality of potential intents according to the loss value corresponding to each potential intent and a preset threshold, and determine a plurality of candidate intent labels for the to-be-recognized text based on the screened loss values; The computing unit is also used to input the document vector into the improved machine learning model FastText for negative sampling to obtain multiple sampling vectors; Calling a hierarchical classifier to import the plurality of sampling vectors into a tree structure to obtain a tree classification structure, wherein the tree classification structure includes a plurality of potential intent labels; The multiple potential intent labels are calculated using the S-type threshold function Sigmoid and the cross entropy loss function BCELoss for binary classification to obtain loss values ​​of the multiple potential intents.

8. An intention recognition device based on a loss function, characterized in that: It includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the loss function-based intention recognition method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the loss function-based intention recognition method as described in any one of claims 1 to 6.

Images